Sigmoid function

Many natural processes, including those of complex system learning curve

Learning curve

A learning curve is a graphical representation of the changing rate of learning for a given activity or tool. Typically, the increase in retention of information is sharpest after the initial attempts, and then gradually evens out, meaning that less and less new information is retained after each...

s, exhibit a progression from small beginnings that accelerates and approaches a climax over time. When a detailed description is lacking, a sigmoid function is often used. A sigmoid curve is produced by a mathematical function

Function (mathematics)

In mathematics, a function associates one quantity, the argument of the function, also known as the input, with another quantity, the value of the function, also known as the output. A function assigns exactly one output to each input. The argument and the value may be real numbers, but they can...

 having an "S" shape. Often, sigmoid function refers to the special case of the logistic function

Logistic function

A logistic function or logistic curve is a common sigmoid curve, given its name in 1844 or 1845 by Pierre François Verhulst who studied it in relation to population growth. It can model the "S-shaped" curve of growth of some population P...

 shown at right and defined by the formula
Another example is the Gompertz curve

Gompertz curve

A Gompertz curve or Gompertz function, named after Benjamin Gompertz, is a sigmoid function. It is a type of mathematical model for a time series, where growth is slowest at the start and end of a time period...

. It is used in modeling systems that saturate at large values of t.
Another example is the ogee curve as used in the spillway

Spillway

A spillway is a structure used to provide the controlled release of flows from a dam or levee into a downstream area, typically being the river that was dammed. In the UK they may be known as overflow channels. Spillways release floods so that the water does not overtop and damage or even destroy...

 of some dam

Dam

A dam is a barrier that impounds water or underground streams. Dams generally serve the primary purpose of retaining water, while other structures such as floodgates or levees are used to manage or prevent water flow into specific land regions. Hydropower and pumped-storage hydroelectricity are...

s.
A wide variety of sigmoid functions have been used as the activation function

Activation function

In computational networks, the activation function of a node defines the output of that node given an input or set of inputs. A standard computer chip circuit can be seen as a digital network of activation functions that can be "ON" or "OFF" , depending on input. This is similar to the behavior of...

 of artificial neuron

Artificial neuron

An artificial neuron is a mathematical function conceived as a crude model, or abstraction of biological neurons. Artificial neurons are the constitutive units in an artificial neural network...

s, including the logistic function and tanh(x).

Properties

In general, a sigmoid function is real

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

-valued and differentiable, having either a non-negative or non-positive first derivative

Derivative

In calculus, a branch of mathematics, the derivative is a measure of how a function changes as its input changes. Loosely speaking, a derivative can be thought of as how much one quantity is changing in response to changes in some other quantity; for example, the derivative of the position of a...

 which is bell shaped. There are also a pair of horizontal asymptotes as . The logistic function

Logistic function

A logistic function or logistic curve is a common sigmoid curve, given its name in 1844 or 1845 by Pierre François Verhulst who studied it in relation to population growth. It can model the "S-shaped" curve of growth of some population P...

s are sigmoidal and are characterized as the solutions of the differential equation
When the graph of growth of plant is plotted against time the graph obtained is Sigmoid function.

Examples

Besides the logistic function

Logistic function

A logistic function or logistic curve is a common sigmoid curve, given its name in 1844 or 1845 by Pierre François Verhulst who studied it in relation to population growth. It can model the "S-shaped" curve of growth of some population P...

, sigmoid functions include the ordinary arctangent

Inverse trigonometric function

In mathematics, the inverse trigonometric functions are the inverse functions of the trigonometric functions with suitably restricted domains .The notations sin−1, cos−1, etc...

, the hyperbolic tangent

Hyperbolic function

In mathematics, hyperbolic functions are analogs of the ordinary trigonometric, or circular, functions. The basic hyperbolic functions are the hyperbolic sine "sinh" , and the hyperbolic cosine "cosh" , from which are derived the hyperbolic tangent "tanh" and so on.Just as the points form a...

, and the error function

Error function

In mathematics, the error function is a special function of sigmoid shape which occurs in probability, statistics and partial differential equations...

, but also the generalised logistic function and algebraic functions

Algebraic function

In mathematics, an algebraic function is informally a function that satisfies a polynomial equation whose coefficients are themselves polynomials with rational coefficients. For example, an algebraic function in one variable x is a solution y for an equationwhere the coefficients ai are polynomial...

 like .

The integral

Integral

Integration is an important concept in mathematics and, together with its inverse, differentiation, is one of the two main operations in calculus...

 of any smooth, positive, "bump-shaped" function will be sigmoidal, thus the cumulative distribution function

Cumulative distribution function

In probability theory and statistics, the cumulative distribution function , or just distribution function, describes the probability that a real-valued random variable X with a given probability distribution will be found at a value less than or equal to x. Intuitively, it is the "area so far"...

s for many common probability distribution

Probability distribution

In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....

s are sigmoidal. The most famous such example is the error function

Error function

In mathematics, the error function is a special function of sigmoid shape which occurs in probability, statistics and partial differential equations...

.

See also

• Logistic function
  Logistic function
  A logistic function or logistic curve is a common sigmoid curve, given its name in 1844 or 1845 by Pierre François Verhulst who studied it in relation to population growth. It can model the "S-shaped" curve of growth of some population P...
  
  
• Logistic distribution
• Logistic regression
  Logistic regression
  In statistics, logistic regression is used for prediction of the probability of occurrence of an event by fitting data to a logit function logistic curve. It is a generalized linear model used for binomial regression...
  
  
• Logit
  Logit
  The logit function is the inverse of the sigmoidal "logistic" function used in mathematics, especially in statistics.Log-odds and logit are synonyms.-Definition:The logit of a number p between 0 and 1 is given by the formula:...
  
  
• Hyperbolic function
  Hyperbolic function
  In mathematics, hyperbolic functions are analogs of the ordinary trigonometric, or circular, functions. The basic hyperbolic functions are the hyperbolic sine "sinh" , and the hyperbolic cosine "cosh" , from which are derived the hyperbolic tangent "tanh" and so on.Just as the points form a...
  
  
• Weibull distribution