Home      Discussion      Topics      Dictionary      Almanac
Signup       Login
Complexity

Complexity

Discussion
Ask a question about 'Complexity'
Start a new discussion about 'Complexity'
Answer questions from other users
Full Discussion Forum
 
Encyclopedia
In general usage, complexity tends to be used to characterize something with many parts in intricate arrangement. The study of these complex linkages is the main goal of network theory
Network theory
Network theory is an area of applied mathematics and network science and part of graph theory. It has application in many disciplines including particle physics, computer science, biology, economics, operations research, and sociology...

 and network science
Network science
Network science is a new and emerging scientific discipline that examines the interconnections among diverse physical or engineered networks, information networks, biological networks, cognitive and semantic networks, and social networks. This field of science seeks to discover common principles,...

. In science
Science
Science is in its broadest sense to any systematic knowledge-base or prescriptive practice that is capable of resulting in a prediction or predictable type of outcome...

 there are at this time a number of approaches to characterizing complexity, many of which are reflected in this article.

Definitions are often tied to the concept of a ‘system
System
System is a set of interacting or interdependent entities forming an integrated whole....

’ – a set of parts or elements which have relationships among them differentiated from relationships with other elements outside the relational regime. Many definitions tend to postulate or assume that complexity expresses a condition of numerous elements in a system and numerous forms of relationships among the elements. At the same time, what is complex and what is simple is relative and changes with time.

Some definitions key on the question of the probability of encountering a given condition of a system once characteristics of the system are specified. Warren Weaver
Warren Weaver
Warren Weaver was an American scientist, mathematician, and science administrator...

 has posited that the complexity of a particular system
System
System is a set of interacting or interdependent entities forming an integrated whole....

 is the degree of difficulty in predicting the properties of the system if the properties of the system’s parts are given. In Weaver's view, complexity comes in two forms: disorganized complexity, and organized complexity.

Weaver’s
Warren Weaver
Warren Weaver was an American scientist, mathematician, and science administrator...

 paper has influenced contemporary thinking about complexity.
The approaches which embody concepts of systems, multiple elements, multiple relational regimes, and state spaces might be summarized as implying that complexity arises from the number of distinguishable relational regimes (and their associated state spaces) in a defined system.

Some definitions relate to the algorithmic basis for the expression of a complex phenomenon or model or mathematical expression, as is later set out herein.



Disorganized complexity vs. organized complexity


One of the problems in addressing complexity issues has been distinguishing conceptually between the large number of variances in relationships extant in random collections, and the sometimes large, but smaller, number of relationships between elements in systems where constraints (related to correlation of otherwise independent elements) simultaneously reduce the variations from element independence and create distinguishable regimes of more-uniform, or correlated, relationships, or interactions.

Weaver perceived and addressed this problem, in at least a preliminary way, in drawing a distinction between 'disorganized complexity' and 'organized complexity'.

In Weaver's view, disorganized complexity results from the particular system having a very large number of parts, say millions of parts, or many more. Though the interactions of the parts in a 'disorganized complexity' situation can be seen as largely random, the properties of the system as a whole can be understood by using probability and statistical methods.

A prime example of disorganized complexity is a gas in a container, with the gas molecules as the parts. Some would suggest that a system of disorganized complexity may be compared, for example, with the (relative) simplicity
Simplicity
Simplicity is being simple. It is a property, condition, or quality which things can be judged to have. It usually relates to the burden which a thing puts on someone trying to explain or understand it. Something which is easy to understand or explain is simple, in contrast to something...

 of the planetary orbits – the latter can be known by applying Newton’s laws of motion, though this example involved highly correlated events.

Organized complexity, in Weaver's view, resides in nothing else than the non-random, or correlated, interaction between the parts. These non-random, or correlated, relationships create a differentiated structure which can, as a system, interact with other systems. The coordinated system manifests properties not carried by, or dictated by, individual parts. The organized aspect of this form of complexity vis a vis other systems than the subject system can be said to "emerge,"
Emergence
In philosophy, systems theory and science, emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions. Emergence is central to the theories of integrative levels and of complex systems.-Definitions:...

 without any “guiding hand.”

The number of parts does not have to be very large for a particular system to have emergent properties. A system of organized complexity may be understood in its properties (behavior among the properties) through modeling and simulation
Simulation
Simulation is the imitation of some real thing, state of affairs, or process. The act of simulating something generally entails representing certain key characteristics or behaviours of a selected physical or abstract system....

, particularly modeling and simulation with computers
Computer simulation
A computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...

. An example of organized complexity is a city neighborhood as a living mechanism, with the neighborhood people among the system’s parts.

Sources and factors of complexity


The source of disorganized complexity is the large number of parts in the system of interest, and the lack of correlation between elements in the system.

There is no consensus at present on general rules regarding the sources of organized complexity, though the lack of randomness implies correlations between elements. See e.g. Robert Ulanowicz
Robert Ulanowicz
Robert Edward Ulanowicz is an American theoretical ecologist and philosopher who is best known for his search for a unified theory of ecology. He was born September 17, 1943 in Baltimore, Maryland....

's treatment of ecosystems. Consistent with prior statements here, the number of parts (and types of parts) in the system and the number of relations between the parts would have to be non-trivial – however, there is no general rule to separate “trivial” from “non-trivial.

Complexity of an object or system is a relative property. For instance, for many functions (problems), such a computational complexity as time of computation is smaller when multitape Turing machine
Turing machine
A Turing machine is a theoretical device that manipulates symbols contained on a strip of tape. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside of a computer. The Turing...

s are used than when Turing machine
Turing machine
A Turing machine is a theoretical device that manipulates symbols contained on a strip of tape. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside of a computer. The Turing...

s with one tape are used. Random Access Machine
Random access machine
In computer science, random access machine is an abstract machine in the general class of register machines. The RAM is very similar to the counter machine but with the added capability of 'indirect addressing' of its registers...

s allow one to even more decrease time complexity (Greenlaw and Hoover 1998: 226), while inductive Turing machines can decrease even the complexity class of a function, language or set (Burgin 2005). This shows that tools of activity can be an important factor of complexity.

Specific meanings of complexity


In several scientific fields, "complexity" has a specific meaning :
  • In computational complexity theory
    Computational complexity theory
    Computational complexity theory is a branch of the theory of computation in computer science that focuses on classifying computational problems according to their inherent difficulty. In this context, a computational problem is understood to be a task that is in principle amenable to being solved...

    , the amounts of resources
    Computational resource
    In computational complexity theory, a computational resource is a resource used by some computational models in the solution of computational problems....

     required for the execution of algorithm
    Algorithm
    In mathematics, computing, linguistics, and related subjects, an algorithm is an effective method for solving a problem using a finite sequence of instructions. Algorithms are used for calculation, data processing, and many other fields....

    s is studied. The most popular types of computational complexity are the time complexity of a problem equal to the number of steps that it takes to solve an instance of the problem as a function of the size of the input (usually measured in bits), using the most efficient algorithm
    Algorithm
    In mathematics, computing, linguistics, and related subjects, an algorithm is an effective method for solving a problem using a finite sequence of instructions. Algorithms are used for calculation, data processing, and many other fields....

    , and the space complexity of a problem equal to the volume of the memory
    Computer storage
    Computer data storage, often called storage or memory, refers to computer components, devices, and recording media that retain digital data used for computing for some interval of time. Computer data storage provides one of the core functions of the modern computer, that of information retention...

     used by the algorithm (e.g., cells of the tape) that it takes to solve an instance of the problem as a function of the size of the input (usually measured in bits), using the most efficient algorithm
    Algorithm
    In mathematics, computing, linguistics, and related subjects, an algorithm is an effective method for solving a problem using a finite sequence of instructions. Algorithms are used for calculation, data processing, and many other fields....

    . This allows to classify computational problems by complexity class
    Complexity class
    In computational complexity theory, a complexity class is a set of problems of related complexity. A typical complexity class has a definition of the form:...

     (such as P
    P (complexity)
    In computational complexity theory, P, also known as PTIME or DTIME, is one of the most fundamental complexity classes...

    , NP
    NP (complexity)
    In computational complexity theory, NP is one of the most fundamental complexity classes.The abbreviation NP refers to "nondeterministic polynomial time"....

     ... ). An axiomatic approach to computational complexity
    Computational complexity theory
    Computational complexity theory is a branch of the theory of computation in computer science that focuses on classifying computational problems according to their inherent difficulty. In this context, a computational problem is understood to be a task that is in principle amenable to being solved...

     was developed by Manuel Blum
    Manuel Blum
    Manuel Blum is a computer scientist who received the Turing Award in 1995 "In recognition of his contributions to the foundations of computational complexity theory and its application to cryptography and program checking".-Biography:Blum attended MIT, where he received his bachelor's degree and...

    . It allows one to deduce many properties of concrete computational complexity measures, such as time complexity or space complexity, from properties of axiomatically defined measures.

  • In algorithmic information theory
    Algorithmic information theory
    Algorithmic information theory is a subfield of information theory and computer science that concerns itself with the relationship between computation and information...

    , the Kolmogorov complexity
    Kolmogorov complexity
    In algorithmic information theory , the Kolmogorov complexity of an object such as a piece of text is a measure of the computational resources needed to specify the object...

    (also called descriptive complexity, algorithmic complexity or algorithmic entropy) of a string
    String (computer science)
    In mathematics, a string is an sequence of symbols that are chosen from a set or alphabet.In computer programming, a string is, essentially, a sequence of characters...

     is the length of the shortest binary program
    Computer program
    Computer programs are instructions for a computer. A computer requires programs to function, typically executing the program's instructions in a central processor. The program has an executable form that the computer can use directly to execute the instructions...

     which outputs that string. Different kinds of Kolmogorov complexity are studied: the uniform complexity, prefix complexity, monotone complexity, time-bounded Kolmogorov complexity, and space-bounded Kolmogorov complexity. An axiomatic approach to Kolmogorov complexity
    Kolmogorov complexity
    In algorithmic information theory , the Kolmogorov complexity of an object such as a piece of text is a measure of the computational resources needed to specify the object...

     based on Blum axioms
    Blum axioms
    In computational complexity theory the Blum axioms or Blum complexity axioms are axioms which specify desirable properties of complexity measures on the set of computable functions. The axioms were first defined by Manuel Blum in 1967....

     (Blum 1967) was introduced by Mark Burgin in the paper presented for publication by Andrey Kolmogorov
    Andrey Kolmogorov
    Andrey Nikolaevich Kolmogorov was a Soviet Russian mathematician, preeminent in the 20th century, who advanced various scientific fields .-Early life:Kolmogorov was born at Tambov in 1903...

     (Burgin 1982). The axiomatic approach encompasses other approaches to Kolmogorov complexity
    Kolmogorov complexity
    In algorithmic information theory , the Kolmogorov complexity of an object such as a piece of text is a measure of the computational resources needed to specify the object...

    . It is possible to treat different kinds of Kolmogorov complexity
    Kolmogorov complexity
    In algorithmic information theory , the Kolmogorov complexity of an object such as a piece of text is a measure of the computational resources needed to specify the object...

     as particular cases of axiomatically defined generalized Kolmogorov complexity
    Kolmogorov complexity
    In algorithmic information theory , the Kolmogorov complexity of an object such as a piece of text is a measure of the computational resources needed to specify the object...

    . Instead, of proving similar theorems, such as the basic invariance theorem, for each particular measure, it is possible to easily deduce all such results from one corresponding theorem proved in the axiomatic setting. This is a general advantage of the axiomatic approach in mathematics. The axiomatic approach to Kolmogorov complexity was further developed in the book (Burgin 2005) and applied to software metrics (Burgin and Debnath, 2003; Debnath and Burgin, 2003).

  • In information processing
    Information processing
    Information processing is the change of information in any manner detectable by an observer. As such, it is a process which describes everything which happens in the universe, from the falling of a rock to the printing of a text file from a digital computer system...

    , complexity is a measure of the total number of properties
    Property
    Property is any physical or intangible entity that is owned by a person or jointly by a group of persons. Depending on the nature of the property, an owner of property has the right to consume, sell, rent, mortgage, transfer, exchange or destroy his or her property, and/or to exclude others from...

     transmitted by an object and detected by an observer
    Observation
    Observation is either an activity of a living being , consisting of receiving knowledge of the outside world through the senses, or the recording of data using scientific instruments. The term may also refer to any datum collected during this activity.-Observation in science:A scientific method...

    . Such a collection of properties is often referred to as a state
    State (computer science)
    In computer science and automata theory, a state is a unique configuration of information in a program or machine. It is a concept that occasionally extends into some forms of systems programming such as lexers and parsers....

    .

  • In physical systems, complexity is a measure of the probability
    Probability
    Probability is a way of expressing knowledge or belief that an event will occur or has occurred. In mathematics the concept has been given an exact meaning in probability theory, that is used extensively in such areas of study as mathematics, statistics, finance, gambling, science, and philosophy...

     of the state vector
    State vector
    *A state vector in general control systems describes the observed states of an object in state space, e.g. in variables of the degrees of freedom for motion *A state vector in general control systems describes the observed states of an object in state space, e.g. in variables of the degrees of...

     of the system
    System
    System is a set of interacting or interdependent entities forming an integrated whole....

    . This should not be confused with entropy
    Entropy (disambiguation)
    Entropy, in thermodynamics, is a measure of the unavailability of a system’s energy to do work.Entropy may also refer to:-Thermodynamics:*Entropy , the macroscopic approach to thermodynamic entropy...

    ; it is a distinct mathematical measure, one in which two distinct states are never conflated and considered equal, as is done for the notion of entropy statistical mechanics
    Statistical mechanics
    Statistical mechanics is the application of probability theory, which includes mathematical tools for dealing with large populations, to the field of mechanics, which is concerned with the motion of particles or objects when subjected to a force...

    .

  • In mathematics
    Mathematics
    Mathematics is the science and study of quantity, structure, space, and change. Mathematicians seek out patterns, formulate new conjectures, and establish truth by rigorous deduction from appropriately chosen axioms and definitions....

    , Krohn-Rhodes complexity is an important topic in the study of finite semigroup
    Semigroup
    In mathematics, a semigroup is an algebraic structure consisting of a nonempty set S together with an associative binary operation. In other words, a semigroup is an associative magma. The terminology is derived from the anterior notion of a group...

    s and automata
    Automata theory
    In theoretical computer science, automata theory is the study of abstract machines and problems which they are able to solve. Automata theory is closely related to formal language theory as the automata are often classified by the class of formal languages they are able to recognize.An automaton is...

    .


There are different specific forms of complexity:
  • In the sense of how complicated a problem is from the perspective of the person trying to solve it, limits of complexity are measured using a term from cognitive psychology
    Cognitive psychology
    Cognitive psychology is a discipline within psychology that investigates the internal mental processes of thought such as visual processing, memory, problem solving, and language....

    , namely the hrair limit.

  • Unruly complexity denotes situations that do not have clearly defined boundaries, coherent internal dynamics, or simply mediated relations with their external context, as coined by Peter Taylor.

  • Complex adaptive system
    Complex adaptive system
    Complex adaptive systems are special cases of complex systems. They are complex in that they are diverse and made up of multiple interconnected elements and adaptive in that they have the capacity to change and learn from experience...

     denotes systems which have some or all of the following attributes
    • The number of parts (and types of parts) in the system and the number of relations between the parts is non-trivial – however, there is no general rule to separate “trivial” from “non-trivial;”
    • The system has memory or includes feedback
      Feedback
      Feedback describes the situation when output from an event or phenomenon in the past will influence the same event/phenomenon in the present or future....

      ;
    • The system can adapt itself according to its history or feedback;
    • The relations between the system and its environment are non-trivial or non-linear;
    • The system can be influenced by, or can adapt itself to, its environment; and
    • The system is highly sensitive to initial conditions.

Study of complexity


Complexity has always been a part of our environment, and therefore many scientific
Science
Science is in its broadest sense to any systematic knowledge-base or prescriptive practice that is capable of resulting in a prediction or predictable type of outcome...

 fields have dealt with complex systems and phenomena. Indeed, some would say that only what is somehow complex – what displays variation without being random
Randomness
Randomness is a concept with somewhat disparate meanings in several fields. It also has common meanings which may have loose connections with some of those more definite meanings...

 – is worthy of interest.

The use of the term complex is often confused with the term complicated. In today’s systems, this is the difference between myriad connecting "stovepipes" and effective "integrated" solutions. This means that complex is the opposite of independent, while complicated is the opposite of simple.

While this has led some fields to come up with specific definitions of complexity, there is a more recent movement to regroup observations from different fields
Interdisciplinarity
An interdisciplinary field or multidisciplinary field is a field of study that crosses traditional boundaries between academic disciplines or schools of thought, as new needs and professions have emerged....

 to study complexity in itself, whether it appears in anthills, human brain
Human brain
The human brain is the center of the human nervous system and is a highly complex organ. Enclosed in the cranium, it has the same general structure as the brains of other mammals, but is over three times as large as the brain of a typical mammal with an equivalent body size...

s, or stock market
Stock market
A stock market is a public market for the trading of company stock and derivatives at an agreed price; these are securities listed on a stock exchange as well as those only traded privately....

s. One such interndisciplinary group of fields is relational order theories
Relational order theories
A number of independent lines of research depict the universe, including the social organization of living creatures which is of particular interest to humans, as systems, or networks, of relationships....

.

Complex behaviour


The behaviour of a complex system is often said to be due to emergence
Emergence
In philosophy, systems theory and science, emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions. Emergence is central to the theories of integrative levels and of complex systems.-Definitions:...

 and self-organization
Self-organization
Self-organization is a process of attraction and repulsion in which the internal organization of a system, normally an open system, increases in complexity without being guided or managed by an outside source...

. Chaos theory
Chaos theory
Chaos theory is a branch of mathematics which studies the behavior of certain dynamical systems that may be highly sensitive to initial conditions. This sensitivity is popularly referred to as the butterfly effect. As a result of this sensitivity, which manifests itself as an exponential growth of...

 has investigated the sensitivity of systems to variations in initial conditions as one cause of complex behaviour.

Complex mechanisms


Recent developments around artificial life
Artificial life
Artificial life is a field of study and an associated art form which examine systems related to life, its processes, and its evolution through simulations using computer models, robotics, and biochemistry...

, evolutionary computation
Evolutionary computation
In computer science, evolutionary computation is a subfield of artificial intelligence that involves combinatorial optimization problems....

 and genetic algorithm
Genetic algorithm
A genetic algorithm is a search technique used in computing to find exact or approximate solutions to optimization and search problems. Genetic algorithms are categorized as global search heuristics...

s have led to an increasing emphasis on complexity and complex adaptive systems.

Complex simulations


In social science, the study on the emergence of macro-properties from the micro-properties, also known as macro-micro view in sociology
Sociology
Sociology is the scientific or systematic study of human societies. It is a branch of social science that uses various methods of empirical investigation and critical analysis to develop and refine a body of knowledge about human social structure and activity, often with the goal of applying such...

. The topic is commonly recognized as social complexity
Social complexity
Social complexity is the study of social phenomenon as complex systems. The social complexity can be seen as an impact on the social analysis of increasingly influential complexity theory...

 that is often related to the use of computer simulation
Computer simulation
A computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...

 in social science, i.e.: computational sociology
Computational sociology
Computational sociology is a recently developed branch of sociology that uses computation to analyze social phenomena. The basic premise of computational sociology is to take advantage of computer simulation in the construction of social theories...

.

Complex systems



Systems theory
Systems theory
Systems theory is an interdisciplinary theory about the nature of complex systems in nature, society, and science, and is a framework by which one can investigate and/or describe any group of objects that work together to produce some result. This could be a single organism, any organization or...

 has long been concerned with the study of complex system
Complex system
A complex system is a system composed of interconnected parts that as a whole exhibit one or more properties not obvious from the properties of the individual parts...

s (In recent times, complexity theory and complex systems have also been used as names of the field). These system
System
System is a set of interacting or interdependent entities forming an integrated whole....

s can be biological
Biology
Biology is the natural science concerned with the study of life and living organisms, including their structure, function, growth, origin, evolution, distribution, and taxonomy...

, economic, technological
Technology
Technology is a broad concept that deals with human as well as other animal species' usage and knowledge of tools and crafts, and how it affects a species' ability to control and adapt to its environment...

, etc. Recently, complexity is a natural domain of interest of the real world socio-cognitive systems and emerging systemics
Systemics
Systemics is an emerging branch of philosophy that studies systems based on holistic view points. It tries to develop logical, mathematical, engineering and philosophical paradigms and frameworks in which physical, technological, biological, social, cognitive and metaphysical systems can be studied...

 research. Complex systems tend to be high-dimension
Dimension
In mathematics and physics, the dimension of a space or object is informally defined as the minimum number of coordinates needed to specify each point within it. Thus a line has a dimension of one because only one coordinate is needed to specify a point on it...

al, non-linear and hard to model. In specific circumstances they may exhibit low dimensional behaviour.

Complexity in data


In information theory
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Historically, information theory was developed by Claude E. Shannon to find fundamental limits on compressing and reliably storing and communicating data...

, algorithmic information theory
Algorithmic information theory
Algorithmic information theory is a subfield of information theory and computer science that concerns itself with the relationship between computation and information...

 is concerned with the complexity of strings
String (computer science)
In mathematics, a string is an sequence of symbols that are chosen from a set or alphabet.In computer programming, a string is, essentially, a sequence of characters...

 of data.

Complex strings are harder to compress. While intuition tells us that this may depend on the codec
Codec
A codec is a device or computer program capable of encoding and/or decoding a digital data stream or signal. The word codec is a portmanteau of compressor-decompressor' or, more accurately, coder-decoder'.Historically a modem was a contraction of modulator/demodulator and converted...

 used to compress a string (a codec could be theoretically created in any arbitrary language, including one in which the very small command "X" could cause the computer to output a very complicated string like '18995316'"), any two Turing-complete
Turing completeness
In computability theory, several closely-related terms are used to describe the "computational power" of a computational system :Turing completenessTuring equivalence universality- Overview :...

 languages can be implemented in each other, meaning that the length of two encodings in different languages will vary by at most the length of the "translation" language - which will end up being negligible for sufficiently large data strings.

These algorithmic measures of complexity tend to assign high values to random noise
Signal noise
In science, and especially in physics and telecommunication, noise is fluctuations in and the addition of external factors to the stream of target information being received at a detector. In communications, it may be deliberate as for instance jamming of a radio or TV signal, but in most cases it...

. However, those studying complex systems would not consider randomness
Randomness
Randomness is a concept with somewhat disparate meanings in several fields. It also has common meanings which may have loose connections with some of those more definite meanings...

 as complexity.

Information entropy
Information entropy
In information theory, entropy is a measure of the uncertainty associated with a random variable. The term by itself in this context usually refers to the Shannon entropy, which quantifies, in the sense of an expected value, the information contained in a message, usually in units such as bits...

 is also sometimes used in information theory as indicative of complexity.

Applications of complexity


Computational complexity theory
Computational complexity theory
Computational complexity theory is a branch of the theory of computation in computer science that focuses on classifying computational problems according to their inherent difficulty. In this context, a computational problem is understood to be a task that is in principle amenable to being solved...

 is the study of the complexity of problems - that is, the difficulty of solving
Problem solving
Problem solving is a mental process and is part of the larger problem process that includes problem finding and problem shaping. Consideredthe most complex of all intellectual functions, problem solving has been defined as higher-order cognitive process that requires the modulation and control of...

 them. Problems can be classified by complexity class
Complexity class
In computational complexity theory, a complexity class is a set of problems of related complexity. A typical complexity class has a definition of the form:...

 according to the time it takes for an algorithm
Algorithm
In mathematics, computing, linguistics, and related subjects, an algorithm is an effective method for solving a problem using a finite sequence of instructions. Algorithms are used for calculation, data processing, and many other fields....

 - usually a computer program - to solve them as a function of the problem size. Some problems are difficult to solve, while others are easy. For example, some difficult problems need algorithms that take an exponential amount of time in terms of the size of the problem to solve. Take the travelling salesman problem
Travelling salesman problem
The Travelling Salesman Problem is a problem in combinatorial optimization studied in operations research and theoretical computer science...

, for example. It can be solved in time (where n is the size of the network to visit - let's say the number of cities the travelling salesman must visit exactly once). As the size of the network of cities grows, the time needed to find the route grows (more than) exponentially.

Even though a problem may be computationally solvable in principle, in actual practice it may not be that simple. These problems might require large amounts of time or an inordinate amount of space. Computational complexity
Computational Complexity
Computational Complexity may refer to:*Computational complexity theory*Computational Complexity...

 may be approached from many different aspects. Computational complexity
Computational Complexity
Computational Complexity may refer to:*Computational complexity theory*Computational Complexity...

 can be investigated on the basis of time, memory or other resources used to solve the problem. Time and space are two of the most important and popular considerations when problems of complexity are analyzed.

There exist a certain class of problems that although they are solvable in principle they require so much time or space that it is not practical to attempt to solve them. These problems are called intractable.

There is another form of complexity called hierarchical complexity
Model of hierarchical complexity
The model of hierarchical complexity, is a framework for scoring how complex a behavior is. It quantifies the order of hierarchical complexity of a task based on mathematical principles of how the information is organized and of information science...

. It is orthogonal to the forms of complexity discussed so far, which are called horizontal complexity

Further reading



  • Greenlaw, N. and Hoover, H.J. Fundamentals of the Theory of Computation, Morgan Kauffman Publishers, San Francisco, 1998
  • Blum, M. (1967) On the Size of Machines, Information and Control, v. 11, pp. 257-265
  • Burgin, M. (1982) Generalized Kolmogorov complexity and duality in theory of computations, Notices of the Russian Academy of Sciences, v.25, No. 3, pp.19-23
  • Mark Burgin (2005), Super-recursive algorithms, Monographs in computer science, Springer.
  • Burgin, M. and Debnath, N. Hardship of Program Utilization and User-Friendly Software, in Proceedings of the International Conference “Computer Applications in Industry and Engineering”, Las Vegas, Nevada, 2003, pp. 314-317
  • Debnath, N.C. and Burgin, M., (2003) Software Metrics from the Algorithmic Perspective, in Proceedings of the ISCA 18th International Conference “Computers and their Applications”, Honolulu, Hawaii, pp. 279-282
  • Meyers, R.A., (2009) "Encyclopedia of Complexity and Systems Science", ISBN 978-0-387-75888-6
  • Caterina Liberati, J. Andrew Howe, Hamparsum Bozdogan, Data Adaptive Simultaneous Parameter and Kernel Selection in Kernel Discriminant Analysis Using Information Complexity, Journal of Pattern Recognition Research, JPRR, Vol 4, No 1, 2009.

External links