Python syntax and semantics
Encyclopedia
The syntax of the Python programming language
is the set of rules that defines how a Python program will be written and interpreted (by both the runtime system and by human readers). Python was designed to be a highly readable language. It has a relatively uncluttered visual layout and uses English keywords frequently where other languages use punctuation. Python aims towards simplicity and generality in the design of its syntax, encapsulated in the mantra "There should be one — and preferably only one — obvious way to do it", from "The Zen of Python".
This mantra is deliberately opposed to the Perl
and Ruby
mantra, "there's more than one way to do it".
to delimit program blocks, following the off-side rule
. Its uncommon block marking convention is a feature that many programmers, otherwise unfamiliar with Python, have heard of. Python borrows this feature from its predecessor ABC — instead of punctuation or keywords, it uses indentation to indicate the run of a block.
In so-called "free-format" languages, that use the block structure derived from ALGOL
, blocks of code are set off with braces (
ing).
Consider a function,
,
with K&R indent style:
void foo(int x)
{
if (x 0) {
def foo(x):
if x 0:
bar
baz
else:
qux(x)
foo(x - 1)
Python mandates a convention that programmers in ALGOL-style languages often follow. Moreover, in free-form syntax, since indentation is ignored, good indentation cannot be enforced by an interpreter or compiler. Incorrectly indented code can be understood by human reader differently than does a compiler or interpreter. For example:
Misleading indentation in C:
for (i = 0; i < 20; ++i)
a;
b;
c;
This code was intended to call functions
Both space
characters and tab
characters are currently accepted as forms of indentation in Python. Since many tools do not visually distinguish them, mixing spaces and tabs can create bugs that take specific efforts to find (a perennial suggestion among Python users has been removing tabs as block markers — except, of course, among those Python users who propound removing spaces instead). Moreover, formatting routines which remove whitespace — for instance, many Internet forum
s — can destroy the syntax of a Python program, whereas a program in a bracketed language would merely become more difficult to read.
Many popular code editors handle Python's indentation conventions seamlessly, sometimes after a configuration option is enabled.
All variables in Python hold references to objects, and these references are passed to functions; a function cannot change the value of variable references in its calling function. Some people (including Guido van Rossum
himself) have called this parameter-passing scheme "Call by object reference."
Among dynamically typed languages, Python is moderately type-checked. Implicit conversion is defined for numeric types (as well as booleans), so one may validly multiply a complex number by a long integer (for instance) without explicit casting. However, there is no implicit conversion between (e.g.) numbers and strings; a string is an invalid argument to a mathematical function expecting a number.
, complex number
s, and decimal floating point numbers
.
Python supports a wide variety of string operations. Strings in Python are immutable
, so a string operation such as a substitution of characters, that in other programming languages might alter a string in place, returns a new string in Python. Performance considerations sometimes push for using special techniques in programs that modify strings intensively, such as joining character arrays into strings only as needed.
The ordered sequential types are lists (dynamic arrays
), tuple
s, and strings. All sequences are indexed positionally (0 through length − 1) and all but strings can contain any type of object, including multiple types in the same sequence. Both strings and tuples are immutable, making them perfect candidates for dictionary keys (see below). Lists, on the other hand, are mutable; elements can be inserted, deleted, modified, appended, or sorted in-place
.
On the other side of the collections coin are mappings, which are unordered types implemented in the form of dictionaries which "map" a set of immutable
keys, to corresponding elements much like a mathematical function. The keys in a dictionary must be of an immutable Python type such as an integer or a string.
For example, one could define a dictionary having a string
which makes for faster lookup times, but is also the culprit for a dictionary's lack of order and is the reason mutable objects (i.e. other dictionaries or lists) cannot be used as keys.
Dictionaries are also central to the internals of the language as they reside at the core of all Python objects and classes: the mapping between variable names (strings) and the values which the names reference is stored as a dictionary (see Object system). Since these dictionaries are directly accessible (via an object's
is a straightforward and natural process in Python.
A set
collection type was added to the core language in version 2.4. A set is an unindexed, unordered collection that contains no duplicates, and implements set theoretic
operations such as union
, intersection
, difference, symmetric difference
, and subset
testing. There are two types of sets:
Python also provides extensive collection manipulating abilities such as built in containment checking and a generic iteration protocol.
. Python also supports multiple inheritance
and mixin
s (see also MixinsForPython).
The language supports extensive introspection of types and classes. Types can be read and compared—types are instances of
Operators can be overloaded in Python by defining special member functions—for instance, defining
or Perl-influenced languages such as Ruby
or Groovy, single quotes and double quotes function identically, i.e. there is no string interpolation of $foo expressions. However, interpolation can be done using the % string-format operator, e.g. the Perl statement
print "I just printed $num pages to the printer $printer\n"
is equivalent to the Python statement
print("I just printed %s pages to the printer %s" % (num, printer))
However, the current standard for this sort of string formatting is to use the format method of strings:
print("I just printed {0} pages to the printer {1}".format(num, printer))
and Ruby
.
A simple example with variable interpolation (using the % string-format operator) is:
print("""Dear %(recipient)s,
I wish you to leave Sunnydale and never return.
Not Quite Love,
%(sender)s
""" % {'sender': 'Buffy the Vampire Slayer', 'recipient': 'Spike'})
s; compare "@-quoting" in C#. Raw strings were originally included specifically for regular expressions. Due to limitations of the tokenizer, raw strings may not have a trailing backslash. Creating a raw string holding a Windows path ending with a backslash requires some variety of workaround (commonly, using forward slashes instead of backslashes, since Windows accepts both).
Examples are
r"C:\Foo\Bar\Baz\ ".rstrip
r'"([^"\\]|\\.)*"'
re.sub(r'\(([^,]*?),([^,]*?)\)', r'(\2, \1)', code)
Python has arbitrary-length integers and automatically increases the storage size as necessary. Prior to Python version 3, there were two kinds of integral numbers: traditional fixed size integers and "long" integers of arbitrary range. The conversion to "long" integers was performed automatically when required, and thus the programmer usually didn't have to be aware of the two integral types. In newer language versions the fixed-size integers are completely gone.
Python supports normal floating point numbers, which are created when a dot is used in a literal (e.g.
Python (programming language)
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. Python claims to "[combine] remarkable power with very clear syntax", and its standard library is large and comprehensive...
is the set of rules that defines how a Python program will be written and interpreted (by both the runtime system and by human readers). Python was designed to be a highly readable language. It has a relatively uncluttered visual layout and uses English keywords frequently where other languages use punctuation. Python aims towards simplicity and generality in the design of its syntax, encapsulated in the mantra "There should be one — and preferably only one — obvious way to do it", from "The Zen of Python".
This mantra is deliberately opposed to the Perl
Perl
Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall in 1987 as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular...
and Ruby
Ruby (programming language)
Ruby is a dynamic, reflective, general-purpose object-oriented programming language that combines syntax inspired by Perl with Smalltalk-like features. Ruby originated in Japan during the mid-1990s and was first developed and designed by Yukihiro "Matz" Matsumoto...
mantra, "there's more than one way to do it".
Indentation
Python uses whitespaceWhitespace (computer science)
In computer science, whitespace is any single character or series of characters that represents horizontal or vertical space in typography. When rendered, a whitespace character does not correspond to a visual mark, but typically does occupy an area on a page...
to delimit program blocks, following the off-side rule
Off-side rule
A computer programming language is said to adhere to the off-side rule if the scope of declarations in that language is expressed by their indentation. The term and the idea are attributed to Peter J. Landin, and the term can be seen as a pun on the offside law of football .- Definition :Peter J...
. Its uncommon block marking convention is a feature that many programmers, otherwise unfamiliar with Python, have heard of. Python borrows this feature from its predecessor ABC — instead of punctuation or keywords, it uses indentation to indicate the run of a block.
In so-called "free-format" languages, that use the block structure derived from ALGOL
ALGOL
ALGOL is a family of imperative computer programming languages originally developed in the mid 1950s which greatly influenced many other languages and became the de facto way algorithms were described in textbooks and academic works for almost the next 30 years...
, blocks of code are set off with braces (
{ }) or keywords. In most coding conventions for these languages, programmers conventionally indent the code within a block, to visually set it apart from the surrounding code (prettyprintPrettyprint
Prettyprint is the application of any of various stylistic formatting conventions to text, source code, markup, and other similar kinds of content. These formatting conventions usually consist of changes in positioning, spacing, color, contrast, size and similar modifications intended to make the...
ing).
Consider a function,
foo, which is passed a single parameterParameter (computer science)
In computer programming, a parameter is a special kind of variable, used in a subroutine to refer to one of the pieces of data provided as input to the subroutine. These pieces of data are called arguments...
,
x, and if the parameter is 0 will call bar and baz, otherwise it will call qux, passing x, and also call itself recursively, passing x-1 as the parameter. Here are implementations of this function in both C and Python:foo function in CC (programming language)
C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
with K&R indent style:
void foo(int x)
{
if (x
0) {
bar;
baz;
} else {
qux(x);
foo(x - 1);
}
}
foo function in Python:def foo(x):
if x 0:
bar
baz
else:
qux(x)
foo(x - 1)
Python mandates a convention that programmers in ALGOL-style languages often follow. Moreover, in free-form syntax, since indentation is ignored, good indentation cannot be enforced by an interpreter or compiler. Incorrectly indented code can be understood by human reader differently than does a compiler or interpreter. For example:
Misleading indentation in C:
for (i = 0; i < 20; ++i)
a;
b;
c;
This code was intended to call functions
a and b 20 times. However, the iterated code block is just {a;}. The code calls a 20 times, and then calls b and c only one time each. Later readers of the code may be fooled by the misalignment of the calls to functions b and c.Both space
Space (punctuation)
In writing, a space is a blank area devoid of content, serving to separate words, letters, numbers, and punctuation. Conventions for interword and intersentence spaces vary among languages, and in some cases the spacing rules are quite complex....
characters and tab
Tab key
Tab key on a keyboard is used to advance the cursor to the next tab stop.- Origin :The word tab derives from the word tabulate, which means "to arrange data in a tabular, or table, form"...
characters are currently accepted as forms of indentation in Python. Since many tools do not visually distinguish them, mixing spaces and tabs can create bugs that take specific efforts to find (a perennial suggestion among Python users has been removing tabs as block markers — except, of course, among those Python users who propound removing spaces instead). Moreover, formatting routines which remove whitespace — for instance, many Internet forum
Internet forum
An Internet forum, or message board, is an online discussion site where people can hold conversations in the form of posted messages. They differ from chat rooms in that messages are at least temporarily archived...
s — can destroy the syntax of a Python program, whereas a program in a bracketed language would merely become more difficult to read.
Many popular code editors handle Python's indentation conventions seamlessly, sometimes after a configuration option is enabled.
Data structures
Since Python is a dynamically typed language, Python values, not variables, carry type. This has implications for many aspects of the way the language functions.All variables in Python hold references to objects, and these references are passed to functions; a function cannot change the value of variable references in its calling function. Some people (including Guido van Rossum
Guido van Rossum
Guido van Rossum is a Dutch computer programmer who is best known as the author of the Python programming language. In the Python community, Van Rossum is known as a "Benevolent Dictator For Life" , meaning that he continues to oversee the Python development process, making decisions where necessary...
himself) have called this parameter-passing scheme "Call by object reference."
Among dynamically typed languages, Python is moderately type-checked. Implicit conversion is defined for numeric types (as well as booleans), so one may validly multiply a complex number by a long integer (for instance) without explicit casting. However, there is no implicit conversion between (e.g.) numbers and strings; a string is an invalid argument to a mathematical function expecting a number.
Base types
Python has a broad range of basic data types. Alongside conventional integer and floating point arithmetic, it transparently supports arbitrary-precision arithmeticArbitrary-precision arithmetic
In computer science, arbitrary-precision arithmetic indicates that calculations are performed on numbers whose digits of precision are limited only by the available memory of the host system. This contrasts with the faster fixed-precision arithmetic found in most ALU hardware, which typically...
, complex number
Complex number
A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...
s, and decimal floating point numbers
Decimal floating point
Decimal floating point arithmetic refers to both a representation and operations on decimal floating point numbers. Working directly with decimal fractions can avoid the rounding errors that otherwise typically occur when converting between decimal fractions and binary fractions.The...
.
Python supports a wide variety of string operations. Strings in Python are immutable
Immutable object
In object-oriented and functional programming, an immutable object is an object whose state cannot be modified after it is created. This is in contrast to a mutable object, which can be modified after it is created...
, so a string operation such as a substitution of characters, that in other programming languages might alter a string in place, returns a new string in Python. Performance considerations sometimes push for using special techniques in programs that modify strings intensively, such as joining character arrays into strings only as needed.
Collection types
One of the very useful aspects of Python is the concept of collection (or container) types. In general a collection is an object that contains other objects in a way that is easily referenced or indexed. Collections come in two basic forms: sequences and mappings.The ordered sequential types are lists (dynamic arrays
Array data type
In computer science, an array type is a data type that is meant to describe a collection of elements , each selected by one or more indices that can be computed at run time by the program. Such a collection is usually called an array variable, array value, or simply array...
), tuple
Tuple
In mathematics and computer science, a tuple is an ordered list of elements. In set theory, an n-tuple is a sequence of n elements, where n is a positive integer. There is also one 0-tuple, an empty sequence. An n-tuple is defined inductively using the construction of an ordered pair...
s, and strings. All sequences are indexed positionally (0 through length − 1) and all but strings can contain any type of object, including multiple types in the same sequence. Both strings and tuples are immutable, making them perfect candidates for dictionary keys (see below). Lists, on the other hand, are mutable; elements can be inserted, deleted, modified, appended, or sorted in-place
In-place algorithm
In computer science, an in-place algorithm is an algorithm which transforms input using a data structure with a small, constant amount of extra storage space. The input is usually overwritten by the output as the algorithm executes...
.
On the other side of the collections coin are mappings, which are unordered types implemented in the form of dictionaries which "map" a set of immutable
Immutable object
In object-oriented and functional programming, an immutable object is an object whose state cannot be modified after it is created. This is in contrast to a mutable object, which can be modified after it is created...
keys, to corresponding elements much like a mathematical function. The keys in a dictionary must be of an immutable Python type such as an integer or a string.
For example, one could define a dictionary having a string
"toast" mapped to the integer 42 or vice versa. This is done under the covers via a hash functionHash function
A hash function is any algorithm or subroutine that maps large data sets to smaller data sets, called keys. For example, a single integer can serve as an index to an array...
which makes for faster lookup times, but is also the culprit for a dictionary's lack of order and is the reason mutable objects (i.e. other dictionaries or lists) cannot be used as keys.
Dictionaries are also central to the internals of the language as they reside at the core of all Python objects and classes: the mapping between variable names (strings) and the values which the names reference is stored as a dictionary (see Object system). Since these dictionaries are directly accessible (via an object's
__dict__ attribute), metaprogrammingMetaprogramming
Metaprogramming is the writing of computer programs that write or manipulate other programs as their data, or that do part of the work at compile time that would otherwise be done at runtime...
is a straightforward and natural process in Python.
A set
Set (computer science)
In computer science, a set is an abstract data structure that can store certain values, without any particular order, and no repeated values. It is a computer implementation of the mathematical concept of a finite set...
collection type was added to the core language in version 2.4. A set is an unindexed, unordered collection that contains no duplicates, and implements set theoretic
Naive set theory
Naive set theory is one of several theories of sets used in the discussion of the foundations of mathematics. The informal content of this naive set theory supports both the aspects of mathematical sets familiar in discrete mathematics , and the everyday usage of set theory concepts in most...
operations such as union
Union (set theory)
In set theory, the union of a collection of sets is the set of all distinct elements in the collection. The union of a collection of sets S_1, S_2, S_3, \dots , S_n\,\! gives a set S_1 \cup S_2 \cup S_3 \cup \dots \cup S_n.- Definition :...
, intersection
Intersection (set theory)
In mathematics, the intersection of two sets A and B is the set that contains all elements of A that also belong to B , but no other elements....
, difference, symmetric difference
Symmetric difference
In mathematics, the symmetric difference of two sets is the set of elements which are in either of the sets and not in their intersection. The symmetric difference of the sets A and B is commonly denoted by A\,\Delta\,B\,orA \ominus B....
, and subset
Subset
In mathematics, especially in set theory, a set A is a subset of a set B if A is "contained" inside B. A and B may coincide. The relationship of one set being a subset of another is called inclusion or sometimes containment...
testing. There are two types of sets:
set and frozenset, the only difference being that set is mutable and frozenset is immutable. Elements in a set must be hashable and immutable. Thus, for example, a frozenset can be an element of a regular set whereas the opposite is not true.Python also provides extensive collection manipulating abilities such as built in containment checking and a generic iteration protocol.
Object system
In Python, everything is an object, even classes. Classes, as objects, have a class, which is known as their metaclassMetaclass
In object-oriented programming, a metaclass is a class whose instances are classes. Just as an ordinary class defines the behavior of certain objects, a metaclass defines the behavior of certain classes and their instances. Not all object-oriented programming languages support metaclasses...
. Python also supports multiple inheritance
Multiple inheritance
Multiple inheritance is a feature of some object-oriented computer programming languages in which a class can inherit behaviors and features from more than one superclass....
and mixin
Mixin
In object-oriented programming languages, a mixin is a class that provides a certain functionality to be inherited or just reused by a subclass, while not meant for instantiation , Mixins are synonymous functionally with abstract base classes...
s (see also MixinsForPython).
The language supports extensive introspection of types and classes. Types can be read and compared—types are instances of
type. The attributes of an object can be extracted as a dictionary.Operators can be overloaded in Python by defining special member functions—for instance, defining
__add__ on a class permits one to use the + operator on members of that class.Normal string literals
Either single or double quotes can be used to quote strings. Unlike in Unix shell languages, PerlPerl
Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall in 1987 as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular...
or Perl-influenced languages such as Ruby
Ruby (programming language)
Ruby is a dynamic, reflective, general-purpose object-oriented programming language that combines syntax inspired by Perl with Smalltalk-like features. Ruby originated in Japan during the mid-1990s and was first developed and designed by Yukihiro "Matz" Matsumoto...
or Groovy, single quotes and double quotes function identically, i.e. there is no string interpolation of $foo expressions. However, interpolation can be done using the % string-format operator, e.g. the Perl statement
print "I just printed $num pages to the printer $printer\n"
is equivalent to the Python statement
print("I just printed %s pages to the printer %s" % (num, printer))
However, the current standard for this sort of string formatting is to use the format method of strings:
print("I just printed {0} pages to the printer {1}".format(num, printer))
Multi-line string literals
There are also multi-line strings, which begin and end with a series of three single or double quotes and function like here documents in PerlPerl
Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall in 1987 as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular...
and Ruby
Ruby (programming language)
Ruby is a dynamic, reflective, general-purpose object-oriented programming language that combines syntax inspired by Perl with Smalltalk-like features. Ruby originated in Japan during the mid-1990s and was first developed and designed by Yukihiro "Matz" Matsumoto...
.
A simple example with variable interpolation (using the % string-format operator) is:
print("""Dear %(recipient)s,
I wish you to leave Sunnydale and never return.
Not Quite Love,
%(sender)s
""" % {'sender': 'Buffy the Vampire Slayer', 'recipient': 'Spike'})
Raw strings
Finally, all of the previously-mentioned string types come in "raw" varieties (denoted by placing a literal r before the opening quote), which do no backslash-interpolation and hence are very useful for regular expressionRegular expression
In computing, a regular expression provides a concise and flexible means for "matching" strings of text, such as particular characters, words, or patterns of characters. Abbreviations for "regular expression" include "regex" and "regexp"...
s; compare "@-quoting" in C#. Raw strings were originally included specifically for regular expressions. Due to limitations of the tokenizer, raw strings may not have a trailing backslash. Creating a raw string holding a Windows path ending with a backslash requires some variety of workaround (commonly, using forward slashes instead of backslashes, since Windows accepts both).
Examples are
- A Windows path, even raw strings cannot end in a backslash
r"C:\Foo\Bar\Baz\ ".rstrip
- A regular expression matching a quoted string with possible backslash quoting
r'"([^"\\]|\\.)*"'
- Reverse the arguments in a two-arg function call, e.g. foo(2, bar) -> foo(bar, 2);
- will fail if either argument has parens or commas in it
re.sub(r'\(([^,]*?),([^,]*?)\)', r'(\2, \1)', code)
Numbers
Numeric literals in Python are of the normal sort, e.g.0, -1, 3.4, 3.5e-8.Python has arbitrary-length integers and automatically increases the storage size as necessary. Prior to Python version 3, there were two kinds of integral numbers: traditional fixed size integers and "long" integers of arbitrary range. The conversion to "long" integers was performed automatically when required, and thus the programmer usually didn't have to be aware of the two integral types. In newer language versions the fixed-size integers are completely gone.
Python supports normal floating point numbers, which are created when a dot is used in a literal (e.g.
1.1), when an integer and a floating point number are used in an expression, or as a result of some mathematical operations ("true division" via the / operator, or exponentiation with a negative exponent).
Python also supports complex numberComplex numberA complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...
s natively. Complex numbers are indicated with the J or j suffix, e.g. 3 + 4j.
Lists, tuples, sets, dictionaries
Python has syntactic support for the creation of container types.
Lists (class list) are mutable sequences of items of arbitrary types, and can be created either with the special syntax
a_list = [1, 2, 3, "a dog"]
or using normal object creation
a_second_list = list
a_second_list.append(4)
a_second_list.append(5)
Tuples (class tuple) are immutable sequences of items of arbitrary types. There is also a special syntax to create tuples
a_tuple = 1, 2, 3, "four"
Although tuples are created by separating items with commas, the whole construct is usually wrapped in parentheses to increase readability. An empty tuple is denoted by
Sets (class set) are mutable containers of items of arbitrary types. The items are not ordered, but sets support iteration over the items. A syntax for set creation appeared in Python 2.7/3.0
some_set = {0, , False}
In earlier Python versions, sets would be created by calling initializing the set class with a list argument. Python sets are very much like mathematical sets, and support operations like set intersection and union.
Python also features a frozenset class for immutable sets.
Dictionaries (class dict) are mutable mappings tying keys and corresponding values.
Python has special syntax to create dictionaries ({key: value})
a_dictionary = {"key 1":"value 1", 2:3, 4:[]}
The dictionary syntax is similar to the set syntax, the difference is the presence of colons. The empty literal {} results in an empty dictionary rather than an empty set, which is instead created using the non-literal constructor: set.
Arithmetic
Python includes the +, -, *, /, % (modulus), and ** (exponentiationExponentiationExponentiation is a mathematical operation, written as an, involving two numbers, the base a and the exponent n...
) operators, with their usual mathematical precedenceOrder of operationsIn mathematics and computer programming, the order of operations is a rule used to clarify unambiguously which procedures should be performed first in a given mathematical expression....
.
Traditionally, x / y performed integer division if both x and y were integers (returning the floor of the quotient), and returned a float if either was a float. However, because Python is a dynamically-typed language, it was not always possible to tell which operation was being performed, which often led to subtle bugs. For example, with
def mean(seq):
return sum(seq) / len(seq)
A call to mean([3.0, 4.0]) would return 3.5, but mean([3, 4]) would return 3. If this was not the intended behavior, it was necessary to use a workaround such as
def mean(seq):
return float(sum(seq)) / len(seq)
To avoid this issue, a proposal was made to change the behavior of the Python division operator. In Python 2.2, a new operator // was introduced for floor division, both for integer and floating-point arguments. The / operator was changed so that the quotient of two integers returned a float, but for backwards compatibility, this behavior had to be explicitly requested until Python 3.0.
Comparison operators
The basic comparison operators such as , <, >=, and so forth are used on all manner of values. Numbers, strings, sequences, and mappings can all be compared. Although disparate types (such as a str and an int) are defined to have a consistent relative ordering, this is considered a historical design quirk and will no longer be allowed in Python 3.0.
Chained comparison expressions such as a < b < c have roughly the meaning that they have in mathematics, rather than the unusual meaning found in CC (programming language)C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
and similar languages. The terms are evaluated and compared in order. The operation has short-circuit semanticsMinimal evaluationShort-circuit evaluation, minimal evaluation, or McCarthy evaluation denotes the semantics of some Boolean operators in some programming languages in which the second argument is only executed or evaluated if the first argument does not suffice to determine the value of the expression: when the...
, meaning that evaluation is guaranteed to stop as soon as a verdict is clear: if a < b is false, c is never evaluated as the expression cannot possibly be true anymore.
For expressions without side effects, a < b < c is equivalent to a < b and b < c. However, there is a substantial difference when the expressions have side effects. a < f(x) < b will evaluate f(x) exactly once, whereas a < f(x) and f(x) < b will evaluate it twice if the value of a is less than f(x) and once otherwise.
Logical operators
Python 2.2 and earlier does not have an explicit boolean type. In all versions of Python, boolean operators treat zero values or empty values such as "", 0, None, 0.0, [], and {} as false, while in general treating non-empty, non-zero values as true. In Python 2.2.1 the boolean constants True and False were added to the language (subclassed from 1 and 0). The binary comparison operators such as > return either True or False.
The boolean operators and and or use minimal evaluationMinimal evaluationShort-circuit evaluation, minimal evaluation, or McCarthy evaluation denotes the semantics of some Boolean operators in some programming languages in which the second argument is only executed or evaluated if the first argument does not suffice to determine the value of the expression: when the...
. For example, y 0 or x/y > 100 will never raise a divide-by-zero exception. Note that these operators return the value of the last operand evaluated, rather than True or False. Thus the expression (4 and 5) evaluates to 5, and (4 or 5) evaluates to 4.
Functional programming
As mentioned above, another strength of Python is the availability of a functional programmingFunctional programmingIn computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state...
style. As may be expected, this makes working with lists and other collections much more straightforward.
List comprehensions
One such construction is the list comprehension, as seen here in calculating the first five powers of two:
powers_of_two = [2**n for n in range(1, 6)]
The Quicksort algorithm can be expressed elegantly, albeit inefficiently, using list comprehensions:
def qsort(L):
if L []:
return []
pivot = L[0]
return (qsort([x for x in L[1:] if x < pivot]) +
[pivot] +
qsort([x for x in L[1:] if x >= pivot]))
First-class functions
In Python, functions are first-classFirst-class functionIn computer science, a programming language is said to have first-class functions if it treats functions as first-class objects. Specifically, this means that the language supports passing functions as arguments to other functions, returning them as the values from other functions, and assigning...
objects that can be created and passed around dynamically.
Python's limited support for anonymous functionAnonymous functionIn programming language theory, an anonymous function is a function defined, and possibly called, without being bound to an identifier. Anonymous functions are convenient to pass as an argument to a higher-order function and are ubiquitous in languages with first-class functions such as Haskell...
s is the lambda construct.
Since the availability of full anonymous functions is non-existent, the main use for lambda functions is named functions. Lambdas are limited to containing expressionsExpression (programming)An expression in a programming language is a combination of explicit values, constants, variables, operators, and functions that are interpreted according to the particular rules of precedence and of association for a particular programming language, which computes and then produces another value...
rather than statementsStatement (programming)In computer programming a statement can be thought of as the smallest standalone element of an imperative programming language. A program written in such a language is formed by a sequence of one or more statements. A statement will have internal components .Many languages In computer programming...
, although control flow can still be implemented less elegantly within lambda by using short-circuiting.
Closures
Python has had support for lexical closuresClosure (computer science)In computer science, a closure is a function together with a referencing environment for the non-local variables of that function. A closure allows a function to access variables outside its typical scope. Such a function is said to be "closed over" its free variables...
since version 2.2. Here's an example:
def derivative(f, dx):
"""Return a function that approximates the derivative of f
using an interval of dx, which should be appropriately small.
"""
def function(x):
return (f(x + dx) - f(x)) / dx
return function
Python's syntax, though, sometimes leads programmers of other languages to think that closures are not supported. Variable scope in Python is implicitly determined by the scope in which one assigns a value to the variable, unless scope is explicitly declared with global or nonlocal.
Generators
Introduced in Python 2.2 as an optional feature and finalized in version 2.3, generatorsGenerator (computer science)In computer science, a generator is a special routine that can be used to control the iteration behaviour of a loop. A generator is very similar to a function that returns an array, in that a generator has parameters, can be called, and generates a sequence of values...
are Python's mechanism for lazy evaluationLazy evaluationIn programming language theory, lazy evaluation or call-by-need is an evaluation strategy which delays the evaluation of an expression until the value of this is actually required and which also avoids repeated evaluations...
of a function that would otherwise return a space-prohibitive or computationally intensive list.
This is an example to lazily generate the prime numbers:
from itertools import count
def generate_primes(stop_at=0):
primes = []
for n in count(2):
if 0 < stop_at < n:
return # raises the StopIteration exception
composite = False
for p in primes:
if not n % p:
composite = True
break
elif p**2 > n:
break
if not composite:
primes.append(n)
yield n
To use this function simply call, e.g.:
for i in generate_primes: # iterate over ALL primes
if i > 100:
break
print(i)
The definition of a generator appears identical to that of a function, except the keyword yield is used in place of return. However, a generator is an object with persistent state, which can repeatedly enter and leave the same scope. A generator call can then be used in place of a list, or other structure whose elements will be iterated over. Whenever the for loop in the example requires the next item, the generator is called, and yields the next item.
Generators don't have to be infinite like the prime-number example above. When a generator terminates, an internal exception is raised which indicates to any calling context that there are no more values. A for loop or other iteration will then terminate.
Generator expressions
Introduced in Python 2.4, generator expressions are the lazy evaluation equivalent of list comprehensions. Using the prime number generator provided in the above section, we might define a lazy, but not quite infinite collection.
from itertools import islice
primes_under_million = (i for i in generate_primes if i < 1000000)
two_thousandth_prime = islice(primes_under_million, 2000, 2001).next
Most of the memory and time needed to generate this many primes will not be used until the needed element is actually accessed. Unfortunately, you cannot perform simple indexing and slicing of generators, but must use the itertools modules or "roll your own" loops. In contrast, a list comprehension is functionally equivalent, but is greedy in performing all the work:
primes_under_million = [i for i in generate_primes(2000000) if i < 1000000]
two_thousandth_prime = primes_under_million[1999]
The list comprehension will immediately create a large list (with 78498 items, in the example, but transiently creating a list of primes under two million), even if most elements are never accessed. The generator comprehension is more parsimonious.
Dictionary and set comprehensions
While lists and generators had comprehensions/expressions, in Python versions older than 2.7 the other Python built-in collection types (dicts and sets) had to be kludged in using lists or generators:
>>> dict((n, n*n) for n in range(5))
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
Python 2.7 and 3.0 unify all collection types by introducing dict and set comprehensions, similar to list comprehensions:
>>> [ n*n for n in range(5) ] # regular list comprehension
[0, 1, 4, 9, 16]
>>>
>>> { n*n for n in range(5) } # set comprehension
{0, 1, 4, 16, 9}
>>>
>>> { n: n*n for n in range(5) } # dict comprehension
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
Objects
Python supports most object oriented programming techniques. It allows polymorphism, not only within a class hierarchyClass hierarchyAs in taxonomy, the classifications of species, a class hierarchy in computer science is a classification of object types, denoting objects as the instantiations of classes inter-relating the various classes by relationships such as "inherits", "extends", "is an abstraction of", "an interface...
but also by duck typingDuck typingIn computer programming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface...
. Any object can be used for any type, and it will work so long as it has the proper methods and attributes. And everything in Python is an object, including classes, functions, numbers and modules. Python also has support for metaclassMetaclassIn object-oriented programming, a metaclass is a class whose instances are classes. Just as an ordinary class defines the behavior of certain objects, a metaclass defines the behavior of certain classes and their instances. Not all object-oriented programming languages support metaclasses...
es, an advanced tool for enhancing classes' functionality. Naturally, inheritanceInheritance (computer science)In object-oriented programming , inheritance is a way to reuse code of existing objects, establish a subtype from an existing object, or both, depending upon programming language support...
, including multiple inheritanceMultiple inheritanceMultiple inheritance is a feature of some object-oriented computer programming languages in which a class can inherit behaviors and features from more than one superclass....
, is supported. It has limited support for private variables using name manglingName manglingIn compiler construction, name mangling is a technique used to solve various problems caused by the need to resolve unique names for programming entities in many modern programming languages....
. See the "Classes" section of the tutorial for details.
Many Python users don't feel the need for private variables, though.
The slogan "We're all consenting adults here" is used to describe this attitude.
Some consider information hidingInformation hidingIn computer science, information hiding is the principle of segregation of the design decisions in a computer program that are most likely to change, thus protecting other parts of the program from extensive modification if the design decision is changed...
to be unpythonic, in that it suggests that the class in question contains unaesthetic or ill-planned internals. However, the strongest argument for name mangling is prevention of unpredictable breakage of programs: introducing a new public variable in a superclass can break subclasses if they don't use "private" variables.
From the tutorial: As is true for modules, classes in Python do not put an absolute barrier between definition and user, but rather rely on the politeness of the user not to "break into the definition."
OOP doctrines such as the use of accessor methods to read data members are not enforced in Python. Just as Python offers functional-programming constructs but does not attempt to demand referential transparency, it offers an object system but does not demand OOP behavior. Moreover, it is always possible to redefine the class using properties so that when a certain variable is set or retrieved in calling code, it really invokes a function call, so that spam.eggs = toast might really invoke spam.set_eggs(toast). This nullifies the practical advantage of accessor functions, and it remains OOP because the property eggs becomes a legitimate part of the object's interface: it need not reflect an implementation detail.
In version 2.2 of Python, "new-style" classes were introduced. With new-style classes, objects and types were unified, allowing the subclassing of types.
Even entirely new types can be defined, complete with custom behavior for infix operators. This allows for many radical things to be done syntactically within Python. A new method resolution order for multiple inheritanceMultiple inheritanceMultiple inheritance is a feature of some object-oriented computer programming languages in which a class can inherit behaviors and features from more than one superclass....
was also adopted with Python 2.3. It is also possible to run custom code while accessing or setting attributes, though the details of those techniques have evolved between Python versions.
Properties
Properties allow specially defined methods to be invoked on an object instance by using the same syntax as used for attribute access. An example of a class defining some properties is:
class MyClass(object):
def get_a(self):
return self._a + 1
def set_a(self, value):
self._a = value - 1
a = property(get_a, set_a, doc="Off by one a")
- Python 2.6 style
class MyClass(object):
@property
def a(self):
return self._a + 1
@a.setter # makes the property writable
def a(self, value):
self._a = value - 1
Descriptors
A class that defines one or more of the special methods __get__(self,instance,owner), __set__(self,instance,value), __delete__(self,instance) can be used as a descriptor. Creating an instance of a descriptor as a class member of a second class makes the instance a property of the second class.
Class and Static Methods
Python allows the creation of class methods and static method via the use of the @classmethod and @staticmethod decorators. The first argument to a class method is the class object instead of the self reference to the instance. A static method has no special first argument. Neither the instance, nor the class object is passed to a static method.
Exceptions
Python supports (and extensively uses) exception handlingException handlingException handling is a programming language construct or computer hardware mechanism designed to handle the occurrence of exceptions, special conditions that change the normal flow of program execution....
as a means of testing for error conditions and other "exceptional" events in a program. Indeed, it is even possible to trap the exception caused by a syntax errorSyntax errorIn computer science, a syntax error refers to an error in the syntax of a sequence of characters or tokens that is intended to be written in a particular programming language....
.
Python style calls for the use of exceptions whenever an error condition might arise. Rather than testing for access to a file or resource before actually using it, it is conventional in Python to just go ahead and try to use it, catching the exception if access is rejected.
Exceptions can also be used as a more general means of non-local transfer of control, even when an error is not at issue. For instance, the MailmanGNU MailmanGNU Mailman is a computer software application from the GNU project for managing electronic mailing lists.Mailman is coded primarily in Python and currently maintained by Barry Warsaw...
mailing list software, written in Python, uses exceptions to jump out of deeply-nested message-handling logic when a decision has been made to reject a message or hold it for moderator approval.
Exceptions are often used as an alternative to the if-block, especially in threadedThread (computer science)In computer science, a thread of execution is the smallest unit of processing that can be scheduled by an operating system. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process...
situations. A commonly-invoked motto is EAFP, or "It is Easier to Ask for Forgiveness than Permission," which is attributed to Grace Hopper In this first code sample, there is an explicit check for the attribute (i.e., "asks permission"):
if hasattr(spam, 'eggs'):
ham = spam.eggs
else:
handle_error
This second sample follows the EAFP paradigm:
try:
ham = spam.eggs
except AttributeError:
handle_error
These two code samples have the same effect, although there will be performance differences. When spam has the attribute eggs, the EAFP sample will run faster. When spam does not have the attribute eggs (the "exceptional" case), the EAFP sample will run slower. The Python profiler can be used in specific cases to determine performance characteristics. If exceptional cases are rare, then the EAFP version will have superior average performance than the alternative. In addition, it avoids the whole class of time-of-check-to-time-of-useTime-of-check-to-time-of-useIn software development, time-of-check-to-time-of-use is a class of software bug caused by changes in a system between the checking of a condition and the use of the results of that check...
(TOCTTOU) vulnerabilities, other race conditions, and is compatible with duck typingDuck typingIn computer programming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface...
.
Comments and docstrings
Python has two ways to annotate Python code. One is by using comments to indicate what some part of the code does. Comments begin with the hash character ("#") and are terminated by the end of line. Python does not support comments that span more than one line.
Commenting a piece of code:
def getline:
return sys.stdin.readline # Get one line and return it
The other way is to use docstringDocstringIn programming, a docstring is a string literal specified in source code that is used, like a comment, to document a specific segment of code. Unlike conventional source code comments, or even specifically formatted comments like Javadoc documentation, docstrings are not stripped from the source...
s (documentation string), that is a string that is located alone without assignment as the first line within a module, class, method or function. Such strings can be delimited with " or ' """ or """) are preferred for both single and multi-line docstrings.
Single line docstring:
def getline:
"""Get one line from stdin and return it."""
return sys.stdin.readline
Multi-line docstring:
def getline:
"""Get one line
from stdin
and return it."""
return sys.stdin.readline
Docstrings can be as large as the programmer wants and contain line breaksNewlineIn computing, a newline, also known as a line break or end-of-line marker, is a special character or sequence of characters signifying the end of a line of text. The name comes from the fact that the next character after the newline will appear on a new line—that is, on the next line below the...
. In contrast with comments, docstrings are themselves Python objects and are part of the interpreted code that Python runs. That means that a running program can retrieve its own docstrings and manipulate that information. But the normal usage is to give other programmers information about how to invoke the object being documented in the docstring.
There are tools available that can extract the docstrings to generate an APIApplication programming interfaceAn application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
documentation from the code. Docstring documentation can also be accessed from the interpreter with the help function, or from the shell with the pydocPydocPydoc is a documentation module for the programming language Python. Similar to the functionality of Perldoc within Perl, Pydoc allows Python programmers to access Python's documentation help files, generate HTML pages with documentation specifics, and find the appropriate module for a particular job...
command pydoc.
The doctestDoctestdoctest is a module included in the Python programming language's standard library that allows the easy generation of tests based on output from the standard Python interpreter shell, cut and pasted into docstrings.-Implementation specifics:...
standard module uses interactions copied from Python shell sessions into docstrings, to create tests.
Decorators
A decorator is any callable Python object that is used to modify a function, method or class definition. A decorator is passed the original object being defined and returns a modified object, which is then bound to the name in the definition. Python decorators were inspired in part by Java annotationJava annotationAn annotation, in the Java computer programming language, is a special form of syntactic metadata that can be added to Java source code. Classes, methods, variables, parameters and packages may be annotated...
s, and have a similar syntax; the decorator syntax is pure syntactic sugarSyntactic sugarSyntactic sugar is a computer science term that refers to syntax within a programming language that is designed to make things easier to read or to express....
, using @ as the keyword:
@viking_chorus
def menu_item:
print("spam")
is equivalent to
def menu_item:
print("spam")
menu_item = viking_chorus(menu_item)
Decorators are a form of metaprogrammingMetaprogrammingMetaprogramming is the writing of computer programs that write or manipulate other programs as their data, or that do part of the work at compile time that would otherwise be done at runtime...
; they enhance the action of the function or method they decorate. For example, in the above sample, viking_chorus might cause menu_item to be run 8 times (see Spam sketch) for each time it is called:
def viking_chorus(myfunc):
def inner_func(*args, **kwargs):
for i in range(8):
myfunc(*args, **kwargs)
return inner_func
Canonical uses of function decorators are for creating class methods or static methods, adding function attributes, tracingTracing (software)In software engineering, tracing is a specialized use of logging to record information about a program's execution. This information is typically used by programmers for debugging purposes, and additionally, depending on the type and detail of information contained in a trace log, by experienced...
, setting pre-PreconditionIn computer programming, a precondition is a condition or predicate that must always be true just prior to the execution of some section of code or before an operation in a formal specification....
and postconditionPostconditionIn computer programming, a postcondition is a condition or predicate that must always be true just after the execution of some section of code or after an operation in a formal specification. Postconditions are sometimes tested using assertions within the code itself...
s, and synchronisation, but can be used for far more besides, including tail recursion elimination, memoizationMemoizationIn computing, memoization is an optimization technique used primarily to speed up computer programs by having function calls avoid repeating the calculation of results for previously processed inputs...
and even improving the writing of decorators.
Decorators can be chained by placing several on adjacent lines:
@invincible
@favorite_color("Blue")
def black_knight:
pass
is equivalent to
def black_knight:
pass
black_knight = invincible(favorite_color("Blue")(black_knight))
or, using intermediate variables
def black_knight:
pass
blue_decorator = favorite_color("Blue")
decorated_by_blue = blue_decorator(black_knight)
black_knight = invincible(decorated_by_blue)
In the above example, the favorite_color decorator factory takes an argument. Decorator factories must return a decorator, which is then called with the object to be decorated as its argument:
def favorite_color(color):
def decorator(func):
def wrapper:
print(color)
func
return wrapper
return decorator
This would then decorate the black_knight function such that the color, "Blue", would be printed prior to the black_knight function running.
In Python prior to version 2.6, decorators apply to functions and methods, but not to classes. Decorating a (dummy) __new__ method can modify a class, however.
Class decorators are supported starting with Python 2.6.
Despite the name, Python decorators are not an implementation of the decorator patternDecorator patternIn object-oriented programming, the decorator pattern is a design pattern that allows behaviour to be added to an existing object dynamically.-Introduction:...
. The decorator pattern is a design patternDesign patternA design pattern in architecture and computer science is a formal way of documenting a solution to a design problem in a particular field of expertise. The idea was introduced by the architect Christopher Alexander in the field of architecture and has been adapted for various other disciplines,...
used in statically typed object-oriented programming languageObject-oriented programming languageThis is a list of object-oriented programming programming languages.-Languages with object-oriented features:*ABAP*Ada 95*AmigaE*BETA*Blue*Boo*C++*C#*COBOL*Cobra*ColdFusion*Common Lisp*COOL*CorbaScript*Clarion*CLU*Curl*D*Dylan*E*Eiffel...
s to allow functionality to be added to objects at run time; Python decorators add functionality to functions and methods at definition time, and thus are a higher-level construct than decorator-pattern classes. The decorator pattern itself is trivially implementable in Python, because the language is duck typed, and so is not usually considered as such.
See also: Advice in Lisp.
Easter eggs
Users of curly bracket programming languages, such as CC (programming language)C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
or JavaJava (programming language)Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...
, sometimes expect or wish Python to follow a block-delimiter convention. Brace-delimited block syntax has been repeatedly requested, and consistently rejected by core developers. The Python interpreter contains an easter egg that summarizes its developers' feelings on this issue. The code from __future__ import braces raises the exception SyntaxError: not a chance. The __future__ module is normally used to provide features from future versionsBackportingBackporting is the action of taking a certain software modification and applying it to an older version of the software than it was initially created for. It is part of the maintenance step in a software development process....
of Python.
Another hidden message, The Zen of Python (a summary of Python philosophy), is displayed when trying to import this.
The message Hello world... is printed when the import statement import __hello__ is used.
An antigravity module was added to Python 2.7 and 3.0. Importing it opens a web browser to an xkcdXkcdxkcd is a webcomic created by Randall Munroe. The comic's tagline describes it as "a webcomic of romance, sarcasm, math, and language." It has been recognized in such mainstream media as The Guardian and The New York Times....
comic that portrays a humorous fictional use for such a module, intended to demonstrate the ease with which Python modules enable additional functionality.
External links
- Python tutorial - Tutorial written by the author of Python, Guido van Rossum.