ADMB
Encyclopedia
ADMB or AD Model Builder is a free and open source software
suite for non-linear statistical
modeling. It was created by David Fournier and now being developed by the ADMB Project, a creation of the non-profit ADMB Foundation. The "AD" in AD Model Builder refers to the automatic differentiation
capabilities that come from the AUTODIF Library, a C++ language extension also created by David Fournier, which implements reverse mode automatic differentiation. A related software package, ADMB-RE, provides additional support for modeling random effects.
methods are integrated into the ADMB software, making it useful for Bayesian
modeling. In addition to Bayesian hierarchical models, ADMB provides support for modeling random effects in a frequentist framework using Laplace approximation and importance sampling.
ADMB is widely used by scientists in academic institutions, government agencies, and international commissions, most commonly for ecological modeling. In particular, many fisheries stock assessment
models have been built using this software. ADMB is freely available under the New BSD License,
with versions available for Windows
, Linux
, Mac OS X
, and OpenSolaris
operating system
s. Source code for ADMB was made publicly available in March, 2009.
integrated statistical models in fisheries motivated the
development of the AUTODIF Library, and ultimately ADMB.
The likelihood equations
in these models are typically non-linear and estimates of the
parameters are
obtained by numerical methods.
Early in Fournier's work, it became clear that general numerical
solutions to these likelihood problems could only be reliably
achieved using function minimization algorithms that
incorporate accurate information about the gradients of the likelihood
surface. Computing the gradients (i.e. partial derivatives
of the likelihood with respect to all model variables) must also be done with
the same accuracy as the likelihood computation itself.
Fournier developed a protocol for writing code to compute the required
derivatives based on the chain rule of differential calculus. This
protocol is very similar to the suite of methods that came to be known
as ``reverse mode automatic differentiation
.
The statistical models using these methods
typically included eight constituent code segments:
Model developers are usually only interested in the first of these
constituents. Any programming tools that can reduce the overhead of
developing and maintaining the other seven will greatly increase their
productivity.
Bjarne Stroustrup began development of C++ in the 1970s at Bell Labs as an
enhancement to the C programming language. C++ spread widely, and by
1989, C++ compilers were available for personal computers.
The polymorphism of C++ makes it possible to envisage a programming
system in which all mathematical operators and functions can be
overloaded to automatically compute the derivative contributions of
every differentiable numerical operation in any computer program.
by 1990 the AUTODIF Library included special classes for derivative computation and
the requisite overloaded functions for all C++ operators and
all functions in the standard C++ math library.
The AUTODIF Library automatically computes the derivatives of the objective function
with the same accuracy as the objective function itself and thereby
frees the developer from the onerous task of writing
and maintaining derivative code for statistical models.
Equally important from the standpoint of model development,
the AUTODIF Library includes a "gradient stack",
a quasi-Newton function minimizer, a derivative checker, and
container classes for vectors and matrices.
The first application of the AUTODIF Library was published in 1992
The AUTODIF Library does not, however, completely liberate the developer from
writing all of the model constituents listed above. In 1993, Fournier further
abstracted the writing of statistical models by creating ADMB, a special
"template" language to simplify model specification by
creating the tools to
transform models written using the templates into the AUTODIF Library
applications. ADMB produces code to manage the exchange of model
parameters between the model and the function minimizer,
automatically computes the Hessian matrix and inverts it to provide
an estimate the covariance of the estimated parameters. ADMB thus
completes the liberation of the model developer from all of the tedious
overhead of managing non-linear optimization, thereby freeing him or her to
focus on the more interesting aspects of the statistical model.
By the mid 1990s, ADMB had earned acceptance by researchers working on
all aspects of resource management. Population models based on the
ADMB are used to monitor a range of both endangered
species and commercially valuable fish populations including
whales, dolphins,
sea lions, penguins, albatross, abalone, lobsters, tunas, marlins,
sharks, rays, anchovy, and pollock. ADMB has also been
used to reconstruct movements of many species of animals tracked with
electronic tags.
In 2002, Fournier teamed up with Hans Skaug to introduce random
effects into ADMB. This
development included automatic computation of second and third
derivatives and the use of forward mode automatic differentiation followed by
two sweeps of reverse model AD in certain cases.
Maunder and Anders Nielsen became concerned about ADMB's long-term
development and maintenance. An agreement was reached with
Otter Research to sell the copyright to ADMB for the purpose of
making ADMB an open-source project and distributing it without
charge. The non-profit ADMB Foundation was created
to coordinate development and promote use of ADMB.
The ADMB Foundation drafted a proposal to the Gordon and Betty Moore
Foundation for the funds to purchase ADMB from Otter Research.
The Moore Foundation provided a grant to
the National Center of Ecological Analysis and Synthesis
at the University of California at Santa Barbara
in late 2007 so that the Regents of the University of California could
purchase the rights to ADMB.
The purchase was completed in mid 2008, and the complete ADMB libraries were posted
on the ADMB Project website in December 2008. By May 2009, more
than 3000 downloads of the libraries had occurred. The
source code was made available in December 2009. In mid 2010,
ADMB was supported on all common operating systems (Windows,
Linux, MacOS and Sun/SPARC), for all common C++ compilers
(GCC, Visual Studio, Borland), and for both 32 and 64 bit
architectures.
ADMB Foundation efforts during the first two years of the ADMB
Project have focused on
automating the building of ADMB for different platforms,
streamlining installation, and creation of
a user-friendly working environments. Planned technical
developments
include parallelization of internal computations,
implementation of hybrid MCMC, and improvement of the large sparse matrix
for use in random effects models.
Free and open source software
Free and open-source software or free/libre/open-source software is software that is liberally licensed to grant users the right to use, study, change, and improve its design through the availability of its source code...
suite for non-linear statistical
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
modeling. It was created by David Fournier and now being developed by the ADMB Project, a creation of the non-profit ADMB Foundation. The "AD" in AD Model Builder refers to the automatic differentiation
Automatic differentiation
In mathematics and computer algebra, automatic differentiation , sometimes alternatively called algorithmic differentiation, is a set of techniques to numerically evaluate the derivative of a function specified by a computer program...
capabilities that come from the AUTODIF Library, a C++ language extension also created by David Fournier, which implements reverse mode automatic differentiation. A related software package, ADMB-RE, provides additional support for modeling random effects.
Features and use
Markov chain Monte CarloMarkov chain Monte Carlo
Markov chain Monte Carlo methods are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a large number of steps is then used as a sample of the...
methods are integrated into the ADMB software, making it useful for Bayesian
Bayesian probability
Bayesian probability is one of the different interpretations of the concept of probability and belongs to the category of evidential probabilities. The Bayesian interpretation of probability can be seen as an extension of logic that enables reasoning with propositions, whose truth or falsity is...
modeling. In addition to Bayesian hierarchical models, ADMB provides support for modeling random effects in a frequentist framework using Laplace approximation and importance sampling.
ADMB is widely used by scientists in academic institutions, government agencies, and international commissions, most commonly for ecological modeling. In particular, many fisheries stock assessment
Stock assessment
Stock assessments provide fisheries managers with the information that is used in the regulation of a fish stock. Biological and fisheries data are collected in a stock assessment.A wide array of biological data may be collected for an assessment...
models have been built using this software. ADMB is freely available under the New BSD License,
with versions available for Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
, Mac OS X
Mac OS X
Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
, and OpenSolaris
OpenSolaris
OpenSolaris was an open source computer operating system based on Solaris created by Sun Microsystems. It was also the name of the project initiated by Sun to build a developer and user community around the software...
operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
s. Source code for ADMB was made publicly available in March, 2009.
Implementation
Work by David Fournier in the 1970s on development of highly parameterizedintegrated statistical models in fisheries motivated the
development of the AUTODIF Library, and ultimately ADMB.
The likelihood equations
in these models are typically non-linear and estimates of the
parameters are
obtained by numerical methods.
Early in Fournier's work, it became clear that general numerical
solutions to these likelihood problems could only be reliably
achieved using function minimization algorithms that
incorporate accurate information about the gradients of the likelihood
surface. Computing the gradients (i.e. partial derivatives
of the likelihood with respect to all model variables) must also be done with
the same accuracy as the likelihood computation itself.
Fournier developed a protocol for writing code to compute the required
derivatives based on the chain rule of differential calculus. This
protocol is very similar to the suite of methods that came to be known
as ``reverse mode automatic differentiation
.
The statistical models using these methods
typically included eight constituent code segments:
- the objective function;
- adjoint code to compute the partial derivatives of the objective function with respect to the parameters to be estimated;
- dedicated memory to contain intermediate data for derivative computations, known as the "gradient stack", and the software to manage it;
- a function minimizer;
- an algorithm to check that the derivatives are correct with respect to finite difference approximations;
- an algorithm to insert model parameters into a vector that can be manipulated by the function minimizer and the corresponding derivative code;
- an algorithm to return the parameter values to the likelihood computation and the corresponding derivative code; and
- an algorithm to compute the second partial derivativePartial derivativeIn mathematics, a partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held constant...
s of the objective unction with respect to the parameters to be estimated, the Hessian matrixHessian matrixIn mathematics, the Hessian matrix is the square matrix of second-order partial derivatives of a function; that is, it describes the local curvature of a function of many variables. The Hessian matrix was developed in the 19th century by the German mathematician Ludwig Otto Hesse and later named...
.
Model developers are usually only interested in the first of these
constituents. Any programming tools that can reduce the overhead of
developing and maintaining the other seven will greatly increase their
productivity.
Bjarne Stroustrup began development of C++ in the 1970s at Bell Labs as an
enhancement to the C programming language. C++ spread widely, and by
1989, C++ compilers were available for personal computers.
The polymorphism of C++ makes it possible to envisage a programming
system in which all mathematical operators and functions can be
overloaded to automatically compute the derivative contributions of
every differentiable numerical operation in any computer program.
Otter Research
Fournier formed Otter Research Ltd. in 1989, andby 1990 the AUTODIF Library included special classes for derivative computation and
the requisite overloaded functions for all C++ operators and
all functions in the standard C++ math library.
The AUTODIF Library automatically computes the derivatives of the objective function
with the same accuracy as the objective function itself and thereby
frees the developer from the onerous task of writing
and maintaining derivative code for statistical models.
Equally important from the standpoint of model development,
the AUTODIF Library includes a "gradient stack",
a quasi-Newton function minimizer, a derivative checker, and
container classes for vectors and matrices.
The first application of the AUTODIF Library was published in 1992
The AUTODIF Library does not, however, completely liberate the developer from
writing all of the model constituents listed above. In 1993, Fournier further
abstracted the writing of statistical models by creating ADMB, a special
"template" language to simplify model specification by
creating the tools to
transform models written using the templates into the AUTODIF Library
applications. ADMB produces code to manage the exchange of model
parameters between the model and the function minimizer,
automatically computes the Hessian matrix and inverts it to provide
an estimate the covariance of the estimated parameters. ADMB thus
completes the liberation of the model developer from all of the tedious
overhead of managing non-linear optimization, thereby freeing him or her to
focus on the more interesting aspects of the statistical model.
By the mid 1990s, ADMB had earned acceptance by researchers working on
all aspects of resource management. Population models based on the
ADMB are used to monitor a range of both endangered
species and commercially valuable fish populations including
whales, dolphins,
sea lions, penguins, albatross, abalone, lobsters, tunas, marlins,
sharks, rays, anchovy, and pollock. ADMB has also been
used to reconstruct movements of many species of animals tracked with
electronic tags.
In 2002, Fournier teamed up with Hans Skaug to introduce random
effects into ADMB. This
development included automatic computation of second and third
derivatives and the use of forward mode automatic differentiation followed by
two sweeps of reverse model AD in certain cases.
ADMB Project
In 2007, a group of ADMB users that included John Sibert, MarkMaunder and Anders Nielsen became concerned about ADMB's long-term
development and maintenance. An agreement was reached with
Otter Research to sell the copyright to ADMB for the purpose of
making ADMB an open-source project and distributing it without
charge. The non-profit ADMB Foundation was created
to coordinate development and promote use of ADMB.
The ADMB Foundation drafted a proposal to the Gordon and Betty Moore
Foundation for the funds to purchase ADMB from Otter Research.
The Moore Foundation provided a grant to
the National Center of Ecological Analysis and Synthesis
at the University of California at Santa Barbara
in late 2007 so that the Regents of the University of California could
purchase the rights to ADMB.
The purchase was completed in mid 2008, and the complete ADMB libraries were posted
on the ADMB Project website in December 2008. By May 2009, more
than 3000 downloads of the libraries had occurred. The
source code was made available in December 2009. In mid 2010,
ADMB was supported on all common operating systems (Windows,
Linux, MacOS and Sun/SPARC), for all common C++ compilers
(GCC, Visual Studio, Borland), and for both 32 and 64 bit
architectures.
ADMB Foundation efforts during the first two years of the ADMB
Project have focused on
automating the building of ADMB for different platforms,
streamlining installation, and creation of
a user-friendly working environments. Planned technical
developments
include parallelization of internal computations,
implementation of hybrid MCMC, and improvement of the large sparse matrix
for use in random effects models.
See also
- List of statistical packages
- List of numerical analysis software
- Comparison of numerical analysis softwareComparison of numerical analysis softwareThe following tables provide a comparison of numerical analysis software.- General :- Operating system support :The operating systems the software can run on natively .- Language features :Colors indicate features available as...
External links
- The ADMB Project
- The ADMB Foundation
- Otter Research Ltd, the company which produced ADMB prior to the ADMB Foundation
- Google Code site for ADMB