ESMF
Encyclopedia
The ESMF is open source software for building climate
, numerical weather prediction
, data assimilation
, and other Earth science
software applications. These applications are computationally demanding and usually run on supercomputers. The ESMF project is distinguished by its strong emphasis on community governance
and distributed development
, and by a diverse customer base that includes modeling groups from universities, major U.S. research centers, the National Weather Service
, the Department of Defense
, and NASA
. The ESMF development team was centered at NCAR until 2009, after which it moved to the NOAA Earth System Research Laboratory
.
of Earth science modeling software developed at different sites and promotes code reuse
. The idea is to transform distributed, specialized knowledge and resources into a collaborative, integrated modeling community that operates more efficiently, can address a wider variety of problems more effectively, and is more responsive to societal needs.
. The components within an ESMF software application usually represent large-scale physical domains such as the atmosphere
, ocean
, cryosphere
, or land surface. Some models also represent specific processes (e.g. ocean biogeochemistry, the impact of solar radiation on the atmosphere) as components. In ESMF, components can create and drive other components so that an ocean biogeochemistry component can be part of a larger ocean component.
The software that connects physical domains is called a coupler in the Earth system modeling community. Couplers follow the mediator pattern
and take the outputs from one component and transform them into the inputs that are needed to run another component. Transformations may include unit conversions, grid interpolation or remapping, mergers (i.e., combining land and ocean surfaces to form a completely covered global surface) or other specialized transformations. In ESMF, couplers are also software components.
Grid interpolation
and remapping are core utilities of ESMF. Interpolation weights can be generated in ESMF using bilinear interpolation
, finite element patch recovery, and conservative remapping methods.
ESMF can associate metadata
with data objects. The metadata, in the form of name and value pairs, is grouped into packages, which can be written out in XML
and other standard formats. ESMF metadata packages are based on community conventions including the Climate and Forecast Metadata Conventions
and the METAFOR
Common Information Model.
Working Group (CMIWG), an unfunded, grass-roots effort to explore ways of enhancing collaborative Earth system model development. The CMIWG attracted broad participation from major weather and climate modeling groups at research and operational centers. In a series of meetings held from 1998 to 2000, CMIWG members established general requirements and a preliminary design for a common software framework
.
In September 2000, the NASA Earth Science Technology Office (ESTO) released a solicitation that called for the creation of an ESMF. A critical mass of CMIWG participants agreed to develop a coordinated response, based on their strawman framework design, and submitted three linked proposals. The first focused on development of the core ESMF software, the second on deployment of Earth science modeling applications, and the third on deployment of ESMF data assimilation applications. All three proposals were funded, at a collective level of $9.8M over a three year period. As the ESMF project gained momentum, it replaced the CMIWG as the focal point for developing community modeling infrastructure.
During the period of NASA funding, the ESMF team developed a prototype of the framework and used it in a number of experiments that demonstrated coupling of modeling components from different institutions. ESMF was also used as the basis for the construction of a new model, the GEOS-5 atmospheric general circulation model at NASA Goddard.
As the end of the first funding cycle for ESMF neared, its collaborators wrote a Project Plan that described how ESMF could transition to an organization with multi-agency sponsorship for its next funding cycle. Major new five-year grants came from NASA, through the Modeling Analysis and Prediction (MAP) program for Climate Change
and Variability, and from the Department of Defense
Battlespace
Environments Institute. The NSF
continued funding part of the development team through NCAR core funds. Many smaller ESMF-based application adoption projects were funded in domains as diverse as space weather
and sediment
transport.
Also at the end of the first funding cycle, the ESMF collaborators wrote a white paper on Future Directions for the ESMF. This paper formed the basis for a proposal to NSF
to combine ESMF (and other software frameworks) with data services to create a computational environment that supports an end-to-end modeling workflow
. It was funded and has become the Earth System Curator project.
Climate
Climate encompasses the statistics of temperature, humidity, atmospheric pressure, wind, rainfall, atmospheric particle count and other meteorological elemental measurements in a given region over long periods...
, numerical weather prediction
Numerical weather prediction
Numerical weather prediction uses mathematical models of the atmosphere and oceans to predict the weather based on current weather conditions. Though first attempted in the 1920s, it was not until the advent of computer simulation in the 1950s that numerical weather predictions produced realistic...
, data assimilation
Data assimilation
Applications of data assimilation arise in many fields of geosciences, perhaps most importantly in weather forecasting and hydrology. Data assimilation proceeds by analysis cycles...
, and other Earth science
Earth science
Earth science is an all-embracing term for the sciences related to the planet Earth. It is arguably a special case in planetary science, the Earth being the only known life-bearing planet. There are both reductionist and holistic approaches to Earth sciences...
software applications. These applications are computationally demanding and usually run on supercomputers. The ESMF project is distinguished by its strong emphasis on community governance
Governance
Governance is the act of governing. It relates to decisions that define expectations, grant power, or verify performance. It consists of either a separate process or part of management or leadership processes...
and distributed development
Distributed development
A distributed development project is a research & development project that is done across many business worksites or locations. It is a form of R&D where the project members may not see each other face to face, but they are all working collaboratively toward the outcome of the project...
, and by a diverse customer base that includes modeling groups from universities, major U.S. research centers, the National Weather Service
National Weather Service
The National Weather Service , once known as the Weather Bureau, is one of the six scientific agencies that make up the National Oceanic and Atmospheric Administration of the United States government...
, the Department of Defense
United States Department of Defense
The United States Department of Defense is the U.S...
, and NASA
NASA
The National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research...
. The ESMF development team was centered at NCAR until 2009, after which it moved to the NOAA Earth System Research Laboratory
Earth System Research Laboratory
The Earth System Research Laboratory is a laboratory in National Oceanic and Atmospheric Administration /Office of Oceanic and Atmospheric Research ....
.
Purpose
ESMF increases the interoperabilityInteroperability
Interoperability is a property referring to the ability of diverse systems and organizations to work together . The term is often used in a technical systems engineering sense, or alternatively in a broad sense, taking into account social, political, and organizational factors that impact system to...
of Earth science modeling software developed at different sites and promotes code reuse
Code reuse
Code reuse, also called software reuse, is the use of existing software, or software knowledge, to build new software.-Overview:Ad hoc code reuse has been practiced from the earliest days of programming. Programmers have always reused sections of code, templates, functions, and procedures...
. The idea is to transform distributed, specialized knowledge and resources into a collaborative, integrated modeling community that operates more efficiently, can address a wider variety of problems more effectively, and is more responsive to societal needs.
Software architecture
ESMF is based on principles of component-based software engineeringComponent-based software engineering
Component-based software engineering is a branch of software engineering that emphasizes the separation of concerns in respect of the wide-ranging functionality available throughout a given software system...
. The components within an ESMF software application usually represent large-scale physical domains such as the atmosphere
Atmosphere
An atmosphere is a layer of gases that may surround a material body of sufficient mass, and that is held in place by the gravity of the body. An atmosphere may be retained for a longer duration, if the gravity is high and the atmosphere's temperature is low...
, ocean
Ocean
An ocean is a major body of saline water, and a principal component of the hydrosphere. Approximately 71% of the Earth's surface is covered by ocean, a continuous body of water that is customarily divided into several principal oceans and smaller seas.More than half of this area is over 3,000...
, cryosphere
Cryosphere
The cryosphere is the term which collectively describes the portions of the Earth’s surface where water is in solid form, including sea ice, lake ice, river ice, snow cover, glaciers, ice caps and ice sheets, and frozen ground . Thus there is a wide overlap with the hydrosphere...
, or land surface. Some models also represent specific processes (e.g. ocean biogeochemistry, the impact of solar radiation on the atmosphere) as components. In ESMF, components can create and drive other components so that an ocean biogeochemistry component can be part of a larger ocean component.
The software that connects physical domains is called a coupler in the Earth system modeling community. Couplers follow the mediator pattern
Mediator pattern
The mediator pattern, one of the 23 design patterns described in Design Patterns: Elements of Reusable Object-Oriented Software, provides a unified interface to a set of interfaces in a subsystem. This pattern is considered to be a behavioral pattern due to the way it can alter the program's...
and take the outputs from one component and transform them into the inputs that are needed to run another component. Transformations may include unit conversions, grid interpolation or remapping, mergers (i.e., combining land and ocean surfaces to form a completely covered global surface) or other specialized transformations. In ESMF, couplers are also software components.
Capabilities
ESMF represents user data in the form of data objects such as grids, fields, and arrays. The user data within a component may be copied or referenced into these ESMF objects. Once user data is part of an ESMF data object, framework methods can be used to transform and transfer the data as required to other components in the system. This generally happens within a coupler component.Grid interpolation
Interpolation
In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....
and remapping are core utilities of ESMF. Interpolation weights can be generated in ESMF using bilinear interpolation
Bilinear interpolation
In mathematics, bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables on a regular grid. The interpolated function should not use the term of x^2 or y^2, but x y, which is the bilinear form of x and y.The key idea is to perform linear...
, finite element patch recovery, and conservative remapping methods.
ESMF can associate metadata
Metadata
The term metadata is an ambiguous term which is used for two fundamentally different concepts . Although the expression "data about data" is often used, it does not apply to both in the same way. Structural metadata, the design and specification of data structures, cannot be about data, because at...
with data objects. The metadata, in the form of name and value pairs, is grouped into packages, which can be written out in XML
XML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
and other standard formats. ESMF metadata packages are based on community conventions including the Climate and Forecast Metadata Conventions
Climate and Forecast Metadata Conventions
The Climate and Forecast metadata conventions are conventions for the description of Earth sciences data, intended to promote the processing and sharing of data files. The metadata defined by the CF conventions are generally included in the same file as the data, thus making the file...
and the METAFOR
METAFOR
The Common Metadata for Climate Modelling Digital Repositories, or METAFOR project, is creating a Common Information Model for climate data and the models that produce it....
Common Information Model.
History
The ESMF collaboration had its roots in the Common Modeling InfrastructureCommon Modeling Infrastructure
Common modeling infrastructure refers to software libraries that can be shared across multiple institutions in order to increase software reuse and interoperability in complex modeling systems...
Working Group (CMIWG), an unfunded, grass-roots effort to explore ways of enhancing collaborative Earth system model development. The CMIWG attracted broad participation from major weather and climate modeling groups at research and operational centers. In a series of meetings held from 1998 to 2000, CMIWG members established general requirements and a preliminary design for a common software framework
Software framework
In computer programming, a software framework is an abstraction in which software providing generic functionality can be selectively changed by user code, thus providing application specific software...
.
In September 2000, the NASA Earth Science Technology Office (ESTO) released a solicitation that called for the creation of an ESMF. A critical mass of CMIWG participants agreed to develop a coordinated response, based on their strawman framework design, and submitted three linked proposals. The first focused on development of the core ESMF software, the second on deployment of Earth science modeling applications, and the third on deployment of ESMF data assimilation applications. All three proposals were funded, at a collective level of $9.8M over a three year period. As the ESMF project gained momentum, it replaced the CMIWG as the focal point for developing community modeling infrastructure.
During the period of NASA funding, the ESMF team developed a prototype of the framework and used it in a number of experiments that demonstrated coupling of modeling components from different institutions. ESMF was also used as the basis for the construction of a new model, the GEOS-5 atmospheric general circulation model at NASA Goddard.
As the end of the first funding cycle for ESMF neared, its collaborators wrote a Project Plan that described how ESMF could transition to an organization with multi-agency sponsorship for its next funding cycle. Major new five-year grants came from NASA, through the Modeling Analysis and Prediction (MAP) program for Climate Change
Climate change
Climate change is a significant and lasting change in the statistical distribution of weather patterns over periods ranging from decades to millions of years. It may be a change in average weather conditions or the distribution of events around that average...
and Variability, and from the Department of Defense
United States Department of Defense
The United States Department of Defense is the U.S...
Battlespace
Battlespace
Battlespace is a term used to signify a unified military strategy to integrate and combine armed forces for the military theatre of operations, including air, information, land, sea, and space to achieve military goals. It includes the environment, factors, and conditions that must be understood...
Environments Institute. The NSF
National Science Foundation
The National Science Foundation is a United States government agency that supports fundamental research and education in all the non-medical fields of science and engineering. Its medical counterpart is the National Institutes of Health...
continued funding part of the development team through NCAR core funds. Many smaller ESMF-based application adoption projects were funded in domains as diverse as space weather
Space weather
Space weather is the concept of changing environmental conditions in near-Earth space or thespace from the Sun's atmosphere to the Earth's atmosphere. It is distinct from the concept ofweather within the Earth's planetary atmosphere...
and sediment
Sediment
Sediment is naturally occurring material that is broken down by processes of weathering and erosion, and is subsequently transported by the action of fluids such as wind, water, or ice, and/or by the force of gravity acting on the particle itself....
transport.
Also at the end of the first funding cycle, the ESMF collaborators wrote a white paper on Future Directions for the ESMF. This paper formed the basis for a proposal to NSF
National Science Foundation
The National Science Foundation is a United States government agency that supports fundamental research and education in all the non-medical fields of science and engineering. Its medical counterpart is the National Institutes of Health...
to combine ESMF (and other software frameworks) with data services to create a computational environment that supports an end-to-end modeling workflow
Workflow
A workflow consists of a sequence of connected steps. It is a depiction of a sequence of operations, declared as work of a person, a group of persons, an organization of staff, or one or more simple or complex mechanisms. Workflow may be seen as any abstraction of real work...
. It was funded and has become the Earth System Curator project.