Brownfield (software development)
Encyclopedia
Brownfield development is a term commonly used in the IT industry to describe problem spaces needing the development and deployment of new software systems in the immediate presence of existing (legacy) software applications/systems. This implies that any new software architecture
must take into account and coexist with live software already in situ
. In contemporary civil engineering
, Brownfield land
means places where new buildings may need to be designed and erected considering the other structures and services already in place.
Brownfield development adds a number of improvements to conventional software engineering
practices. These traditionally assume a "clean sheet of paper" or "green field" target environment throughout the design and implementation phases of software development. Brownfield extends such traditions by insisting that the context (local landscape) of the system being created be factored into any development exercise. This requires a detailed knowledge of the systems, services and data in the immediate vicinity of the solution under construction.
As a result, an increasing proportion of the effort of developing new business capabilities is spent on understanding and integrating with the existing complex system and business landscape rather than delivering value. It has been observed that up to 75% of overall project effort is now spent on software integration and migration rather than new function.
The IT industry as a whole has a poor success rate at delivering such large scale change for its clients. The CHAOS survey from the Standish Group has tracked an overall improvement in IT project delivery success over the last twenty years, but even in 2006 large IT projects still failed more often than succeeded. Engineering changes and in such environments has many parallels with the concerns of the construction industry in redeveloping industrial or contaminated sites. They are full of hazards, unexpected complexities and tend to be risky and expensive to redevelop. The accumulated complexity of IT environments has made them “Brownfield” sites.
Ironically it is not the complexity of the new function or any new system characteristics that are the root of large project failures – it is our understanding and communication of the overall requirement (as identified in The Mythical Man Month). To succeed the requirements need to include a precise and thorough understanding of the constraints of the existing business and IT. Current “Greenfield
” tooling and methods use early, informal and often imprecise abstractions that essentially ignore such complexity. However the devil is always in the detail. Early, poorly informed abstractions are usually wrong and are often detected late in construction, resulting in delays, expensive rework and even failed developments. A Brownfield oriented approach embraces existing complexity, and is used to reliably accelerate the overall solution engineering process, including enabling phased, incremental change wherever possible.
Brownfield takes the standard OMG
model/pattern driven approach and turns it on its head. Rather than taking the conventional approach of starting with a Conceptual Model and driving down to Platform Specific Models and code generation, Brownfield starts by harvesting code and other existing artifacts and uses patterns to formally abstract upwards towards the Architecture and Business tier.
Standard Greenfield techniques are then used in combination to define the preferred business target. This “meet in the middle” technique is familiar from other development methods, but the extensive use of formal abstraction and the use of patterns for both discovery and generation is novel.
The underlying conceptual architecture of all Brownfield tools is known as VITA. VITA stands for Views, Inventory, Transformation and Artifacts. In a VITA architecture, the problem definition of the target space can be maintained as separate (though related) native "headfulls" of knowledge known as Views. The core advantage of a View is that it can be based on pretty much any formal tool. Brownfield does not impose a single tool or language on a problem space – a core tenet is that the headfulls continue to be maintained in their native forms and tools.
Native Views are then brought together and linked into a single Inventory. The Inventory is then used with a series of Transformation capabilities to produce the Artifacts that the solution needs.
Views can currently be imported from a wide variety of sources including UML
, XML
sources, DDL
, spreadsheets etc. The Analysis and Renovation Catalyst tool from IBM
has taken this capability even further via the use of formal grammars and Abstract Syntax Trees to enable almost any program to be parsed and tokenized into a View for inclusion into the Inventory.
The rapid cyclic nature of the discovery, re-engineer, generate and test cycle used in this approach means that solutions can be refined iteratively in terms of their logical and physical definitions as more of the constraints become known and the solution architecture is refined.
Iterative Brownfield development can allow the gradual refinement of logical and physical architectures and incremental testing for the whole approach, resulting in development acceleration, improved solution quality and cheaper defect removal. Brownfield can also be used to generate solution documentation, ensuring it is always up to date and consistent across different viewpoints.
The Inventory that is created through Brownfield processed may be highly complex, being an interconnected multi-dimensional semantic network
. The level of knowledge in the Inventory can be very fine grained, highly detailed and interrelated. Such things are hard to understand and can provide barriers to communication however. Brownfield solves this problem by abstracting concepts via an artisan’s best guess, using known patterns in its Inventories to extract and infer higher level relationships.
Formal abstractions enable the complexity of the Inventory to be translated into simpler, but inherently accurate, representations for easier consumption by those that need to understand the problem space. These abstracted Inventory models can be used to automatically render multi-layered architecture representations in tools such as Second Life.
Such visualizations enable complex information to be shared and experienced by multiple individuals from around the globe in real time. This enhances both understanding and a sense of a single team.
Software architecture
The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both...
must take into account and coexist with live software already in situ
In situ
In situ is a Latin phrase which translated literally as 'In position'. It is used in many different contexts.-Aerospace:In the aerospace industry, equipment on board aircraft must be tested in situ, or in place, to confirm everything functions properly as a system. Individually, each piece may...
. In contemporary civil engineering
Civil engineering
Civil engineering is a professional engineering discipline that deals with the design, construction, and maintenance of the physical and naturally built environment, including works like roads, bridges, canals, dams, and buildings...
, Brownfield land
Brownfield land
Brownfield sites are abandoned or underused industrial and commercial facilities available for re-use. Expansion or redevelopment of such a facility may be complicated by real or perceived environmental contaminations. Cf. Waste...
means places where new buildings may need to be designed and erected considering the other structures and services already in place.
Brownfield development adds a number of improvements to conventional software engineering
Software engineering
Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software...
practices. These traditionally assume a "clean sheet of paper" or "green field" target environment throughout the design and implementation phases of software development. Brownfield extends such traditions by insisting that the context (local landscape) of the system being created be factored into any development exercise. This requires a detailed knowledge of the systems, services and data in the immediate vicinity of the solution under construction.
Addressing environmental complexity
Reliably reengineering existing business and IT environments into modern competitive, integrated architectures is non-trivial. The complexity of business and IT environments has been accumulating almost unchecked for forty years making changes ever more expensive. This is because:- Environmental complexity is often expressed in legacy codeLegacy codeLegacy code is source code that related to a no-longer supported or manufactured operating system or other computer technology. The term can also mean code inserted into modern software for the purpose of maintaining an older or previously supported feature — for example supporting a serial...
. Legacy skills shortages are driving up maintenance and integration costs. - Existing complex environments must be re-engineered in phases that make operational sense to their associated business function. These phases often default to wholesale, risky replacements of systems as ignorance of existing complexity means that potential incremental changes are too difficult to understand and engineer.
- Accelerated development methods have left enterprises with modern legacy systems. Complex JavaJava (programming language)Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...
and .NET.NET FrameworkThe .NET Framework is a software framework that runs primarily on Microsoft Windows. It includes a large library and supports several programming languages which allows language interoperability...
applications have many of the same problems as older COBOLCOBOLCOBOL is one of the oldest programming languages. Its name is an acronym for COmmon Business-Oriented Language, defining its primary domain in business, finance, and administrative systems for companies and governments....
applications.
As a result, an increasing proportion of the effort of developing new business capabilities is spent on understanding and integrating with the existing complex system and business landscape rather than delivering value. It has been observed that up to 75% of overall project effort is now spent on software integration and migration rather than new function.
The IT industry as a whole has a poor success rate at delivering such large scale change for its clients. The CHAOS survey from the Standish Group has tracked an overall improvement in IT project delivery success over the last twenty years, but even in 2006 large IT projects still failed more often than succeeded. Engineering changes and in such environments has many parallels with the concerns of the construction industry in redeveloping industrial or contaminated sites. They are full of hazards, unexpected complexities and tend to be risky and expensive to redevelop. The accumulated complexity of IT environments has made them “Brownfield” sites.
Ironically it is not the complexity of the new function or any new system characteristics that are the root of large project failures – it is our understanding and communication of the overall requirement (as identified in The Mythical Man Month). To succeed the requirements need to include a precise and thorough understanding of the constraints of the existing business and IT. Current “Greenfield
Greenfield project
In many disciplines a greenfield is a project that lacks any constraints imposed by prior work. The analogy is to that of construction on greenfield land where there is no need to remodel or demolish an existing structure...
” tooling and methods use early, informal and often imprecise abstractions that essentially ignore such complexity. However the devil is always in the detail. Early, poorly informed abstractions are usually wrong and are often detected late in construction, resulting in delays, expensive rework and even failed developments. A Brownfield oriented approach embraces existing complexity, and is used to reliably accelerate the overall solution engineering process, including enabling phased, incremental change wherever possible.
Brownfield takes the standard OMG
Object Management Group
Object Management Group is a consortium, originally aimed at setting standards for distributed object-oriented systems, and is now focused on modeling and model-based standards.- Overview :...
model/pattern driven approach and turns it on its head. Rather than taking the conventional approach of starting with a Conceptual Model and driving down to Platform Specific Models and code generation, Brownfield starts by harvesting code and other existing artifacts and uses patterns to formally abstract upwards towards the Architecture and Business tier.
The underlying conceptual architecture of all Brownfield tools is known as VITA. VITA stands for Views, Inventory, Transformation and Artifacts. In a VITA architecture, the problem definition of the target space can be maintained as separate (though related) native "headfulls" of knowledge known as Views. The core advantage of a View is that it can be based on pretty much any formal tool. Brownfield does not impose a single tool or language on a problem space – a core tenet is that the headfulls continue to be maintained in their native forms and tools.
Native Views are then brought together and linked into a single Inventory. The Inventory is then used with a series of Transformation capabilities to produce the Artifacts that the solution needs.
Views can currently be imported from a wide variety of sources including UML
Unified Modeling Language
Unified Modeling Language is a standardized general-purpose modeling language in the field of object-oriented software engineering. The standard is managed, and was created, by the Object Management Group...
, XML
Extensible Markup Language
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
sources, DDL
DDL
DDL may be an abbreviation for:* Don't Do Lip!*Daniel Day-Lewis, Academy Award winning Irish/English actor*Data Definition Language or Data Description Language, relating to databases...
, spreadsheets etc. The Analysis and Renovation Catalyst tool from IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
has taken this capability even further via the use of formal grammars and Abstract Syntax Trees to enable almost any program to be parsed and tokenized into a View for inclusion into the Inventory.
The rapid cyclic nature of the discovery, re-engineer, generate and test cycle used in this approach means that solutions can be refined iteratively in terms of their logical and physical definitions as more of the constraints become known and the solution architecture is refined.
Iterative Brownfield development can allow the gradual refinement of logical and physical architectures and incremental testing for the whole approach, resulting in development acceleration, improved solution quality and cheaper defect removal. Brownfield can also be used to generate solution documentation, ensuring it is always up to date and consistent across different viewpoints.
The Inventory that is created through Brownfield processed may be highly complex, being an interconnected multi-dimensional semantic network
Semantic network
A semantic network is a network which represents semantic relations among concepts. This is often used as a form of knowledge representation. It is a directed or undirected graph consisting of vertices, which represent concepts, and edges.- History :...
. The level of knowledge in the Inventory can be very fine grained, highly detailed and interrelated. Such things are hard to understand and can provide barriers to communication however. Brownfield solves this problem by abstracting concepts via an artisan’s best guess, using known patterns in its Inventories to extract and infer higher level relationships.
Formal abstractions enable the complexity of the Inventory to be translated into simpler, but inherently accurate, representations for easier consumption by those that need to understand the problem space. These abstracted Inventory models can be used to automatically render multi-layered architecture representations in tools such as Second Life.
Such visualizations enable complex information to be shared and experienced by multiple individuals from around the globe in real time. This enhances both understanding and a sense of a single team.