Build Automation
Encyclopedia
Build automation is the act of scripting
or automating
a wide variety of tasks that software developer
s do in their day-to-day activities including things like:
" which allowed some source code dependency management as well as incremental build processing. This was the beginning of Build Automation. Its primary focus was on automating the calls to the compilers and linkers. As the build process grew more complex, developers began adding pre and post actions around the calls to the compilers such as a check-out from version control to the copying of deployable objects to a test location. The term "build automation" now includes managing the pre and post compile and link activities as well as the compile and link activities.
builds where frequent calls to the compile process are required and incremental build processing is needed.
Not all build automation tools can perform distributed builds. Most only provide distributed processing support. In addition, most solutions that do support distributed builds can only handle C or C++. Build automation solutions that support distributed processing are often make based and many do not support Maven
or Ant
.
An example of a distributed build solution is Xoreax's IncrediBuild
for the Microsoft Visual Studio platform or the open-source CMake. These may require particular configurations of a product environment so that it can run successfully on a distributed platform—library locations, environment variables, and so forth.
Optional requirements:
Scripting language
A scripting language, script language, or extension language is a programming language that allows control of one or more applications. "Scripts" are distinct from the core code of the application, as they are usually written in a different language and are often created or at least modified by the...
or automating
Automation
Automation is the use of control systems and information technologies to reduce the need for human work in the production of goods and services. In the scope of industrialization, automation is a step beyond mechanization...
a wide variety of tasks that software developer
Software developer
A software developer is a person concerned with facets of the software development process. Their work includes researching, designing, developing, and testing software. A software developer may take part in design, computer programming, or software project management...
s do in their day-to-day activities including things like:
- compiling computer source codeSource codeIn computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...
into binary code - packagingSoftware package (installation)In package management systems, which are commonly used with Linux-based operating systems, a package is a specific piece of software which the system can install and uninstall....
binary code - running tests
- deploymentSoftware deploymentSoftware deployment is all of the activities that make a software system available for use.The general deployment process consists of several interrelated activities with possible transitions between them. These activities can occur at the producer site or at the consumer site or both...
to production systems - creating documentation and/or release notesRelease notesRelease notes are documents that are distributed with software products, often when the product is still in the development or test state...
History
Historically, developers used build automation to call compilers and linkers from inside a build script versus attempting to make the compiler calls from the command line. It is simple to use the command line to pass a single source module to a compiler and then to a linker to create the final deployable object. However, when attempting to compile and link many source code modules, in a particular order, using the command line process is not a reasonable solution. The make scripting language offered a better alternative. It allowed a build script to be written to call in a series, the needed compile and link steps to build a software application. GNU Make also offered additional features such as "makedependMakedepend
makedepend is a Unix tool used to generate dependencies of C source files.-History:makedepend was developed as part of MIT's Project Athena. It was used extensively in building X11 and ancillary packages, but has since become superseded by the dependency generation facilities of various compilers,...
" which allowed some source code dependency management as well as incremental build processing. This was the beginning of Build Automation. Its primary focus was on automating the calls to the compilers and linkers. As the build process grew more complex, developers began adding pre and post actions around the calls to the compilers such as a check-out from version control to the copying of deployable objects to a test location. The term "build automation" now includes managing the pre and post compile and link activities as well as the compile and link activities.
New breed of solutions
In recent years, build management solutions have provided even more relief when it comes to automating the build process. Both commercial and open source solutions are available to perform more automated build and workflow processing. Some solutions focus on automating the pre and post steps around the calling of the build scripts, while others go beyond the pre and post build script processing and drive down into streamlining the actual compile and linker calls without much manual scripting. These tools are particularly useful for continuous integrationContinuous integration
In software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently...
builds where frequent calls to the compile process are required and incremental build processing is needed.
Advanced build automation
Advanced build automation offers remote agent processing for distributed builds and/or distributed processing. The term "distributed builds" means that the actual calls to the compiler and linkers can be served out to multiple locations for improving the speed of the build. This term is often confused with "distributed processing". Distributed processing means that each step in a process or workflow can be sent to a different machine for execution. For example, a post step to the build may require the execution of multiple test scripts on multiple machines. Distributed processing can send the different test scripts to different machines. Distributed processing is not distributed builds. Distributed processing cannot take a make, ant or maven script, break it up and send it to different machines for compiling and linking. The distributed build process must have the machine intelligence to understand the source code dependencies in order to send the different compile and link steps to different machines. A build automation solution must be able to manage these dependencies in order to perform distributed builds. Some build tools can discover these relationships programmatically (Rational ClearMake distributed, Electric Cloud ElectricAccelerator), while others depend on user-configured dependencies (Platform LSF lsmake) Build automation that can sort out source code dependency relationships can also be configured to run the compile and link activities in a parallelized mode. This means that the compiler and linkers can be called in multi-threaded mode using a machine that is configured with more than one core.Not all build automation tools can perform distributed builds. Most only provide distributed processing support. In addition, most solutions that do support distributed builds can only handle C or C++. Build automation solutions that support distributed processing are often make based and many do not support Maven
Apache Maven
Maven is a build automation and software comprehension tool. While primarily used for Java programming, it can also be used to build and manage projects written in C#, Ruby, Scala, and other languages. Maven serves a similar purpose to the Apache Ant tool, but it is based on different concepts and...
or Ant
Apache Ant
Apache Ant is a software tool for automating software build processes. It is similar to Make but is implemented using the Java language, requires the Java platform, and is best suited to building Java projects....
.
An example of a distributed build solution is Xoreax's IncrediBuild
for the Microsoft Visual Studio platform or the open-source CMake. These may require particular configurations of a product environment so that it can run successfully on a distributed platform—library locations, environment variables, and so forth.
Advantages
- Improve product quality
- Accelerate the compile and link processing
- Eliminate redundant tasks
- Minimize "bad builds"
- Eliminate dependencies on key personnel
- Have history of builds and releases in order to investigate issues
- Save time and money - because of the reasons listed above.
Types
- On-Demand automation such as a user running a scriptScripting languageA scripting language, script language, or extension language is a programming language that allows control of one or more applications. "Scripts" are distinct from the core code of the application, as they are usually written in a different language and are often created or at least modified by the...
at the command line - Scheduled automation such as a continuous integrationContinuous integrationIn software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently...
server running a nightly build - Triggered automation such as a continuous integrationContinuous integrationIn software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently...
server running a build on every commit to a version control system.
Makefile
One specific form of build automation is the automatic generation of Makefiles. This is accomplished by tools like- GNU Automake
- CMakeCMakeCMake is a cross-platform, open-source system for managing the build process of software using a compiler-independent method. It is designed to support directory hierarchies and applications that depend on multiple libraries, and for use in conjunction with native build environments such as Make,...
- imakeImakeimake is a build automation system implemented on top of the C preprocessor.imake generates makefiles from a template, a set of cpp macro functions, and a per-directory input file called an Imakefile...
- qmakeQmakeqmake is a utility that automates the generation of Makefiles. Makefiles are used by the program make to build executable programs from source code; therefore qmake is a make-makefile tool, or makemake for short....
- nmake
- Apache AntApache AntApache Ant is a software tool for automating software build processes. It is similar to Make but is implemented using the Java language, requires the Java platform, and is best suited to building Java projects....
- Apache MavenApache MavenMaven is a build automation and software comprehension tool. While primarily used for Java programming, it can also be used to build and manage projects written in C#, Ruby, Scala, and other languages. Maven serves a similar purpose to the Apache Ant tool, but it is based on different concepts and...
- PremakePremakeIn software development, premake is a open source utility for automatically building configuration from source code.- Features :Some of the features of the system are:* It supports C, C++, and C# languages source code.* It has a simple syntax...
Requirements of a build system
Basic requirements:- Frequent or overnight builds to catch problems early.
- Support for Source Code Dependency Management
- Incremental build processing
- Reporting that traces source to binary matching
- Build acceleration
- Extraction and reporting on build compile and link usage
Optional requirements:
- Generate release notes and other documentation such as help pages
- Build status reporting
- Test pass or fail reporting
- Summary of the features added/modified/deleted with each new build
See also
- BuildBotBuildBotBuildBot is a software development continuous integration tool which automates the compile/test cycle required to validate changes to the project code base...
- Continuous integrationContinuous integrationIn software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently...
- Product family engineeringProduct Family EngineeringProduct family engineering , also known as product line engineering, is a synonym for "domain engineering" created by the Software Engineering Institute, a term coined by James Neighbors in his 1980 dissertation at University of California, Irvine...
- Rake (software)Rake (software)Rake is a software task management tool. It allows you to specify tasks and describe dependencies as well as to group tasks in a namespace.It is similar to SCons and make, but has a number of differences. The tool is written in the Ruby programming language, and the Rakefiles use Ruby syntax...
- Release engineeringRelease engineeringRelease engineering, frequently abbreviated as "releng", is a sub-discipline in software engineering concerned with the compilation, assembly, and delivery of source code into finished products or other software components. Associated with the software release life cycle, it is often said that...
- Scripting languageScripting languageA scripting language, script language, or extension language is a programming language that allows control of one or more applications. "Scripts" are distinct from the core code of the application, as they are usually written in a different language and are often created or at least modified by the...
- Software configuration managementSoftware configuration managementIn software engineering, software configuration management is the task of tracking and controlling changes in the software. Configuration management practices include revision control and the establishment of baselines....
- Unit testing