Test harness
Encyclopedia
In software testing
, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the Test execution engine
and the Test script repository.
Test harnesses allow for the automation of tests
. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework
.
A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.
The typical objectives of a test harness are to:
A test harness may provide some of the following benefits:
An alternative definition of a test harness is software constructed to facilitate integration testing. Where test stubs are typically components of the application under development and are replaced by working component as the application is developed (top-down design), test harnesses are external to the application being tested and simulate services or functionality not available in a test environment.
For example, if you're building an application that needs to interface with an application on a mainframe computer but none is available during development, a test harness maybe built to use as a substitute.
A test harness maybe part of a project deliverable. It’s kept outside of the application source code and maybe reused on multiple projects. Because a test harness simulates application functionality - it has no knowledge of test suites, test cases or test reports. Those things are provided by a testing framework and associated automated testing tools.
Software testing
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...
, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the Test execution engine
Test execution engine
A test execution engine is a type of software used to test software, hardware or complete systems.Synonyms of test execution engine:* Test executive* Test managerA test execution engine may appear in two forms:...
and the Test script repository.
Test harnesses allow for the automation of tests
Test automation
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions...
. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework
Test automation framework
A test automation framework is a set of assumptions, concepts and tools that provide support for automated software testing. The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated and the Driver...
.
A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.
The typical objectives of a test harness are to:
- Automate the testing process.
- Execute test suites of test cases.
- Generate associated test reports.
A test harness may provide some of the following benefits:
- Increased productivity due to automation of the testing process.
- Increased probability that regression testingRegression testingRegression testing is any type of software testing that seeks to uncover new errors, or regressions, in existing functionality after changes have been made to a system, such as functional enhancements, patches or configuration changes....
will occur. - Increased quality of software components and application.
- Ensure that subsequent test runs are exact duplicates of previous ones.
- Testing can occur at times that the office is not staffed (ie. at night)
- A test script may include conditions and/or uses that are otherwise difficult to simulate (load, for example)
An alternative definition of a test harness is software constructed to facilitate integration testing. Where test stubs are typically components of the application under development and are replaced by working component as the application is developed (top-down design), test harnesses are external to the application being tested and simulate services or functionality not available in a test environment.
For example, if you're building an application that needs to interface with an application on a mainframe computer but none is available during development, a test harness maybe built to use as a substitute.
A test harness maybe part of a project deliverable. It’s kept outside of the application source code and maybe reused on multiple projects. Because a test harness simulates application functionality - it has no knowledge of test suites, test cases or test reports. Those things are provided by a testing framework and associated automated testing tools.