Phoronix Test Suite
Encyclopedia
Phoronix Test Suite is a free, open-source benchmark
software for Linux
and other operating systems developed by Phoronix Media
with cooperation from an undisclosed number of hardware and software vendors.
The Phoronix Test Suite has been endorsed by sites such as Linux.com
, LinuxPlanet and has been called "the best benchmarking platform" by Softpedia
. The Phoronix Test Suite is also used by Tom's Hardware, ASELabs and other review site
s.
system for the Phoronix Test Suite. It does automatic scheduling of tests. It is aimed at the enterprise. It can manage multiple test nodes simultaneously within a test farm or distributed environment.
on a daily basis, Fedora
Rawhide and Ubuntu
.
. It is designed for testing/benchmarking computers from a LiveDVD / LiveUSB environment.
for accessing software and hardware information.
On 2008-06-05 Phoronix Test Suite 1.0 was released under the codename Trondheim. This 1.0 release was made up of 57 test profiles and 23 test suites.
On 2008-09-03, Phoronix Test Suite 1.2 was released with support for the OpenSolaris
operating system and a module framework accompanied by tests focusing upon new areas and many new test profiles.
Phoronix Test Suite 1.8 includes a graphical user interface (GUI) using GTK+
written using the PHP-GTK
bindings.
3.4 includes MATISK benchmarking module and initial support for the GNU Hurd
.
uses Phoronix Test Suite to compare performance of different operating systems - Linux distribution
s, OpenSolaris
and FreeBSD
. Most controversies focus on the third party tests within Phoronix Test Suites and not the tests themselves. Selected controversies are covered below.
Benchmark (computing)
In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it...
software for Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
and other operating systems developed by Phoronix Media
Phoronix
Phoronix is a technology website that offers product reviews, Linux distribution screenshots, interviews, and news while maintaining a pure Linux orientation. Phoronix was started in June 2004 by Michael Larabel, who currently serves as the owner and editor-in-chief. The name Phoronix is a...
with cooperation from an undisclosed number of hardware and software vendors.
The Phoronix Test Suite has been endorsed by sites such as Linux.com
Linux.com
Linux.com is the name of a website owned by the Linux Foundation. The site is currently a central source for Linux information, software, documentation and answers across the server, desktop/netbook, mobile, and embedded areas. Linux.com offers free Linux tutorials, news and blogs, discussion...
, LinuxPlanet and has been called "the best benchmarking platform" by Softpedia
Softpedia
Softpedia is an Internet web site that indexes information and provides software downloads. The site also covers major technology, science, health, and entertainment news from both external and in-house sources and provides software and game reviews. , it is one of the top 500 websites according to...
. The Phoronix Test Suite is also used by Tom's Hardware, ASELabs and other review site
Review site
A review site is a website on which reviews can be posted about people, businesses, products, or services. These sites may use Web 2.0 techniques to gather reviews from site users or may employ professional writers to author reviews on the topic of concern for the site...
s.
Features
- Supports over 130 test profiles and over 60 test suites;
- Uses an XML-based testing architecture. Tests include MEncoderMEncoderMEncoder is a free command line video decoding, encoding and filtering tool released under the GNU General Public License. It is a close sibling to MPlayer and can convert all the formats that MPlayer understands into a variety of compressed and uncompressed formats using different codecs.MEncoder...
, FFmpegFFmpegFFmpeg is a free software project that produces libraries and programs for handling multimedia data. The most notable parts of FFmpeg are libavcodec, an audio/video codec library used by several other projects, libavformat, an audio/video container mux and demux library, and the ffmpeg command line...
and lm sensorsLm sensorslm_sensors , a free open source software-tool for Linux, provides tools and drivers for monitoring temperatures, voltage, and fans...
along with OpenGL games such as Doom 3Doom 3Doom 3 is a science fiction horror video game developed by id Software and published by Activision. An example of the first-person shooter genre, Doom 3 was first released for Microsoft Windows on August 3, 2004. The game was later adapted for Linux, as well as being ported by Aspyr Media for Mac...
, NexuizNexuizNexuiz is a free first-person shooter video game developed and published by Alientrap Software. It is free and open-source software and is distributed under the GNU General Public License . Version 1.0 of the game was released on May 31, 2005. The current version, 2.5.2, was released on October 1,...
, and Enemy Territory: Quake WarsEnemy Territory: Quake WarsEnemy Territory: Quake Wars is a first-person shooter video game, and is the follow-up to the 2005 title Quake 4. It is also the first game in the series to be rated T by the ESRB...
, and many others.; - Contains a feature called PTS Global where users are able to upload their test results and system information for sharing. Then through executing a single command, other users can compare their test results to a selected system in an easy-comparison mode.;
- Allow report benchmark results to the Phoronix Global online database;
- Allow compare results side-by-side;
- Is extensible and new tests can be added easily;
- Can do anonymous usage reporting;
- Can do automated GitGit (software)Git is a distributed revision control system with an emphasis on speed. Git was initially designed and developed by Linus Torvalds for Linux kernel development. Every Git working directory is a full-fledged repository with complete history and full revision tracking capabilities, not dependent on...
bisecting on a performance basis to find performance regressions. It features statistical significanceStatistical significanceIn statistics, a result is called statistically significant if it is unlikely to have occurred by chance. The phrase test of significance was coined by Ronald Fisher....
verification.
Phoromatic
Phoromatic is an web-based remote test managementTest management
Test management is the activity of managing some tests. A test management tool is software used to manage tests that have been previously specified. It is often associated with automation software...
system for the Phoronix Test Suite. It does automatic scheduling of tests. It is aimed at the enterprise. It can manage multiple test nodes simultaneously within a test farm or distributed environment.
Phoromatic Tracker
Phoromatic Tracker is an extension of Phoromatic that provides a public interface into test farms. Currently their reference implementations autonomously monitor the performance of the Linux kernelLinux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
on a daily basis, Fedora
Fedora (operating system)
Fedora is a RPM-based, general purpose collection of software, including an operating system based on the Linux kernel, developed by the community-supported Fedora Project and sponsored by Red Hat...
Rawhide and Ubuntu
Ubuntu (operating system)
Ubuntu is a computer operating system based on the Debian Linux distribution and distributed as free and open source software. It is named after the Southern African philosophy of Ubuntu...
.
PTS Desktop Live
PTS Desktop Live is a stripped down x86-64 Linux distributionLinux distribution
A Linux distribution is a member of the family of Unix-like operating systems built on top of the Linux kernel. Such distributions are operating systems including a large collection of software applications such as word processors, spreadsheets, media players, and database applications...
. It is designed for testing/benchmarking computers from a LiveDVD / LiveUSB environment.
Phodevi
Phodevi (Phoronix Device Interface) is a library that provides a clean, stable, platform-independent APIApplication programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
for accessing software and hardware information.
PCQS
Phoronix Certification & Qualification Suite (PCQS) is a reference specification for the Phoronix Test Suite.Release history
| Version | Codename | Date |
|---|---|---|
| 3.4 | Lillesand | September 8, 2011 |
| 3.2 | Grimstad | June 15, 2011 |
| 3.0 | Iveland | February 26, 2011 |
| 2.8 | Torsken | August 31, 2010 |
| 2.6 | Lyngen | May 24, 2010 |
| 2.4 | Lenvik | February 2, 2010 |
| 2.2 | Bardu | November 16, 2009 |
| 2.0 | Sandtorg | August 4, 2009 |
| 1.8 | Selbu | April 6, 2009 |
| 1.6 | Tydal | January 20, 2009 |
| 1.4 | Orkdal | November 3, 2008 |
| 1.2 | Malvik | September 3, 2008 |
| 1.0 | Trondheim | June 5, 2008 |
On 2008-06-05 Phoronix Test Suite 1.0 was released under the codename Trondheim. This 1.0 release was made up of 57 test profiles and 23 test suites.
On 2008-09-03, Phoronix Test Suite 1.2 was released with support for the OpenSolaris
OpenSolaris
OpenSolaris was an open source computer operating system based on Solaris created by Sun Microsystems. It was also the name of the project initiated by Sun to build a developer and user community around the software...
operating system and a module framework accompanied by tests focusing upon new areas and many new test profiles.
Phoronix Test Suite 1.8 includes a graphical user interface (GUI) using GTK+
GTK+
GTK+ is a cross-platform widget toolkit for creating graphical user interfaces. It is licensed under the terms of the GNU LGPL, allowing both free and proprietary software to use it. It is one of the most popular toolkits for the X Window System, along with Qt.The name GTK+ originates from GTK;...
written using the PHP-GTK
PHP-GTK
PHP-GTK is a set of language bindings for PHP which allow GTK+ GUI applications to be written in PHP. PHP-GTK provides an object-oriented interface to GTK+ classes and functions.- History:...
bindings.
3.4 includes MATISK benchmarking module and initial support for the GNU Hurd
GNU Hurd
GNU Hurd is a free software Unix-like replacement for the Unix kernel, released under the GNU General Public License. It has been under development since 1990 by the GNU Project of the Free Software Foundation...
.
Controversies
PhoronixPhoronix
Phoronix is a technology website that offers product reviews, Linux distribution screenshots, interviews, and news while maintaining a pure Linux orientation. Phoronix was started in June 2004 by Michael Larabel, who currently serves as the owner and editor-in-chief. The name Phoronix is a...
uses Phoronix Test Suite to compare performance of different operating systems - Linux distribution
Linux distribution
A Linux distribution is a member of the family of Unix-like operating systems built on top of the Linux kernel. Such distributions are operating systems including a large collection of software applications such as word processors, spreadsheets, media players, and database applications...
s, OpenSolaris
OpenSolaris
OpenSolaris was an open source computer operating system based on Solaris created by Sun Microsystems. It was also the name of the project initiated by Sun to build a developer and user community around the software...
and FreeBSD
FreeBSD
FreeBSD is a free Unix-like operating system descended from AT&T UNIX via BSD UNIX. Although for legal reasons FreeBSD cannot be called “UNIX”, as the direct descendant of BSD UNIX , FreeBSD’s internals and system APIs are UNIX-compliant...
. Most controversies focus on the third party tests within Phoronix Test Suites and not the tests themselves. Selected controversies are covered below.
Compiler focus
The compiler used in building the source based tests have a large impact on the results. Depending on the readers point of view this is seen as a detriment to the value of the results. However other readers may see that as a realistic expectation of performance for that particular system with that operating system.Obvious vs non-obvious
Also, some of the tests that claim to e.g. measure file system performance, in fact measure speed of operations that never happen in real applications. An example of this can be "Threaded I/O Tester", which doesn't actually measure any I/O performance - it measures time required to read from a single empty, sparse file, which is an obviously pointless operation. It would be trivial to change this benchmark to measure something sensible - for example, reads from a file containing data - but such tests were not published by Phoronix, for reasons unknown.See also
- InquisitorInquisitor (hardware testing software)Inquisitor is a software suite used for hardware diagnostics, stress testing, certification and benchmarking platform. It is available in 3 formats:...
- PhoronixPhoronixPhoronix is a technology website that offers product reviews, Linux distribution screenshots, interviews, and news while maintaining a pure Linux orientation. Phoronix was started in June 2004 by Michael Larabel, who currently serves as the owner and editor-in-chief. The name Phoronix is a...