All Topics  
Standard Performance Evaluation Corporation

 

 
   Email Print
   Bookmark   Link
 

 

Standard Performance Evaluation Corporation
 
 
  Topics Standard Performance Evaluation Corporation Topic Home

The Standard Performance Evaluation Corporation (SPEC) is a non-profit organization
Non-profit organization

A nonprofit organization is any organization that does not aim to make a profit, and which is not a public body....
 that aims to produce "fair, impartial and meaningful benchmark
Benchmark (computing)

In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it....
s for computers." SPEC was founded in 1988 and their goal is to ensure that the marketplace has a fair and useful set of metrics to differentiate candidate systems. Its member organizations includes leading computer and software manufacturers. SPEC benchmarks are widely used today in evaluating the performance of computer systems; the results are published on the SPEC web site.






Discussion
Ask a question about 'Standard Performance Evaluation Corporation'
Start a new discussion about 'Standard Performance Evaluation Corporation'
Answer questions from other users
Full Discussion Forum



Encyclopedia


The Standard Performance Evaluation Corporation (SPEC) is a non-profit organization
Non-profit organization

A nonprofit organization is any organization that does not aim to make a profit, and which is not a public body....
 that aims to produce "fair, impartial and meaningful benchmark
Benchmark (computing)

In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it....
s for computers." SPEC was founded in 1988 and their goal is to ensure that the marketplace has a fair and useful set of metrics to differentiate candidate systems. Its member organizations includes leading computer and software manufacturers. SPEC benchmarks are widely used today in evaluating the performance of computer systems; the results are published on the SPEC web site. Results are informally referred to as "SPECmarks", results or even just "SPEC".

The benchmarks aim to test "real-life" situations. SPECweb2005, for example, tests web server
Web server

The term web server can mean one of two things:# A computer program that is responsible for accepting Hypertext Transfer Protocol requests from clients , and Server them HTTP responses along with optional data contents, which usually are web pages such as Hypertext Markup Language documents and linked objects ....
 performance by performing various types of parallel HTTP requests, and SPEC CPU tests CPU
Central processing unit

A central processing unit is an electronic circuit that can execute computer programs. This broad definition can easily be applied to many early computers that existed long before the term "CPU" ever came into widespread usage....
 performance by measuring the run time of several programs such as the compiler
Compiler

A compiler is a computer program that transforms source code written in a programming language into another computer language . The most common reason for wanting to transform source code is to create an executable program....
 gcc
GNU Compiler Collection

The GNU Compiler Collection is a compiler system produced by the GNU Project supporting various programming languages. GCC is a key component of the GNU toolchain....
 and the chess program crafty
Crafty

Crafty is a chess program written by University of Alabama at Birmingham professor Robert Hyatt. It is directly derived from Cray Blitz, winner of the 1983 and 1986 World Computer Chess Championships....
. The various tasks are assigned weights based on their perceived importance; these weights are used to compute a single benchmark result in the end.

SPEC benchmarks are written in a platform neutral programming language
Programming language

A programming language is a machine-readable artificial language designed to express computations that can be performed by a machine, particularly a computer....
 (usually C
C (programming language)

C is a general-purpose computer programming language originally developed in 1972 by Dennis Ritchie at the Bell Telephone Laboratories to implement the Unix operating system....
 or Fortran
Fortran

Fortran is a general-purpose programming language, procedural programming language, imperative programming language programming language that is especially suited to numerical analysis and scientific computing....
), and the interested parties may compile
Compiler

A compiler is a computer program that transforms source code written in a programming language into another computer language . The most common reason for wanting to transform source code is to create an executable program....
 the code using whatever compiler they prefer for their platform, but may not change the code. Manufacturers have been known to optimize their compilers to improve performance of the various SPEC benchmarks.

In order to use a benchmark, a license has to be purchased from SPEC; the costs vary from test to test with a typical range from several hundred to several thousand dollars. This pay-for-license model might seem to be in violation of the GPL
GNU General Public License

The GNU General Public License is a widely used free software license, originally written by Richard Stallman for the GNU project. The GPL is the most popular and well-known example of the type of strong copyleft license that requires derived works to be available under the same copyleft....
 as the benchmarks include software such as GCC
GNU Compiler Collection

The GNU Compiler Collection is a compiler system produced by the GNU Project supporting various programming languages. GCC is a key component of the GNU toolchain....
 that is licensed by the GPL. However, the GPL does not require software to be distributed for free, only that recipients be allowed to redistribute any GPLed software that they receive; the license agreement for SPEC specifically exempts items that are under "licenses that require free distribution", and the files themselves are placed in a separate part of the overall software package.

Benchmarks


Current


  • SPEC CPU2006, combined performance of CPU, memory
    Computer storage

    Computer data storage, often called storage or memory, refers to computer components, devices, and recording medium that retain digital data used for computing for some interval of time....
     and compiler
    • CINT2006 ("SPECint
      SPECint

      SPECint is a Benchmark specification for CPU's integer processing power. It is maintained by the Standard Performance Evaluation Corporation . SPECint is the integer performance testing component of the SPEC test suite....
      "), testing integer
      Integer (computer science)

      In computer science, the term integer is used to refer to a data type which represents some finite subset of the mathematical integers. These are also known as integral data types....
       arithmetic, with programs such as compilers, interpreters, word processors, chess programs etc.
    • CFP2006 ("SPECfp
      SPECfp

      SPECfp is a computer benchmark designed to test the floating point performance of a computer. It is managed by the Standard Performance Evaluation Corporation....
      "), testing floating point
      Floating point

      In computing, floating point describes a system for numerical representation in which a String of digits represents a rational number.The term floating point refers to the fact that the radix point can "float": that is, it can be placed anywhere relative to the Significant figures of the number....
       performance, with physical simulations, 3D graphics, image processing, computational chemistry etc.
  • SPECjms2007, Java Message Service
    Java Message Service

    The Java Message Service Application Programming Interface is a Java Message Oriented Middleware API for sending messages between two or more Client ....
     performance
  • SPECweb2005, PHP and/or JSP performance.
  • SPECviewperf, performance of an OpenGL
    OpenGL

    OpenGL is a standard specification defining a cross-language cross-platform Application programming interface for writing applications that produce 2D computer graphics and 3D computer graphics....
     3D graphics system, tested with various rendering tasks from real applications
  • SPECapc, performance of several 3D-intensive popular applications on a given system
  • SPEC OMP2001 V3.2, for evaluating performance of parallel systems using OpenMP (http://www.openmp.org) applications.
  • SPEC MPI2007, for evaluating performance of parallel systems using MPI (Message Passing Interface
    Message Passing Interface

    Message Passing Interface is a specification for an API that allows many computers to communicate with one another. It is used in computer clusters and supercomputers....
    ) applications.
  • SPECjvm2008, measuring basic Java performance of a Java Runtime Environment on a wide variety of both client and server systems.
  • SPECjAppServer2004, a multi-tier benchmark for measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based application servers.
  • SPECjbb2005, evaluates the performance of server side Java by emulating a three-tier client/server system (with emphasis on the middle tier).
  • SPEC MAIL2001, performance of a mail server, testing SMTP and POP
    Post Office Protocol

    In computing, the Post Office Protocol version 3 is an application layer Internet standard protocol used by local e-mail clients to retrieve e-mail from a remote mail server over a Internet protocol suite connection....
     protocols
  • SPECpower_ssj2008, evaluates the energy efficiency of server systems.
  • SPEC SFS97_R1, NFS file server throughput and response time


Future


  • SPECappPlatform, performance of web enterprise platforms (such as Java EE and .NET)
  • SPECimap2003, performance of an enterprise level mail server, testing SMTP and IMAP4 protocols


Retired

  • SPEC CPU2000
  • SPEC CPU95
  • SPEC HPC96
  • SPEC HPC2002 (no longer available)
  • SPECjAppServer2001
  • SPECjAppServer2002
  • SPECjbb2000
  • SPEC SDM91
  • SPEC SFS97
  • SPECweb96
  • SPECweb99
  • SPECweb99_SSL
  • SPEC JVM98


External links