Blue Gene
Encyclopedia
Computer architecture
In computer science and engineering, computer architecture is the practical art of selecting and interconnecting hardware components to create computers that meet functional, performance and cost goals and the formal modelling of those systems....
project to produce several supercomputer
Supercomputer
A supercomputer is a computer at the frontline of current processing capacity, particularly speed of calculation.Supercomputers are used for highly calculation-intensive tasks such as problems including quantum physics, weather forecasting, climate research, molecular modeling A supercomputer is a...
s, designed to reach operating speeds in the PFLOPS (petaFLOPS)
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
range, and currently reaching sustained speeds of nearly 500 TFLOPS (teraFLOPS)
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
. It is a cooperative project among IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
(particularly IBM Rochester
IBM Rochester
IBM Rochester is the facility of International Business Machines in Rochester, Minnesota, not to be confused with the IBM Global Services facility in Rochester, NY. The initial structure was designed by Eero Saarinen, who clad the structure in blue panels of varying hues after being inspired by...
and the Thomas J. Watson Research Center
Thomas J. Watson Research Center
The Thomas J. Watson Research Center is the headquarters for the IBM Research Division.The center is on three sites, with the main laboratory in Yorktown Heights, New York, 38 miles north of New York City, a building in Hawthorne, New York, and offices in Cambridge, Massachusetts.- Overview :The...
), the Lawrence Livermore National Laboratory
Lawrence Livermore National Laboratory
The Lawrence Livermore National Laboratory , just outside Livermore, California, is a Federally Funded Research and Development Center founded by the University of California in 1952...
, the United States Department of Energy
United States Department of Energy
The United States Department of Energy is a Cabinet-level department of the United States government concerned with the United States' policies regarding energy and safety in handling nuclear material...
(which is partially funding the project), and academia
Academia
Academia is the community of students and scholars engaged in higher education and research.-Etymology:The word comes from the akademeia in ancient Greece. Outside the city walls of Athens, the gymnasium was made famous by Plato as a center of learning...
. There are four Blue Gene projects in development: Blue Gene/L, Blue Gene/C, Blue Gene/P, and Blue Gene/Q.
The project was awarded the National Medal of Technology and Innovation by U.S. President
President of the United States
The President of the United States of America is the head of state and head of government of the United States. The president leads the executive branch of the federal government and is the commander-in-chief of the United States Armed Forces....
Barack Obama
Barack Obama
Barack Hussein Obama II is the 44th and current President of the United States. He is the first African American to hold the office. Obama previously served as a United States Senator from Illinois, from January 2005 until he resigned following his victory in the 2008 presidential election.Born in...
on September 18, 2009. The president bestowed the award on October 7, 2009.
Blue Gene/L
Lawrence Livermore National Laboratory
The Lawrence Livermore National Laboratory , just outside Livermore, California, is a Federally Funded Research and Development Center founded by the University of California in 1952...
(LLNL), originally had a theoretical peak performance of 360 TFLOPS
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
, and scored over 280 TFLOPS sustained on the LINPACK benchmark
LINPACK
LINPACK is a software library for performing numerical linear algebra on digital computers. It was written in Fortran by Jack Dongarra, Jim Bunch, Cleve Moler, and Gilbert Stewart, and was intended for use on supercomputers in the 1970s and early 1980s...
. After an upgrade in 2007 the performance increased to 478 TFLOPS sustained and 596 TFLOPS peak.
The term Blue Gene/L sometimes refers to the computer installed at LLNL; and sometimes refers to the architecture of that computer. In November 2006, there were 27 computers on the TOP500
TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
list using the Blue Gene/L architecture. All these computers were listed as having an architecture of eServer Blue Gene Solution.
Protein folding
Protein folding is the process by which a protein structure assumes its functional shape or conformation. It is the physical process by which a polypeptide folds into its characteristic and functional three-dimensional structure from random coil....
. The project has two main goals: to advance our understanding of the mechanisms behind protein folding via large-scale simulation, and to explore novel ideas in massively parallel machine architecture and software. This project should enable biomolecular simulations that are orders of magnitude larger than current technology permits. Major areas of investigation include: how to use this novel platform to effectively meet its scientific goals, how to make such massively parallel machines more usable, and how to achieve performance targets at a reasonable cost, through novel machine architectures. The design is built largely around the previous QCDSP and QCDOC
QCDOC
The QCDOC, Quantum ChromoDynamics On a Chip, is a supercomputer technology focusing on using relatively cheap low power processing elements to produce a massively parallel machine...
supercomputers.
In November 2001, Lawrence Livermore National Laboratory
Lawrence Livermore National Laboratory
The Lawrence Livermore National Laboratory , just outside Livermore, California, is a Federally Funded Research and Development Center founded by the University of California in 1952...
joined IBM as a research partner for Blue Gene.
On September 29, 2004, IBM announced that a Blue Gene/L prototype at IBM Rochester
IBM Rochester
IBM Rochester is the facility of International Business Machines in Rochester, Minnesota, not to be confused with the IBM Global Services facility in Rochester, NY. The initial structure was designed by Eero Saarinen, who clad the structure in blue panels of varying hues after being inspired by...
(Minnesota
Minnesota
Minnesota is a U.S. state located in the Midwestern United States. The twelfth largest state of the U.S., it is the twenty-first most populous, with 5.3 million residents. Minnesota was carved out of the eastern half of the Minnesota Territory and admitted to the Union as the thirty-second state...
) had overtaken NEC's Earth Simulator
Earth Simulator
The Earth Simulator , developed by the Japanese government's initiative "Earth Simulator Project", was a highly parallel vector supercomputer system for running global climate models to evaluate the effects of global warming and problems in solid earth geophysics...
as the fastest computer in the world, with a speed of 36.01 TFLOPS on the LINPACK benchmark, beating Earth Simulator's 35.86 TFLOPS. This was achieved with an 8-cabinet system, with each cabinet holding 1,024 compute nodes. Upon doubling this configuration to 16 cabinets, the machine reached a speed of 70.72 TFLOPS by November 2004 , taking first place in the TOP500
TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
list.
On March 24, 2005, the US Department of Energy announced that the Blue Gene/L installation at LLNL broke its speed record, reaching 135.5 TFLOPS. This feat was possible because of doubling the number of cabinets to 32.
On the June 2006 TOP500 list, Blue Gene/L installations across several sites worldwide took 3 out of the 10 top positions, and 13 out of the top 64. Three racks of Blue Gene/L are housed at the San Diego Supercomputer Center
San Diego Supercomputer Center
The San Diego Supercomputer Center is an organized research unit of the University of California, San Diego . Physically, SDSC is located on the east end of Eleanor Roosevelt College on the campus of UCSD....
and are available for academic research
Research
Research can be defined as the scientific search for knowledge, or as any systematic investigation, to establish novel facts, solve new or existing problems, prove new ideas, or develop new theories, usually using a scientific method...
. New York Blue/L
New York Blue Gene supercomputer
New York Blue Gene supercomputer, also known as NewYorkBlue, is a 18 rack Blue Gene/L and a 2 rack Blue Gene/P massively parallel supercomputer based on the IBM system-on-chip technology. It is located in the New York Center for Computational Sciences...
, ranked 17th in the June 2008 TOP500 list, also provides time allocations when requested.
On October 27, 2005, LLNL and IBM announced that Blue Gene/L had once again broken its speed record, reaching 280.6 TFLOPS on LINPACK
LINPACK
LINPACK is a software library for performing numerical linear algebra on digital computers. It was written in Fortran by Jack Dongarra, Jim Bunch, Cleve Moler, and Gilbert Stewart, and was intended for use on supercomputers in the 1970s and early 1980s...
, upon reaching its final configuration of 65,536 "compute nodes" (i.e., 216 nodes) and an additional 1024 "I/O nodes" in 64 air-cooled cabinets. The LLNL Blue Gene/L uses Lustre
Lustre (file system)
Lustre is a massively parallel distributed file system, generally used for large scale cluster computing. The name Lustre is a portmanteau word derived from Linux and cluster...
to access multiple filesystems in the 600TB-1PB range.
Blue Gene/L is also the first supercomputer ever to run over 100 TFLOPS
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
sustained on a real world application, namely a three-dimensional molecular dynamics code (ddcMD), simulating solidification (nucleation and growth processes) of molten metal under high pressure and temperature conditions. This achievement won the 2005 Gordon Bell Prize
Gordon Bell Prize
The Gordon Bell Prizes are a set of awards awarded by the Association for Computing Machinery in conjunction with the Institute of Electrical and Electronics Engineers each year at the Supercomputing Conference to recognize outstanding achievement in high-performance computing applications...
.
On June 22, 2006, NNSA and IBM announced that Blue Gene/L has achieved 207.3 TFLOPS on a quantum chemical application (Qbox). On November 14, 2006, at Supercomputing 2006, Blue Gene/L was awarded the winning prize in all HPC Challenge Classes of awards. A team from the IBM Almaden Research Center and the University of Nevada
University of Nevada, Reno
The University of Nevada, Reno , is a teaching and research university established in 1874 and located in Reno, Nevada, USA...
on April 27, 2007 ran an artificial neural network
Artificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
almost half as complex as the brain of a mouse for the equivalent of a second (the network was run at 1/10 of normal speed for 10 seconds).
In November 2007, the LLNL Blue Gene/L remained at the number one spot as the world's fastest supercomputer. It had been upgraded since the previous measurement, and was then almost three times as fast as the second fastest, a Blue Gene/P system.
On June 18, 2008, the new TOP500 List marked the first time a Blue Gene system was not the leader in the TOP500 since it had assumed that position, being topped by IBM's Cell-based Roadrunner system, which was at the time the only system to surpass the petaflops mark.
Major features
The Blue Gene/L supercomputer is unique in the following aspects:- Trading the speed of processors for lower power consumption.
- Dual processors per node with two working modes: co-processor (1 user process/node: computation and communication work is shared by two processors) and virtual node (2 user processes/node)
- System-on-a-chip design
- A large number of nodes (scalable in increments of 1024 up to at least 65,536)
- Three-dimensional torusTorusIn geometry, a torus is a surface of revolution generated by revolving a circle in three dimensional space about an axis coplanar with the circle...
interconnect with auxiliary networks for global communications, I/O, and management - Lightweight OS per node for minimum system overhead (computational noise)
Architecture
Application-specific integrated circuit
An application-specific integrated circuit is an integrated circuit customized for a particular use, rather than intended for general-purpose use. For example, a chip designed solely to run a cell phone is an ASIC...
with associated DRAM
Dynamic random access memory
Dynamic random-access memory is a type of random-access memory that stores each bit of data in a separate capacitor within an integrated circuit. The capacitor can be either charged or discharged; these two states are taken to represent the two values of a bit, conventionally called 0 and 1...
memory chips. The ASIC integrates two 700 MHz PowerPC 440
PowerPC 440
The PowerPC 400 family is a line of 32-bit embedded RISC processor cores built using Power Architecture technology. The cores are designed to fit inside specialized applications ranging from system-on-a-chip microcontrollers, network appliances, application-specific integrated circuits and...
embedded processors, each with a double-pipeline-double-precision Floating Point Unit
Floating point unit
A floating-point unit is a part of a computer system specially designed to carry out operations on floating point numbers. Typical operations are addition, subtraction, multiplication, division, and square root...
(FPU), a cache
Cache
In computer engineering, a cache is a component that transparently stores data so that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere...
sub-system with built-in DRAM controller and the logic to support multiple communication sub-systems. The dual FPUs give each Blue Gene/L node a theoretical peak performance of 5.6 GFLOPS (gigaFLOPS)
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
. Node CPUs are not cache coherent
Cache coherency
In computing, cache coherence refers to the consistency of data stored in local caches of a shared resource.When clients in a system maintain caches of a common memory resource, problems may arise with inconsistent data. This is particularly true of CPUs in a multiprocessing system...
with one another.
Compute nodes are packaged two per compute card, with 16 compute cards plus up to 2 I/O nodes per node board. There are 32 node boards per cabinet/rack. By integration of all essential sub-systems on a single chip, each Compute or I/O node dissipates low power (about 17 watts, including DRAMs). This allows very aggressive packaging of up to 1024 compute nodes plus additional I/O nodes in the standard 19" cabinet, within reasonable limits of electrical power supply and air cooling. The performance metrics in terms of FLOPS per watt, FLOPS per m2 of floorspace and FLOPS per unit cost allow scaling up to very high performance. With so many nodes, component failures are inevitable. The system is able to electrically isolate faulty hardware to allow the machine to continue to run.
Each Blue Gene/L node is attached to three parallel communications networks: a 3D
Dimension
In physics and mathematics, the dimension of a space or object is informally defined as the minimum number of coordinates needed to specify any point within it. Thus a line has a dimension of one because only one coordinate is needed to specify a point on it...
toroidal network for peer-to-peer communication between compute nodes, a collective network for collective communication, and a global interrupt network for fast barriers
Barrier (computer science)
- Threads synchronization primitive :In parallel computing, a barrier is a type of synchronization method. A barrier for a group of threads or processes in the source code means any thread/process must stop at this point and cannot proceed until all other threads/processes reach this barrier.Many...
. The I/O nodes, which run the Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
, provide communication with the world via an Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....
network. The I/O nodes also handle the filesystem operations on behalf of the compute nodes. Finally, a separate and private Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....
network provides access to any node for configuration, booting
Booting
In computing, booting is a process that begins when a user turns on a computer system and prepares the computer to perform its normal operations. On modern computers, this typically involves loading and starting an operating system. The boot sequence is the initial set of operations that the...
and diagnostics. To allow multiple programs to run concurrently, a Blue Gene/L system can be partitioned into electronically isolated sets of nodes. The number of nodes in a partition must be a positive integer
Integer
The integers are formed by the natural numbers together with the negatives of the non-zero natural numbers .They are known as Positive and Negative Integers respectively...
power of 2, and must contain at least 25 = 32 nodes. The maximum partition is all nodes in the computer. To run a program on Blue Gene/L, a partition of the computer must first be reserved. The program is then run on all the nodes within the partition, and no other program may access nodes within the partition while it is in use. Upon completion, the partition nodes are released for future programs to use.
Blue Gene/L compute nodes use a minimal operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
supporting a single user program. Only a subset of POSIX
POSIX
POSIX , an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems...
calls are supported, and only one process may be run at a time. Programmers need to implement green threads
Green threads
In computer programming, green threads are threads that are scheduled by a virtual machine instead of natively by the underlying operating system...
in order to simulate local concurrency. Application development is usually performed in C, C++, or Fortran using MPI
Message Passing Interface
Message Passing Interface is a standardized and portable message-passing system designed by a group of researchers from academia and industry to function on a wide variety of parallel computers...
for communication. However, some scripting languages such as Ruby
Ruby (programming language)
Ruby is a dynamic, reflective, general-purpose object-oriented programming language that combines syntax inspired by Perl with Smalltalk-like features. Ruby originated in Japan during the mid-1990s and was first developed and designed by Yukihiro "Matz" Matsumoto...
and Python
Python (programming language)
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. Python claims to "[combine] remarkable power with very clear syntax", and its standard library is large and comprehensive...
have been ported to the compute nodes.
Plan 9 support
A team composed of members from Bell-Labs, IBM ResearchIBM Research
IBM Research, a division of IBM, is a research and advanced development organization and currently consists of eight locations throughout the world and hundreds of projects....
, Sandia National Laboratories
Sandia National Laboratories
The Sandia National Laboratories, managed and operated by the Sandia Corporation , are two major United States Department of Energy research and development national laboratories....
, and Vita Nuova have completed a port of Plan 9
Plan 9 from Bell Labs
Plan 9 from Bell Labs is a distributed operating system. It was developed primarily for research purposes as the successor to Unix by the Computing Sciences Research Center at Bell Labs between the mid-1980s and 2002...
to Blue Gene/L and Blue Gene/P. Plan 9 kernels are running on both the compute nodes and the I/O nodes. The Ethernet, Torus, Collective Network, Barrier Network, and Management networks are all supported.
Cyclops64 (Blue Gene/C)
Blue Gene/C (now renamed to Cyclops64) is a sister-project to Blue Gene/L. It is a massively parallel, supercomputer-on-a-chip cellular architectureCellular architecture
A cellular architecture is a type of computer architecture prominent in parallel computing. Cellular architectures are relatively new, with IBM's Cell microprocessor being the first one to reach the market...
. It was slated for release in early 2007 but has been delayed.
Blue Gene/P
Argonne National Laboratory
Argonne National Laboratory is the first science and engineering research national laboratory in the United States, receiving this designation on July 1, 1946. It is the largest national laboratory by size and scope in the Midwest...
's Leadership Computing Facility. Designed to run continuously at 1 PFLOPS (petaFLOPS)
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
, it can be configured to reach speeds in excess of 3 PFLOPS. Furthermore, it is at least seven times more energy efficient
Energy conversion efficiency
Energy conversion efficiency is the ratio between the useful output of an energy conversion machine and the input, in energy terms. The useful output may be electric power, mechanical work, or heat.-Overview:...
than any other supercomputer, accomplished by using many small, low-power chips connected through five specialized networks. Four 850 MHz PowerPC 450 processors are integrated on each Blue Gene/P chip. The 1-PFLOPS Blue Gene/P configuration is a 294,912-processor, 72-rack system harnessed to a high-speed, optical network. Blue Gene/P can be scaled to an 884,736-processor, 216-rack cluster to achieve 3-PFLOPS performance. A standard Blue Gene/P configuration will house 4,096 processors per rack.
On November 12, 2007, the first system, JUGENE
JUGENE
JUGENE is a supercomputer built by IBM for Forschungszentrum Jülich in Germany. It is based on the Blue Gene/P and is the successor to the JUBL based on an earlier design. It was at the introduction the second fastest computer in the world. It is currently the ninth fastest computer in the world...
, with 65,536 processors is running at Forschungszentrum Jülich in Germany
Germany
Germany , officially the Federal Republic of Germany , is a federal parliamentary republic in Europe. The country consists of 16 states while the capital and largest city is Berlin. Germany covers an area of 357,021 km2 and has a largely temperate seasonal climate...
with a performance of 167 TFLOPS. When inaugurated it was the fastest supercomputer in Europe and the sixth fastest in the world. The first laboratory in the United States to receive the Blue Gene/P was Argonne National Laboratory
Argonne National Laboratory
Argonne National Laboratory is the first science and engineering research national laboratory in the United States, receiving this designation on July 1, 1946. It is the largest national laboratory by size and scope in the Midwest...
. The first racks of the Blue Gene/P shipped in fall 2007. The first installment was a 111-teraflops system, which has approximately 32,000 processors, and was operational for the US research community in spring 2008. The full Intrepid system is ranked #3 on the June 2008 Top 500 list. Another Blue Gene/P has been installed on September 9, 2008 in Sofia
Sofia
Sofia is the capital and largest city of Bulgaria and the 12th largest city in the European Union with a population of 1.27 million people. It is located in western Bulgaria, at the foot of Mount Vitosha and approximately at the centre of the Balkan Peninsula.Prehistoric settlements were excavated...
, the capital of Bulgaria
Bulgaria
Bulgaria , officially the Republic of Bulgaria , is a parliamentary democracy within a unitary constitutional republic in Southeast Europe. The country borders Romania to the north, Serbia and Macedonia to the west, Greece and Turkey to the south, as well as the Black Sea to the east...
, and is operated by the Bulgarian Academy of Sciences
Bulgarian Academy of Sciences
The Bulgarian Academy of Sciences is the National Academy of Bulgaria, established in 1869. The Academy is autonomous and has a Society of Academicians, Correspondent Members and Foreign Members...
and the Sofia University
Sofia University
The St. Clement of Ohrid University of Sofia or Sofia University is the oldest higher education institution in Bulgaria, founded on 1 October 1888...
. In 2010, a Blue Gene/P was installed at the University of Melbourne
University of Melbourne
The University of Melbourne is a public university located in Melbourne, Victoria. Founded in 1853, it is the second oldest university in Australia and the oldest in Victoria...
for the Victorian Life Sciences Computation Initiative
Victorian Life Sciences Computation Initiative
The Victorian Life Sciences Computation Initiative is an initiative of the government of Victoria in partnership with the University of Melbourne and the IBM Research Collaboratory for Life Sciences, Melbourne...
.
In February 2009 it was announced that JUGENE will be upgraded to reach petaflops performance in June 2009, making it the first petascale
Petascale
In computing, petascale refers to a computer system capable of reaching performance in excess of one petaflop, i.e. one quadrillion floating point operations per second. The standard benchmark tool is LINPACK and Top500.org is the organisation which tracks the fastest supercomputers...
supercomputer in Europe
Europe
Europe is, by convention, one of the world's seven continents. Comprising the westernmost peninsula of Eurasia, Europe is generally 'divided' from Asia to its east by the watershed divides of the Ural and Caucasus Mountains, the Ural River, the Caspian and Black Seas, and the waterways connecting...
. The new configuration has started at April 6, the system will go into production end of June 2009. The new configuration will include 294,912 processor cores, 144 terabyte memory, 6 petabyte storage in 72 racks. The new configuaration will incorporate a new water cooling system that will reduce the cooling cost substantially.
Veselin Topalov
Veselin Topalov
Veselin Aleksandrov Topalov is a Bulgarian chess grandmaster. He currently has the sixth highest rating in the world, and was the challenger facing world champion Viswanathan Anand in the World Chess Championship 2010, losing the match 6½–5½....
, the challenger to the World Chess Champion title in 2010, confirmed in an interview that he had used a Blue Gene/P supercomputer during his preparation for the match. The Blue Gene/P computer has been used to simulate approximately one percent of a human cerebral cortex, containing 1.6 billion neurons with approximately 9 trillion connections.
Web-scale platform
The IBM KittyhawkIBM Kittyhawk
Kittyhawk is a new theoretical IBM supercomputer. The proposed project entails constructing a global-scale shared supercomputer capable of hosting the entire Internet on one platform as an application, whereas the current Internet is a collection of interconnected computer networks.In 2010 IBM...
project team has ported Linux to the compute nodes and demonstrated generic Web 2.0 workloads running at scale on a Blue Gene/P. Their paper published in the ACM Operating Systems Review describes a kernel driver that tunnels Ethernet over the tree network, which results in all-to-all TCP/IP connectivity. Running standard Linux software like MySQL
MySQL
MySQL officially, but also commonly "My Sequel") is a relational database management system that runs as a server providing multi-user access to a number of databases. It is named after developer Michael Widenius' daughter, My...
, their performance results on SpecJBB rank among the highest on record.
Blue Gene/Q
The last known supercomputer design in the Blue Gene series, Blue Gene/Q was aimed to reach 20 Petaflops in the 2011 time frame, but has slipped to 2012. It continues to expand and enhance the Blue Gene/L and /P architectures with higher frequency at much improved performance per wattPerformance per watt
In computing, performance per watt is a measure of the energy efficiency of a particular computer architecture or computer hardware. Literally, it measures the rate of computation that can be delivered by a computer for every watt of power consumed....
(1684 - 2097 MFLOPS/Watt). Blue Gene/Q has a similar number of nodes but many more cores per node.
Design
- The Blue Gene/Q is a 4-way hyperthreaded 64-bit64-bit64-bit is a word size that defines certain classes of computer architecture, buses, memory and CPUs, and by extension the software that runs on them. 64-bit CPUs have existed in supercomputers since the 1970s and in RISC-based workstations and servers since the early 1990s...
PowerPC A2PowerPC A2The PowerPC A2 is a massively multicore capable and multithreaded 64-bit Power Architecture processor core designed by IBM using the Power ISA v.2.06 specification. Versions of processors based on the A2 core range from a 2.3 GHz version with 16 cores consuming 65 W to a less powerful, four core...
based chip with 18 cores, where 1 is reserved for running an operating system and 1 in case one of the others fails. The chips will have integrated memory, I/O controllers and be mounted on a compute node card which also has 1 GB DDR3 RAM for each processor core. - A compute drawer will have 32 compute cards, each water cooled and connected with fiber optics for the 5D network torus.
- Each I/O drawer will be air cooled and contain 8 compute cards and 8 PCIe expansion slots for InfinibandInfiniBandInfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable...
or 10 Gigabit Ethernet10 Gigabit EthernetThe 10 gigabit Ethernet computer networking standard was first published in 2002. It defines a version of Ethernet with a nominal data rate of 10 Gbit/s , ten times faster than gigabit Ethernet.10 gigabit Ethernet defines only full duplex point to point links which are generally connected by...
networking. - Racks will have 32 compute drawers for a total of 1024 compute nodes, 16,384 cores and 16 TB RAM.
Installations
The archetypal Blue Gene/Q system called SequoiaIBM Sequoia
Sequoia is a petascale Blue Gene/Q supercomputer being constructed by IBM for the National Nuclear Security Administration as part of the Advanced Simulation and Computing Program...
will be installed at Lawrence Livermore National Laboratory in 2012 as a part of the Advanced Simulation and Computing Program
Advanced Simulation and Computing Program
The Advanced Simulation and Computing Program is a supercomputing initiative of the United States government, created to help the maintenance of the United States nuclear arsenal after the 1992 moratorium on nuclear testing.Some of the program's supercomputers are on the TOP500...
running nuclear simulations and advanced scientific research. It will consist of 98,304 compute nodes comprising 1.6 million processor cores and 1.6 PB
Petabyte
A petabyte is a unit of information equal to one quadrillion bytes, or 1000 terabytes. The unit symbol for the petabyte is PB...
memory in 96 racks covering an area of about 3000 square feet (278.7 m²), drawing 6 megawatts of power.
A Blue Gene/Q system called Mira will be installed at Argonne National Laboratory
Argonne National Laboratory
Argonne National Laboratory is the first science and engineering research national laboratory in the United States, receiving this designation on July 1, 1946. It is the largest national laboratory by size and scope in the Midwest...
in the Argonne Leadership Computing Facility early in 2012. It will consist of 49,152 compute nodes, with 70 PB
Petabyte
A petabyte is a unit of information equal to one quadrillion bytes, or 1000 terabytes. The unit symbol for the petabyte is PB...
of disk storage (470 GB/s I/O bandwidth).
A single midplane (8192 cores) of a Blue Gene/Q prototype at IBM Watson was #115 on the November 2010 Top 500 list, with more than 100 teraflops peak performance and LINPACK performance of 65 teraflops.
External links
- Blue Gene: A Vision for Protein Science using a Petaflop Supercomputer IBM Systems Journal, Special Issue on Deep Computing for the Life Sciences, Vol. 40, No. 2
- IBM Journal of Research and Development, Blue Gene, Vol. 49, No. 2/3
- IBM Research: Massive parallelism for power and performance efficiency
- IBM Research: Blue Gene
- Project Kittyhawk: A Global-Scale Computer
- CNET story: Blue Gene/L tops its own supercomputer record
- CNET story: It's Linux for IBM supercomputer project
- More detailed presentation of Blue Gene architecture (in pdf format)
- Lawrence Livermore National Laboratory: BlueGene/L
- [ftp://ftp.capsl.udel.edu/pub/doc/memos/memo055.ps.gz A technical description of the Cyclops64 architecture and system software (Gzipped postscript file)]
- Petascale Plan 9 on Blue Gene - Papers, posters and other documents about Plan 9 from Bell LabsPlan 9 from Bell LabsPlan 9 from Bell Labs is a distributed operating system. It was developed primarily for research purposes as the successor to Unix by the Computing Sciences Research Center at Bell Labs between the mid-1980s and 2002...
on the Blue Gene. - Next generation supercomputers - Blue Gene/P overview (pdf)