Main Memory database
Encyclopedia
An in-memory database is a database management system
that primarily relies on main memory for computer data storage. It is contrasted with database management systems which employ a disk storage
mechanism. Main memory databases are faster than disk-optimized databases since the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory reduces the I/O reading activity when querying the data which provides faster and more predictable performance than disk. In applications where response time is critical, such as telecommunications network equipment and mobile ads networks, main memory databases are often used.
devices. These devices lose all stored information when the device loses power or is reset. In this case, MMDBs can be said to lack support for the durability portion of the ACID
(atomicity, consistency, isolation, durability) properties. Volatile memory-based MMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation.
Many MMDBs add durability via the following mechanisms:
Some MMDBs allow the database schema to specify different durability requirements for selected areas of the database - thus, faster-changing data that can easily be regenerated or that has no meaning after a system shut-down would not need to be journaled for durability (though it would have to be replicated for high availability), whereas configuration information would be flagged as needing preservation.
; and form factor, because RAM chips cannot approach the density of a small hard drive.
Manufacturing efficiency is another reason a combined in-memory/on-disk database system may be chosen. Some device product lines, especially in consumer electronics
, include some units with permanent storage, and others that rely on memory for storage (set-top box
es, for example). If such devices require a database system, a manufacturer can adopt a hybrid database system at lower cost, and with less code customization, than using separate in-memory and on-disk databases, respectively, for its disk-less and disk-based products.
, a start-up company founded by Marie-Anne Neimat in 1996 as a spin-off from Hewlett-Packard
, was acquired by Oracle Corporation
in 2005. Oracle now markets this product as both a standalone database and an in-memory database cache to the Oracle database. IBM acquired SolidDB
in 2008, and Microsoft is widely rumored to be launching an in-memory solution in 2009. SAP announced general availability of SAP HANA in June 2011.
Database management system
A database management system is a software package with computer programs that control the creation, maintenance, and use of a database. It allows organizations to conveniently develop databases for various applications by database administrators and other specialists. A database is an integrated...
that primarily relies on main memory for computer data storage. It is contrasted with database management systems which employ a disk storage
Disk storage
Disk storage or disc storage is a general category of storage mechanisms, in which data are digitally recorded by various electronic, magnetic, optical, or mechanical methods on a surface layer deposited of one or more planar, round and rotating disks...
mechanism. Main memory databases are faster than disk-optimized databases since the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory reduces the I/O reading activity when querying the data which provides faster and more predictable performance than disk. In applications where response time is critical, such as telecommunications network equipment and mobile ads networks, main memory databases are often used.
ACID support
In their simplest form, main memory databases store data on volatile memoryVolatile memory
Volatile memory, also known as volatile storage, is computer memory that requires power to maintain the stored information, unlike non-volatile memory which does not require a maintained power supply...
devices. These devices lose all stored information when the device loses power or is reset. In this case, MMDBs can be said to lack support for the durability portion of the ACID
ACID
In computer science, ACID is a set of properties that guarantee database transactions are processed reliably. In the context of databases, a single logical operation on the data is called a transaction...
(atomicity, consistency, isolation, durability) properties. Volatile memory-based MMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation.
Many MMDBs add durability via the following mechanisms:
- Snapshot filesSnapshot (computer storage)In computer systems, a snapshot is the state of a system at a particular point in time. The term was coined as an analogy to that in photography. It can refer to an actual copy of the state of a system or to a capability provided by certain systems....
, or, checkpoint images, which record the state of the database at a given moment in time. These are typically generated periodically, or, at least when the MMDB does a controlled shut-down. While they give a measure of persistence to the data (in that not everything is lost in the case of a system crash) they only offer partial durability (as 'recent' changes will be lost). For full durability, they will need to be supplemented by one of the following:
- Transaction logTransaction logIn the field of databases in computer science, a transaction log is a history of actions executed by a database management system to guarantee ACID properties over crashes or hardware failures...
ging, which records changes to the database in a journal file and facilitates automatic recovery of an in-memory database.
- Non-volatile random access memory (NVRAM), usually in the form of static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM (EEPROMEEPROMEEPROM stands for Electrically Erasable Programmable Read-Only Memory and is a type of non-volatile memory used in computers and other electronic devices to store small amounts of data that must be saved when power is removed, e.g., calibration...
). With this storage, the MMDB system can recover the data store from its last consistent state upon reboot.
- High availabilityHigh availabilityHigh availability is a system design approach and associated service implementation that ensures a prearranged level of operational performance will be met during a contractual measurement period....
implementations that rely on database replicationReplication (computer science)Replication is the process of sharing information so as to ensure consistency between redundant resources, such as software or hardware components, to improve reliability, fault-tolerance, or accessibility. It could be data replication if the same data is stored on multiple storage devices, or...
, with automatic failoverFailoverIn computing, failover is automatic switching to a redundant or standby computer server, system, or network upon the failure or abnormal termination of the previously active application, server, system, or network...
to an identical standby database in the event of primary database failure. To protect against loss of data in the case of a complete system crash, replication of a MMDB is normally used in conjunction with one or more of the mechanisms listed above.
Some MMDBs allow the database schema to specify different durability requirements for selected areas of the database - thus, faster-changing data that can easily be regenerated or that has no meaning after a system shut-down would not need to be journaled for durability (though it would have to be replicated for high availability), whereas configuration information would be flagged as needing preservation.
"Hybrid" in-memory/on-disk databases
The first database engine to support both in-memory and on-disk tables in a single database was released in 2003. The advantage to this approach is flexibility: the developer can strike a balance between performance (which is enhanced by sorting, storing and retrieving specified data entirely in memory, rather than going to disk); cost, because a less expensive hard disk can be substituted for more memory; persistencePersistence (computer science)
Persistence in computer science refers to the characteristic of state that outlives the process that created it. Without this capability, state would only exist in RAM, and would be lost when this RAM loses power, such as a computer shutdown....
; and form factor, because RAM chips cannot approach the density of a small hard drive.
Manufacturing efficiency is another reason a combined in-memory/on-disk database system may be chosen. Some device product lines, especially in consumer electronics
Consumer electronics
Consumer electronics are electronic equipment intended for everyday use, most often in entertainment, communications and office productivity. Radio broadcasting in the early 20th century brought the first major consumer product, the broadcast receiver...
, include some units with permanent storage, and others that rely on memory for storage (set-top box
Set-top box
A set-top box or set-top unit is an information appliance device that generally contains a tuner and connects to a television set and an external source of signal, turning the signal into content which is then displayed on the television screen or other display device.-History:Before the...
es, for example). If such devices require a database system, a manufacturer can adopt a hybrid database system at lower cost, and with less code customization, than using separate in-memory and on-disk databases, respectively, for its disk-less and disk-based products.
Commercial products
In recent years, main memory databases have attracted the interest of larger database vendors. TimesTenTimesTen
TimesTen is an in-memory relational database software product from Oracle Corporation. TimesTen is designed for low latency, high-volume data, event and transaction management. Unlike disk-optimized relational databases such as the Oracle database, DB2, Informix, and SQL Server, TimesTen's data is...
, a start-up company founded by Marie-Anne Neimat in 1996 as a spin-off from Hewlett-Packard
Hewlett-Packard
Hewlett-Packard Company or HP is an American multinational information technology corporation headquartered in Palo Alto, California, USA that provides products, technologies, softwares, solutions and services to consumers, small- and medium-sized businesses and large enterprises, including...
, was acquired by Oracle Corporation
Oracle Corporation
Oracle Corporation is an American multinational computer technology corporation that specializes in developing and marketing hardware systems and enterprise software products – particularly database management systems...
in 2005. Oracle now markets this product as both a standalone database and an in-memory database cache to the Oracle database. IBM acquired SolidDB
SolidDB
IBM solidDB is a product family, developed and sold by IBM. The product family includes two products built around an in-memory relational database management system. The first product is a stand-alone relational database management system entitled solidDB...
in 2008, and Microsoft is widely rumored to be launching an in-memory solution in 2009. SAP announced general availability of SAP HANA in June 2011.
Products
| Product name | License | Description |
|---|---|---|
| Adaptive Server Enterprise (ASE) 15.5 | Proprietary | enterprise database from Sybase Sybase Sybase, an SAP company, is an enterprise software and services company offering software to manage, analyze, and mobilize information, using relational databases, analytics and data warehousing solutions and mobile applications development platforms.... ) |
| Apache Derby Apache Derby Apache Derby is a relational database management system developed by the Apache Software Foundation that can be embedded in Java programs and used for online transaction processing. It has a 2 MB disk-space footprint.Apache Derby is developed as an open source project under the Apache 2.0 license... |
Apache License 2.0 | Java RDBMS |
| Altibase Altibase Altibase helps its customers maximize their data investments by providing real-time data performance solutions. In today’s competitive business environment, Altibase enables companies to drastically improve the speed of data access and analysis across the enterprise.... |
Proprietary | has in-memory and disk table; HYBRID DBMS |
| BlackRay BlackRay BlackRay is an open source main memory high-performance relational database management system developed by SoftMethod GmbH. It is designed to provide high performance for SQL queries and also supports a proprietary API. It performs significantly faster than other open source DBMS... |
GNU General Public Licence (GPLv2) and BSD License | |
| CSQL CSQL CSQL is an open source main memory high-performance relational database management system developed @ sourceforge.net. It is designed to provide high performance for SQL queries and DML statements.... |
GNU General Public Licence or proprietary | |
| Datablitz Datablitz DataBlitz is a general purpose main memory database management system, developed by Lucent Bell Labs Research in 1995. It replaced various home-grown database products used throughout Lucent beginning in 1997.... |
Proprietary | DBMS |
| Eloquera | Proprietary | In-memory, In-memory:persist modes |
| eXtremeDB | commercial product | DBMS, also check out its open source PERST dbms. |
| FleetDB | MIT | NOSQL db with Writing to an append-only log to provide durability. |
| H2 H2 (DBMS) H2 is a relational database management system written in Java. It can be embedded in Java applications or run in the client-server mode. The disk footprint is about 1 MB.... |
Mozilla Public License or Eclipse Public License | has a memory-only mode |
| HSQLDB HSQLDB HSQLDB is a relational database management system written in Java. It has a JDBC driver and supports a large subset of SQL-92 and SQL:2008 standards. It offers a fast, small database engine which offers both in-memory and disk-based tables... |
BSD license | has a memory-only mode |
| IBM TM1 TM1 IBM Cognos TM1 is enterprise planning software used to implement collaborative planning, budgeting and forecasting solutions, as well as analytical and reporting applications. Data in IBM Cognos TM1 is stored and represented as multidimensional OLAP cubes, with data being stored at the "leaf" level... |
Proprietary | in-memory BI and data analysis |
| InfoZoom InfoZoom InfoZoom software is a data analysis, business intelligence and data visualization software product created using in-memory analytics. The software is created and supported by humanIT and the Fraunhofer Institute, the same organization that created MP3 compression technology... |
Proprietary | in-memory BI and data analysis |
| KDB | Proprietary | DBMS, also supports disk based data |
| #liveDB | Open Source | Prevalence engine |
| membase Membase Membase is an Open Source distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data... |
Apache License | NoSQL, hybrid |
| Mercury | Proprietary | object oriented, in-memory data management with persistency support, in-built compiler, hosts application server on same machine; developed by staila technologies, an ETHZ spinoff |
| MicroStrategy MicroStrategy MicroStrategy, Inc. , is a business intelligence software vendor. MicroStrategy's software enables leading organizations worldwide to analyze the vast amounts of data stored across their enterprises to make more strategic business decisions... |
in-memory BI for MicroStrategy 9 | |
| MonetDB MonetDB MonetDB is an open source column-oriented database management system developed at the Centrum Wiskunde & Informatica in the Netherlands.It was designed to provide high performance on complex queries against large databases,e.g... |
MonetDB License | |
| MySQL MySQL MySQL officially, but also commonly "My Sequel") is a relational database management system that runs as a server providing multi-user access to a number of databases. It is named after developer Michael Widenius' daughter, My... |
GNU General Public License or proprietary | has a cluster server which uses a main-memory storage engine |
| Oracle Berkeley DB Berkeley DB Berkeley DB is a computer software library that provides a high-performance embedded database for key/value data. Berkeley DB is a programmatic software library written in C with API bindings for C++, PHP, Java, Perl, Python, Ruby, Tcl, Smalltalk, and most other programming languages... |
Sleepycat License | can be configured to run in memory only |
| Panorama Panorama (database engine) Panorama is a database engine and development environment for Macintosh and Microsoft Windows.-Overview:Panorama is database software for the Apple Inc. Macintosh and Microsoft Windows. It is a product of ProVUE Development... |
for Windows and Macintosh, both single user and server versions | |
| ParAccel Paraccel ParAccel, Inc. is a vendor in the data warehouse appliance market category. Vendors in this category provide a purpose-built database management system used for data warehousing, business intelligence and analytic processing and, according to Hackathorn and White, there are varying degrees of... |
Proprietary | in-memory, columnar, relational, ACID-compliant; disk-based mode as well |
| Polyhedra IMDB Polyhedra DBMS Polyhedra is a family of relational database management systems offered by ENEA AB, a Swedish company. The original version of Polyhedra was an in-memory database management system which could be used in high availability configurations; in 2006 Polyhedra FlashLite was introduced to allow... |
Proprietary | relational, supports High-Availability; acquired in 2001 by ENEA ENEA AB ENEA is global information technology company with its headquarters in Kista, Sweden that provides real-time operating systems and consulting services... |
| QlikView | BI-tool developed by QlikTech | |
| RDM Embedded RDM Embedded RDM Embedded is a high performance, ACID-compliant embedded database management system designed to be linked into C/C++ application programs. RDM Embedded has been designed to utilize multi-core computers, networking , and in-memory or on-disk storage. It provides a low-level C API and a higher... |
Proprietary | including hybrid |
| RDM Server RDM Server RDM Server is an embeddable, heterogeneous, client/server database management system supporting both C/C++ and SQL APIs for programming flexibility. The databases can be disk resident and/or memory resident. RDM Server implements multi-user locking, hot database backup, and a fully ACID compliant... |
Proprietary | including hybrid |
| Redis Redis Redis is used to refer to Romani people.Redis may also refer to:* Redis , an advanced key-value store... |
BSD | NoSQL |
| solidDB SolidDB IBM solidDB is a product family, developed and sold by IBM. The product family includes two products built around an in-memory relational database management system. The first product is a stand-alone relational database management system entitled solidDB... by IBM IBM International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas... |
including hybrid, HSB-based HA, Shared memory, embedded, XA, etc. | |
| SAP HANA database | Proprietary | Database engine of the SAP In-Memory Appliance (SAP HANA) produced by SAP AG SAP AG SAP AG is a German software corporation that makes enterprise software to manage business operations and customer relations. Headquartered in Walldorf, Baden-Württemberg, with regional offices around the world, SAP is the market leader in enterprise application software... |
| SQLite SQLite SQLite is an ACID-compliant embedded relational database management system contained in a relatively small C programming library. The source code for SQLite is in the public domain and implements most of the SQL standard... |
Public domain | hybrid, RAM and disk dbs can be used together |
| Starcounter | in-memory object relational dbms | |
| Tarantool Tarantool Tarantool is an open-source NoSQL database, developed by Mail.ru.Its key properties include:* licence: simplified BSD* all data is maintained in RAM* data persistence is implemented using Write Ahead Log and snapshotting... |
BSD | NoSQL, extendable with Lua stored procedures |
| TimesTen TimesTen TimesTen is an in-memory relational database software product from Oracle Corporation. TimesTen is designed for low latency, high-volume data, event and transaction management. Unlike disk-optimized relational databases such as the Oracle database, DB2, Informix, and SQL Server, TimesTen's data is... by Oracle Oracle Corporation Oracle Corporation is an American multinational computer technology corporation that specializes in developing and marketing hardware systems and enterprise software products – particularly database management systems... |
in memory only or as a cache for Oracle Database | |
| Vertipaq | Proprietary | Microsoft Microsoft Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions... PowerPivot Powerpivot PowerPivot is a free add-in to the 2010 version of the spreadsheet application Microsoft Excel. It extends the capabilities of the PivotTable data summarisation and cross-tabulation feature by introducing the ability to import data from multiple sources... and Microsoft Analysis Services Microsoft Analysis Services Microsoft SQL Server Analysis Services is part of Microsoft SQL Server, a database management system. Microsoft has included a number of services in SQL Server related to business intelligence and data warehousing. These services include Integration Services and Analysis Services... in-memory BI engine |
| VoltDB VoltDB VoltDB is an in-memory database designed by several well-known database system researchers, including Michael Stonebraker , Sam Madden, and Daniel Abadi. It is an ACID-compliant RDBMS which uses a shared nothing architecture. It includes both enterprise and community editions... |
GNU General Public License v3 | in-memory |
| TREX TREX search engine TREX is a search engine in the SAP NetWeaver integrated technology platform produced by SAP AG. The TREX engine is a standalone component that can be used in a range of system environments but is used primarily as an integral part of such SAP products as Enterprise Portal, Knowledge Warehouse, and... |
search engine in the SAP NetWeaver integrated technology platform produced by SAP AG SAP AG SAP AG is a German software corporation that makes enterprise software to manage business operations and customer relations. Headquartered in Walldorf, Baden-Württemberg, with regional offices around the world, SAP is the market leader in enterprise application software... |
|
| Xcelerix by Frontex Frontex Frontex is the European Union agency for external border security... |
commercial product | |
| WX2 by Kognitio | commercial product | |
| Xeround Xeround Xeround is a provider of cloud database software, launched in 2005. The company was founded by Sharon Barkai and Gilad Zlotkin. Zlotkin, a former research fellow at MIT Sloan School of Management, founded five other startups including Radview... |
commercial product, database as a service, in-memory infrastructure with MySQL MySQL MySQL officially, but also commonly "My Sequel") is a relational database management system that runs as a server providing multi-user access to a number of databases. It is named after developer Michael Widenius' daughter, My... front-end |
See also
- In Memory Data Grid - In Memory Distributed Data management
- Embedded DatabasesEmbedded DatabaseAn embedded database system is a database management system which is tightly integrated with an application software that requires access to stored data, such that the database system is “hidden” from the application’s end-user and requires little or no ongoing maintenance...
- NoSQLNosqlIn computing, NoSQL is a broad class of database management systems that differ from the classic model of the relational database management system in some significant ways. These data stores may not require fixed table schemas, usually avoid join operations, and typically scale horizontally...
- Alternative Scale-out Databases
External links
- SAP HANA Now Generally Available to Customers Worldwide
- In-Memory Database Systems Questions and Answers
- IBM Systems and Services for SAP HANA
- SolidDB and the secrets of speed - How the IBM in-memory database redefines high performance
- Oracle TimesTen in-memory database
- The Rebirth of the In-Memory Database