Object storage device
Encyclopedia
An Object-based Storage Device (OSD) is a computer storage
device, similar to disk storage
but working at a higher level. Instead of providing a block-oriented interface that reads and writes fixed sized blocks of data, an OSD organizes data into flexible-sized data containers, called objects. Each object has both data (an uninterpreted sequence of byte
s) and metadata
(an extensible set of attributes describing the object).
The command interface to the OSD includes commands to create and delete objects, write bytes and read bytes to and from individual objects, and to set and get attributes on objects. The OSD is responsible for managing the storage of objects and their metadata.
The OSD implements a security mechanism that provides per-object and per-command access control.
Several research projects have explored object storage and implemented prototype systems with various semantics. The early research spawned a standardization effort that led to a standard OSD command set for SCSI
.
command set for Object Storage Devices was developed by a working group of the Storage Networking Industry Association
(SNIA) for the T10 committee of the International Committee for Information Technology Standards (INCITS). (T10 is responsible for all SCSI
standards.) The initial OSD standard, "Object-based Storage Device Commands" (OSD), was approved in 2004.
In the OSD standard, objects are specified with a 64-bit partition ID and a 64-bit object ID.
Partitions are created and deleted within an OSD, and objects are created and deleted within partitions.
There are no fixed sizes associated with partitions or objects; they are allowed to grow subject to physical size limitations of the device or logical quota constraints on a partition.
An extensible set of attributes describe objects.
Some attributes are implemented directly by the OSD, such as the number of bytes
in an object and the modify time of an object.
There is a special policy tag attribute that is part of the security mechanism.
Other attributes are uninterpreted by the OSD. These are set on objects by the higher-level storage systems that use the OSD for persistent storage. For example, attributes might be used to classify objects, or to capture relationships among different objects stored on different OSDs.
A list command returns a list of identifiers for objects within a partition, optionally filtered by matches against their attribute values. A list command can also return selected attributes of the listed objects.
Read and write commands can be combined, or piggy-backed, with commands to get and set attributes.
This ability reduces the number of times a high-level storage system has to cross the interface to the OSD, which can improve overall efficiency.
The main additions are support for snapshots, collections of objects, and improved error handling.
A snapshot
is a point in time copy of all the objects in a partition into a new partition.
The OSD can implement a space-efficient copy using copy-on-write
techniques so that the two partitions share objects that are unchanged between the snapshots, or the OSD might physically copy the data to the new partition.
The standard defines clones, which are writeable, and snapshots, which are read-only.
A collection is a special kind of object that contains the identifiers of other objects.
There are operations to add and delete from collections, and there are operations to get or set attributes for all the objects in a collection.
Collections are also used for error reporting. If an object becomes damaged by the occurrence of a media defect (i.e., a bad spot on the disk) or by a software error within the OSD implementation, its identifier is put into a special error collection.
The higher-level storage system that uses the OSD can query this collection and take corrective action as necessary.
environment.
A more powerful processor could implement features such as space management (i.e., block allocation) and provide a more abstract interface of reading and writing bytes to flexible data containers.
Access control was an important aspect of the research, and
Howard Gobioff described a security protocol for object storage that allowed a security manager to provide
fine-grained access control to a shared storage device on a network.
The general goal of the research was to provide a higher-level building block that could be aggregated together to provide large scale, secure storage systems.
The first version of the OSD command set for SCSI was standardized in 2004.
The second version of the command set (OSD-2) was finalized in late 2008 but remains stalled in the standardization process. OSD-3 is now in development.
Computer storage
Computer data storage, often called storage or memory, refers to computer components and recording media that retain digital data. Data storage is one of the core functions and fundamental components of computers....
device, similar to disk storage
Disk storage
Disk storage or disc storage is a general category of storage mechanisms, in which data are digitally recorded by various electronic, magnetic, optical, or mechanical methods on a surface layer deposited of one or more planar, round and rotating disks...
but working at a higher level. Instead of providing a block-oriented interface that reads and writes fixed sized blocks of data, an OSD organizes data into flexible-sized data containers, called objects. Each object has both data (an uninterpreted sequence of byte
Byte
The byte is a unit of digital information in computing and telecommunications that most commonly consists of eight bits. Historically, a byte was the number of bits used to encode a single character of text in a computer and for this reason it is the basic addressable element in many computer...
s) and metadata
Metadata
The term metadata is an ambiguous term which is used for two fundamentally different concepts . Although the expression "data about data" is often used, it does not apply to both in the same way. Structural metadata, the design and specification of data structures, cannot be about data, because at...
(an extensible set of attributes describing the object).
The command interface to the OSD includes commands to create and delete objects, write bytes and read bytes to and from individual objects, and to set and get attributes on objects. The OSD is responsible for managing the storage of objects and their metadata.
The OSD implements a security mechanism that provides per-object and per-command access control.
Several research projects have explored object storage and implemented prototype systems with various semantics. The early research spawned a standardization effort that led to a standard OSD command set for SCSI
SCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard disks and tape drives, but it...
.
The OSD Standard
The SCSISCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard disks and tape drives, but it...
command set for Object Storage Devices was developed by a working group of the Storage Networking Industry Association
Storage Networking Industry Association
An association of producers and consumers of storage networking products, whose goal is to further storage networking technology and applications.The Storage Networking Industry Association, or SNIA, was incorporated in December, 1997, and is a registered 501 non-profit trade association...
(SNIA) for the T10 committee of the International Committee for Information Technology Standards (INCITS). (T10 is responsible for all SCSI
SCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard disks and tape drives, but it...
standards.) The initial OSD standard, "Object-based Storage Device Commands" (OSD), was approved in 2004.
In the OSD standard, objects are specified with a 64-bit partition ID and a 64-bit object ID.
Partitions are created and deleted within an OSD, and objects are created and deleted within partitions.
There are no fixed sizes associated with partitions or objects; they are allowed to grow subject to physical size limitations of the device or logical quota constraints on a partition.
An extensible set of attributes describe objects.
Some attributes are implemented directly by the OSD, such as the number of bytes
in an object and the modify time of an object.
There is a special policy tag attribute that is part of the security mechanism.
Other attributes are uninterpreted by the OSD. These are set on objects by the higher-level storage systems that use the OSD for persistent storage. For example, attributes might be used to classify objects, or to capture relationships among different objects stored on different OSDs.
A list command returns a list of identifiers for objects within a partition, optionally filtered by matches against their attribute values. A list command can also return selected attributes of the listed objects.
Read and write commands can be combined, or piggy-backed, with commands to get and set attributes.
This ability reduces the number of times a high-level storage system has to cross the interface to the OSD, which can improve overall efficiency.
OSD Version 2
Work continues on a second generation of the SCSI command set, "Object-Based Storage Devices - 2" (OSD-2).The main additions are support for snapshots, collections of objects, and improved error handling.
A snapshot
Snapshot (computer storage)
In computer systems, a snapshot is the state of a system at a particular point in time. The term was coined as an analogy to that in photography. It can refer to an actual copy of the state of a system or to a capability provided by certain systems....
is a point in time copy of all the objects in a partition into a new partition.
The OSD can implement a space-efficient copy using copy-on-write
Copy-on-write
Copy-on-write is an optimization strategy used in computer programming. The fundamental idea is that if multiple callers ask for resources which are initially indistinguishable, they can all be given pointers to the same resource...
techniques so that the two partitions share objects that are unchanged between the snapshots, or the OSD might physically copy the data to the new partition.
The standard defines clones, which are writeable, and snapshots, which are read-only.
A collection is a special kind of object that contains the identifiers of other objects.
There are operations to add and delete from collections, and there are operations to get or set attributes for all the objects in a collection.
Collections are also used for error reporting. If an object becomes damaged by the occurrence of a media defect (i.e., a bad spot on the disk) or by a software error within the OSD implementation, its identifier is put into a special error collection.
The higher-level storage system that uses the OSD can query this collection and take corrective action as necessary.
History
Research by Garth Gibson et al. on Network Attached Secure Disks (NASD) in the 1990s explored the ability to move more processing power closer to the disk drive in a network-attached storageNetwork-attached storage
Network-attached storage is file-level computer data storage connected to a computer network providing data access to heterogeneous clients. NAS not only operates as a file server, but is specialized for this task either by its hardware, software, or configuration of those elements...
environment.
A more powerful processor could implement features such as space management (i.e., block allocation) and provide a more abstract interface of reading and writing bytes to flexible data containers.
Access control was an important aspect of the research, and
Howard Gobioff described a security protocol for object storage that allowed a security manager to provide
fine-grained access control to a shared storage device on a network.
The general goal of the research was to provide a higher-level building block that could be aggregated together to provide large scale, secure storage systems.
The first version of the OSD command set for SCSI was standardized in 2004.
The second version of the command set (OSD-2) was finalized in late 2008 but remains stalled in the standardization process. OSD-3 is now in development.
External links
- "SCSI Object-Based Storage Device Commands (OSD)" at the INCITS online store
- T10 OSD-2 and OSD-3 drafts (free to download until ratified as standards)
- "Object Based Storage" by Mesnier, Ganger, and Riedel, IEEE Communications MagazineIEEE Communications MagazineIEEE Communications Magazine deals with all areas of communications including light-wave telecommunications, high-speed data communications, personal communications systems , ISDN, and more...
, v.41 n.8 pp 84–90, August 2003. - "Object-Based Storage Devices" by Christian Bandulet, Sun Developer Network, July 2007
- "Linux and object storage devices" by Jonathan Corbet, LWN.netLWN.netLWN.net is a computing webzine with an emphasis on free software and software for Linux and other Unix-like operating systems. It consists of a weekly issue, separate stories which are published most days, and threaded discussion attached to every story. Most news published daily are short...
, November 4, 2008 - open-osd project aims to develop production-quality open sourceOpen sourceThe term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
implementations of OSD