Grid-oriented storage
Encyclopedia
Grid-oriented Storage is a dedicated data storage architecture which can be connected directly to a computational grid to support advanced data bank services and reservoirs for data that can be shared among multiple computers and end users on the grid.
(NAS) products in the Grid Computing
era. A GOS-specific File System (GOS-FS), a P2P
clustering technique, the single-purpose intent of a GOS operating system (OS), and a browser-based management console motivate and enable this new architecture. GOS is the first demonstration that Office/database/Web/media applications can be accelerated by tenfold in real-world tests. GOS systems usually contain one or more hard disks, often arranged into logical, redundant storage containers or RAID
s (redundant arrays of independent disks), as do traditional file servers.
GOS is designed to deal with long-distance, cross-domain and single-image file operations, which is typical in Grid environments. GOS behaves like a file server via the file-based GOS-FS protocol to any entity on the grid. Inspired by the success of GridFTP
, GOS-FS integrates a parallel stream engine and Grid Security Infrastructure
(GSI). Conforming to the universal VFS (Virtual Filesystem Switch), GOS-FS can be pervasively used as an underlying platform to best utilize the increased transfer bandwidth and accelerate the NFS/CIFS-based applications. GOS can also run over SCSI
, Fibre Channel
or iSCSI
, which does not affect the acceleration performance, offering both file level protocols and block level protocols for Storage Area Network
(SAN) from the same system.
Geographically distributed nature of resources that make up the grid infrastructure, along with their heterogeneity and different control policies in different domains, make the availability of these resources dynamic and conditional upon local constraints. Centralized management techniques for these resources are limited in their scalability both in terms of execution efficiency and fault tolerance. Provision of services across such platforms requires a distributed resource management mechanism and the P2P
clustered GOS appliances allow a single storage image to continue to expand, even if a single GOS appliance reaches its capacity limitations. The cluster shares a common, aggregate presentation of the data stored on all participating GOS appliances. Each GOS appliance manages its own internal storage space. The major benefit of this aggregation is that clustered GOS storage can be accessed by users as a single mount point.
GOS products fit the thin-server categorization. Compared with traditional “fat server”-based storage architectures, thin-server GOS appliances deliver numerous advantages, such as the alleviation of potential network/grid bottle-necks, CPU and OS optimized for I/O only, ease of installation, remote management and minimal maintenance, low cost and Plug and Play, etc. Examples of similar innovations include NAS, printers, fax machines, routers and switches.
An Apache server has been installed in the GOS operating system, ensuring an HTTPS-based communication between the GOS server and an administrator via a Web browser. Remote management and monitoring makes it easy to set up, manage, and monitor GOS systems.
and Dr. Na Helian first proposed Grid-oriented Storage (GOS) in a funding proposal to the UK government titled “Grid-Oriented Storage (GOS): Next Generation Data Storage System Architecture for the Grid Computing Era” in 2003. The proposal was approved and granted one million pounds in 2004. The first prototype was constructed in 2005 at Centre for Grid Computing, Cambridge-Cranfield High Performance Computing Facility. The first conference presentation was at IEEE Symposium on Cluster Computing and Grid (CCGrid), 9–12 May 2005, Cardiff, UK. As one of the five best work-in-progress, it was included in the IEEE Distributed Systems Online. In 2006, the GOS architecture and its implementations was published in IEEE Transactions on Computers, titled “Grid-oriented Storage: A Single-Image, Cross-Domain, High-Bandwidth Architecture”. From January 2007, the GOS invention led to invitations to present demonstrations at Princeton University, Cambridge University (Computer Lab), IBM, Rolls Royce, BBC, Xerox, Carnegie Mellon University, CERN, and EADS, etc. A number of collaborations have been fostered with industrial giants.
Description
GOS is a successor of well-used network-attached storageNetwork-attached storage
Network-attached storage is file-level computer data storage connected to a computer network providing data access to heterogeneous clients. NAS not only operates as a file server, but is specialized for this task either by its hardware, software, or configuration of those elements...
(NAS) products in the Grid Computing
Grid computing
Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach a common goal. The grid can be thought of as a distributed system with non-interactive workloads that involve a large number of files...
era. A GOS-specific File System (GOS-FS), a P2P
Peer-to-peer
Peer-to-peer computing or networking is a distributed application architecture that partitions tasks or workloads among peers. Peers are equally privileged, equipotent participants in the application...
clustering technique, the single-purpose intent of a GOS operating system (OS), and a browser-based management console motivate and enable this new architecture. GOS is the first demonstration that Office/database/Web/media applications can be accelerated by tenfold in real-world tests. GOS systems usually contain one or more hard disks, often arranged into logical, redundant storage containers or RAID
RAID
RAID is a storage technology that combines multiple disk drive components into a logical unit...
s (redundant arrays of independent disks), as do traditional file servers.
GOS is designed to deal with long-distance, cross-domain and single-image file operations, which is typical in Grid environments. GOS behaves like a file server via the file-based GOS-FS protocol to any entity on the grid. Inspired by the success of GridFTP
Advanced Resource Connector
Advanced Resource Connector is grid computing middleware introduced by NorduGrid. ARC is an open source software distributed under the Apache License.- History :...
, GOS-FS integrates a parallel stream engine and Grid Security Infrastructure
Grid Security Infrastructure
The Grid Security Infrastructure , formerly called the Globus Security Infrastructure, is a specification for secret, tamper-proof, delegatable communication between software in a grid computing environment...
(GSI). Conforming to the universal VFS (Virtual Filesystem Switch), GOS-FS can be pervasively used as an underlying platform to best utilize the increased transfer bandwidth and accelerate the NFS/CIFS-based applications. GOS can also run over SCSI
SCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard disks and tape drives, but it...
, Fibre Channel
Fibre Channel
Fibre Channel, or FC, is a gigabit-speed network technology primarily used for storage networking. Fibre Channel is standardized in the T11 Technical Committee of the InterNational Committee for Information Technology Standards , an American National Standards Institute –accredited standards...
or iSCSI
ISCSI
In computing, iSCSI , is an abbreviation of Internet Small Computer System Interface, an Internet Protocol -based storage networking standard for linking data storage facilities. By carrying SCSI commands over IP networks, iSCSI is used to facilitate data transfers over intranets and to manage...
, which does not affect the acceleration performance, offering both file level protocols and block level protocols for Storage Area Network
Storage area network
A storage area network is a dedicated network that provides access to consolidated, block level data storage. SANs are primarily used to make storage devices, such as disk arrays, tape libraries, and optical jukeboxes, accessible to servers so that the devices appear like locally attached devices...
(SAN) from the same system.
Geographically distributed nature of resources that make up the grid infrastructure, along with their heterogeneity and different control policies in different domains, make the availability of these resources dynamic and conditional upon local constraints. Centralized management techniques for these resources are limited in their scalability both in terms of execution efficiency and fault tolerance. Provision of services across such platforms requires a distributed resource management mechanism and the P2P
Peer-to-peer
Peer-to-peer computing or networking is a distributed application architecture that partitions tasks or workloads among peers. Peers are equally privileged, equipotent participants in the application...
clustered GOS appliances allow a single storage image to continue to expand, even if a single GOS appliance reaches its capacity limitations. The cluster shares a common, aggregate presentation of the data stored on all participating GOS appliances. Each GOS appliance manages its own internal storage space. The major benefit of this aggregation is that clustered GOS storage can be accessed by users as a single mount point.
GOS products fit the thin-server categorization. Compared with traditional “fat server”-based storage architectures, thin-server GOS appliances deliver numerous advantages, such as the alleviation of potential network/grid bottle-necks, CPU and OS optimized for I/O only, ease of installation, remote management and minimal maintenance, low cost and Plug and Play, etc. Examples of similar innovations include NAS, printers, fax machines, routers and switches.
An Apache server has been installed in the GOS operating system, ensuring an HTTPS-based communication between the GOS server and an administrator via a Web browser. Remote management and monitoring makes it easy to set up, manage, and monitor GOS systems.
History
Prof. Frank WangFrank Zhigang Wang
Frank Wang, is Professor and Chair in e-Science and Grid Computing, Director of . In 1994, he invented spin-tunneling random access memory. In 2003, Frank proposed a new concept of Grid-oriented Storage architecture. In 2004, Frank and his colleagues launched the UK-first Masters Program in Grid...
and Dr. Na Helian first proposed Grid-oriented Storage (GOS) in a funding proposal to the UK government titled “Grid-Oriented Storage (GOS): Next Generation Data Storage System Architecture for the Grid Computing Era” in 2003. The proposal was approved and granted one million pounds in 2004. The first prototype was constructed in 2005 at Centre for Grid Computing, Cambridge-Cranfield High Performance Computing Facility. The first conference presentation was at IEEE Symposium on Cluster Computing and Grid (CCGrid), 9–12 May 2005, Cardiff, UK. As one of the five best work-in-progress, it was included in the IEEE Distributed Systems Online. In 2006, the GOS architecture and its implementations was published in IEEE Transactions on Computers, titled “Grid-oriented Storage: A Single-Image, Cross-Domain, High-Bandwidth Architecture”. From January 2007, the GOS invention led to invitations to present demonstrations at Princeton University, Cambridge University (Computer Lab), IBM, Rolls Royce, BBC, Xerox, Carnegie Mellon University, CERN, and EADS, etc. A number of collaborations have been fostered with industrial giants.
Further reading
- Frank Wang, Na Helian, Sining Wu, Yuhui Deng, Yike Guo, Steve Thompson, Ian Johnson, Dave Milward & Robert Maddock, Grid-Oriented Storage, IEEE Distributed Systems Online, Volume 6, Issue 9, Sept. 2005.
- Frank Wang, Sining Wu, Na Helian, Andy Parker, Yike Guo, Yuhui Deng, Vineet Khare, Grid-oriented Storage: A Single-Image, Cross-Domain, High-Bandwidth Architecture, IEEE Transaction on Computers, Vol.56, No.4, pp. 474–487, 2007.
- Frank Zhigang Wang, Sining Wu, Na Helian, An Underlying Data-Transporting Protocol for Accelerating Web Communications, International Journal of Computer Networks, Elsevier, 2007.
- Frank Zhigang Wang, Sining Wu, Na Helian, Yuhui Deng, Vineet Khare, Chris Thompson and Michael Parker, Grid-based Data Access to Nucleotide Sequence Database with 6x Improvement in Response Times, New Generation Computing, No.2, Vol.25, 2007.
- Frank Wang, Yuhui Deng, Na Helian, Evolutionary Storage: Speeding up a Magnetic Disk by Clustering Frequent Data, IEEE Transactions on Magnetics, Issue.6, Vol.43, 2007.
- Frank Zhigang Wang, Na Helian, Sining Wu, Yuhui Deng, Vineet Khare, Chris Thompson and Michael Parker, Grid-based Storage Architecture for Accelerating Bioinformatics Computing, Journal of VLSI Signal Processing Systems, No.1, Vol.48, 2007.
- Yuhui Deng and Frank Wang, A Heterogeneous Storage Grid Enabled by Grid Service, ACM Operating System Review, No.1, Vol.41, 2007.
- Yuhui Deng & Frank Wang, Optimal Clustering Size of Small File Access in Network Attached Storage Device, Parallel Processing Letters, No.1, Vol.17, 2007.