Virtual file system
Encyclopedia
A virtual file system or virtual filesystem switch is an abstraction layer on top of a more concrete file system
. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows
, Mac OS
and Unix
filesystems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing.
A VFS specifies an interface
(or a "contract") between the kernel and a concrete file system. Therefore, it is easy to add support for new file system types to the kernel simply by fulfilling the contract. The terms of the contract might change incompatibly from release to release, which would require that concrete file system support be recompiled, and possibly modified before recompilation, to allow it to work with a new release of the operating system; or the supplier of the operating system might make only backward-compatible changes to the contract, so that concrete file system support built for a given release of the operating system would work with future versions of the operating system.
systems was introduced by Sun Microsystems
in SunOS
2.0 in 1985. It allowed Unix system calls to access local UFS
file systems and remote NFS file systems transparently. For this reason, Unix vendors who licensed the NFS code from Sun often copied the design of Sun's VFS. Other file systems could be plugged into it also: there was an implementation of the MS-DOS
FAT
file system developed at Sun that plugged into the SunOS VFS, although it wasn't shipped as a product until SunOS 4.1. The SunOS implementation was the basis of the VFS mechanism in System V Release 4.
John Heidemann developed a stacking VFS under SunOS 4.0 for the experimental Ficus file system. This design provided for code reuse
among file system types with differing but similar semantics (e.g., an encrypting file system could reuse all of the naming and storage-management code of a non-encrypting file system). Heidemann adapted this work for use in 4.4BSD as a part of his thesis
research; descendants of this code underpin the file system implementations in modern BSD derivatives including Mac OS X
.
Other Unix virtual file systems include the File System Switch in System V Release 3, the Generic File System in Ultrix
, and the VFS in Linux
. In OS/2
and Microsoft Windows
, the virtual file system mechanism is called the Installable File System
.
The Filesystem in Userspace
(FUSE) mechanism allows userland code to plug into the virtual file system mechanism in Linux, NetBSD
, FreeBSD
, OpenSolaris
, and Mac OS X.
In Microsoft Windows, virtual filesystems can also be implemented through userland Shell namespace extensions; however, they do not support the lowest-level file system access application programming interface
s in Windows, so not all applications will be able to access file systems that are implemented as namespace extensions. KIO
and GVFS
/GIO
provide similar mechanisms in the KDE
and GNOME
desktop environments (respectively), with similar limitations, although they can be made to use FUSE techniques and therefore integrate smoothly into the system.
, Microsoft's Virtual PC, VMware
.
The primary benefit for this type of file system is that it is centralized and easy to remove. A single-file virtual file system may include all the basic features expected of any file system (virtual or otherwise), but access to the internal structure of these file systems is often limited to programs specifically written to make use of the single-file virtual file system (instead of implementation through a driver allowing universal access). Another major drawback is that performance is relatively low when compared to other virtual file systems. Low performance is mostly due to the cost of shuffling virtual files when data is written or deleted from the virtual file system.
based machine clocked at 4.77MHz (and later an 80486SX clocked at 25 MHz). Users of PCTask could create a file of large size on the Amiga filesystem, and this file would be virtually accessed from the emulator as if it were a real PC Hard Disk. The file could be formatted with the FAT16 filesystem to store normal MS-DOS or Windows files.
, WinUAE, allows for large single files on Windows to be treated as Amiga file systems. In WinUAE this file is called a hardfile.
UAE could also treat a directory on the host filesystem -- (Windows
, Linux
, Mac OS
, AmigaOS
) -- as an Amiga filesystem.
C library for POSIX
-compliant operating systems which provides features for accessing and manipulating a single-file virtual file system from within C
and C++ applications.
File system
A file system is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device which contain it. A file system organizes data in an efficient manner and is tuned to the...
. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, Mac OS
Mac OS
Mac OS is a series of graphical user interface-based operating systems developed by Apple Inc. for their Macintosh line of computer systems. The Macintosh user experience is credited with popularizing the graphical user interface...
and Unix
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
filesystems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing.
A VFS specifies an interface
Interface (computer science)
In the field of computer science, an interface is a tool and concept that refers to a point of interaction between components, and is applicable at the level of both hardware and software...
(or a "contract") between the kernel and a concrete file system. Therefore, it is easy to add support for new file system types to the kernel simply by fulfilling the contract. The terms of the contract might change incompatibly from release to release, which would require that concrete file system support be recompiled, and possibly modified before recompilation, to allow it to work with a new release of the operating system; or the supplier of the operating system might make only backward-compatible changes to the contract, so that concrete file system support built for a given release of the operating system would work with future versions of the operating system.
Implementations
One of the first virtual file system mechanisms in Unix-likeUnix-like
A Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification....
systems was introduced by Sun Microsystems
Sun Microsystems
Sun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
in SunOS
SunOS
SunOS is a version of the Unix operating system developed by Sun Microsystems for their workstation and server computer systems. The SunOS name is usually only used to refer to versions 1.0 to 4.1.4 of SunOS...
2.0 in 1985. It allowed Unix system calls to access local UFS
Unix File System
The Unix file system is a file system used by many Unix and Unix-like operating systems. It is also called the Berkeley Fast File System, the BSD Fast File System or FFS...
file systems and remote NFS file systems transparently. For this reason, Unix vendors who licensed the NFS code from Sun often copied the design of Sun's VFS. Other file systems could be plugged into it also: there was an implementation of the MS-DOS
MS-DOS
MS-DOS is an operating system for x86-based personal computers. It was the most commonly used member of the DOS family of operating systems, and was the main operating system for IBM PC compatible personal computers during the 1980s to the mid 1990s, until it was gradually superseded by operating...
FAT
File Allocation Table
File Allocation Table is a computer file system architecture now widely used on many computer systems and most memory cards, such as those used with digital cameras. FAT file systems are commonly found on floppy disks, flash memory cards, digital cameras, and many other portable devices because of...
file system developed at Sun that plugged into the SunOS VFS, although it wasn't shipped as a product until SunOS 4.1. The SunOS implementation was the basis of the VFS mechanism in System V Release 4.
John Heidemann developed a stacking VFS under SunOS 4.0 for the experimental Ficus file system. This design provided for code reuse
Code reuse
Code reuse, also called software reuse, is the use of existing software, or software knowledge, to build new software.-Overview:Ad hoc code reuse has been practiced from the earliest days of programming. Programmers have always reused sections of code, templates, functions, and procedures...
among file system types with differing but similar semantics (e.g., an encrypting file system could reuse all of the naming and storage-management code of a non-encrypting file system). Heidemann adapted this work for use in 4.4BSD as a part of his thesis
Thesis
A dissertation or thesis is a document submitted in support of candidature for an academic degree or professional qualification presenting the author's research and findings...
research; descendants of this code underpin the file system implementations in modern BSD derivatives including Mac OS X
Mac OS X
Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
.
Other Unix virtual file systems include the File System Switch in System V Release 3, the Generic File System in Ultrix
Ultrix
Ultrix was the brand name of Digital Equipment Corporation's native Unix systems. While ultrix is the Latin word for avenger, the name was chosen solely for its sound.-History:...
, and the VFS in Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
. In OS/2
OS/2
OS/2 is a computer operating system, initially created by Microsoft and IBM, then later developed by IBM exclusively. The name stands for "Operating System/2," because it was introduced as part of the same generation change release as IBM's "Personal System/2 " line of second-generation personal...
and Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, the virtual file system mechanism is called the Installable File System
Installable File System
The Installable File System is a filesystem API in IBM OS/2 and Microsoft Windows that enables the operating system to recognize and load drivers for file systems...
.
The Filesystem in Userspace
Filesystem in Userspace
Filesystem in Userspace is a loadable kernel module for Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code...
(FUSE) mechanism allows userland code to plug into the virtual file system mechanism in Linux, NetBSD
NetBSD
NetBSD is a freely available open source version of the Berkeley Software Distribution Unix operating system. It was the second open source BSD descendant to be formally released, after 386BSD, and continues to be actively developed. The NetBSD project is primarily focused on high quality design,...
, FreeBSD
FreeBSD
FreeBSD is a free Unix-like operating system descended from AT&T UNIX via BSD UNIX. Although for legal reasons FreeBSD cannot be called “UNIX”, as the direct descendant of BSD UNIX , FreeBSD’s internals and system APIs are UNIX-compliant...
, OpenSolaris
OpenSolaris
OpenSolaris was an open source computer operating system based on Solaris created by Sun Microsystems. It was also the name of the project initiated by Sun to build a developer and user community around the software...
, and Mac OS X.
In Microsoft Windows, virtual filesystems can also be implemented through userland Shell namespace extensions; however, they do not support the lowest-level file system access application programming interface
Application programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
s in Windows, so not all applications will be able to access file systems that are implemented as namespace extensions. KIO
KIO
KIO is part of the KDE architecture. It provides access to files, web sites and other resources through a single consistent API. Applications, such as Konqueror which are written using this framework can operate on files stored on remote servers in exactly the same way as they operate on those...
and GVFS
GVFS
GVFS is the virtual filesystem for the GNOME desktop, which allows users easy access to remote data via SFTP, FTP, WebDAV, SMB, and local data via HAL integration, OBEX and others....
/GIO
GIO (Gnome)
GIO provides a modern, easy-to-use VFS API that sits at the right level in the library stack. The goal is to overcome the shortcomings of GnomeVFS and provide an API that is so good that developers prefer it over raw POSIX calls. It ships with GLib as a separate library called libgio-2.0...
provide similar mechanisms in the KDE
KDE
KDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Microsoft Windows, Solaris and Mac OS X systems...
and GNOME
GNOME
GNOME is a desktop environment and graphical user interface that runs on top of a computer operating system. It is composed entirely of free and open source software...
desktop environments (respectively), with similar limitations, although they can be made to use FUSE techniques and therefore integrate smoothly into the system.
Single-file virtual file systems
Sometimes Virtual File System refers to a file or a bunch of files (not necessarily inside a concrete file system) that acts as a manageable container which should provide the functionality of a concrete file system through the usage of a software. Examples of such containers are SolFS or a single-file virtual file system in an emulator like PCTask or so-called WinUAE, Sun's VirtualBoxVirtualBox
Oracle VM VirtualBox is an x86 virtualization software package, originally created by software company Innotek GmbH, purchased by Sun Microsystems, and now developed by Oracle Corporation as part of its family of virtualization products...
, Microsoft's Virtual PC, VMware
VMware
VMware, Inc. is a company providing virtualization software founded in 1998 and based in Palo Alto, California, USA. The company was acquired by EMC Corporation in 2004, and operates as a separate software subsidiary ....
.
The primary benefit for this type of file system is that it is centralized and easy to remove. A single-file virtual file system may include all the basic features expected of any file system (virtual or otherwise), but access to the internal structure of these file systems is often limited to programs specifically written to make use of the single-file virtual file system (instead of implementation through a driver allowing universal access). Another major drawback is that performance is relatively low when compared to other virtual file systems. Low performance is mostly due to the cost of shuffling virtual files when data is written or deleted from the virtual file system.
Implementation of single-file virtual filesystems
Direct examples of single-file virtual file systems include emulators, such as PCTask and WinUAE, which encapsulate not only the filesystem data but also emulated disk layout. This makes it easy to treat an OS installation like any other piece of software—transferring it with removable media or over the network.PCTask
The Amiga emulator PCTask emulated an Intel PC 8088Intel 8088
The Intel 8088 microprocessor was a variant of the Intel 8086 and was introduced on July 1, 1979. It had an 8-bit external data bus instead of the 16-bit bus of the 8086. The 16-bit registers and the one megabyte address range were unchanged, however...
based machine clocked at 4.77MHz (and later an 80486SX clocked at 25 MHz). Users of PCTask could create a file of large size on the Amiga filesystem, and this file would be virtually accessed from the emulator as if it were a real PC Hard Disk. The file could be formatted with the FAT16 filesystem to store normal MS-DOS or Windows files.
WinUAE
The UAE for WindowsMicrosoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, WinUAE, allows for large single files on Windows to be treated as Amiga file systems. In WinUAE this file is called a hardfile.
UAE could also treat a directory on the host filesystem -- (Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
, Mac OS
Mac OS
Mac OS is a series of graphical user interface-based operating systems developed by Apple Inc. for their Macintosh line of computer systems. The Macintosh user experience is credited with popularizing the graphical user interface...
, AmigaOS
AmigaOS
AmigaOS is the default native operating system of the Amiga personal computer. It was developed first by Commodore International, and initially introduced in 1985 with the Amiga 1000...
) -- as an Amiga filesystem.
whefs
whefs (http://code.google.com/p/whefs) is an Open SourceOpen source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
C library for POSIX
POSIX
POSIX , an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems...
-compliant operating systems which provides features for accessing and manipulating a single-file virtual file system from within C
C (programming language)
C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
and C++ applications.
See also
- 9P9P9P is a network protocol developed for the Plan 9 from Bell Labs distributed operating system as the means of connecting the components of a Plan 9 system. Files are key objects in Plan 9. They represent windows, network connections, processes, and almost anything else available in the operating...
- Distributed file system protocol that maps directly to the Plan 9 from Bell LabsPlan 9 from Bell LabsPlan 9 from Bell Labs is a distributed operating system. It was developed primarily for research purposes as the successor to Unix by the Computing Sciences Research Center at Bell Labs between the mid-1980s and 2002...
VFS layer making all file system access network transparent. - Toronto Virtual File System, a VFS for OS/2OS/2OS/2 is a computer operating system, initially created by Microsoft and IBM, then later developed by IBM exclusively. The name stands for "Operating System/2," because it was introduced as part of the same generation change release as IBM's "Personal System/2 " line of second-generation personal...
developed by IBM Toronto that allows mounting diverse filesystems under a common structure.
External links
- Embedded File System (EFS) - Open Source cross-platform C++ implementation of Virtual File System
- AVFS - A Virtual File System for mounting compressed or remote files
- fs-driver Ext2 Installable File System for Microsoft Windows
- Anatomy of the Linux file system by M. Tim Jones
- Solid File System - (SolFS) cross-platform single-file virtual file system with encryption and compression
- Callback File System - SDK that lets developers create virtual file systems for Windows in user mode
- FUSE - Filesystem in Userspace - virtual filesystem for Linux.
- LUFS - Linux Userland FileSystem - virtual filesystem with support of localfs, sshSecure ShellSecure Shell is a network protocol for secure data communication, remote shell services or command execution and other secure network services between two networked computers that it connects via a secure channel over an insecure network: a server and a client...
fs , ftpfs, gnutellaGnutellaGnutella is a large peer-to-peer network which, at the time of its creation, was the first decentralized peer-to-peer network of its kind, leading to other, later networks adopting the model...
fs, locasefs, gvfsGVFSGVFS is the virtual filesystem for the GNOME desktop, which allows users easy access to remote data via SFTP, FTP, WebDAV, SMB, and local data via HAL integration, OBEX and others....
, cardfs, cefs and more. Latest file release: 2003-10-29 - TrueZIP - Virtual File System for Java, with thread-safe read/write I/O for ZIP, ZIP.RAES, TAR, TAR.BZ2, TAR.GZ, HTTP(S) etc and multiple client API options.
- Commons VFS - virtual filesystem for Java, with support for Cifs, ftp, http, Zip (file format)ZIP (file format)Zip is a file format used for data compression and archiving. A zip file contains one or more files that have been compressed, to reduce file size, or stored as is...
, Tar (file format)Tar (file format)In computing, tar is both a file format and the name of a program used to handle such files...
, gzipGzipGzip is any of several software applications used for file compression and decompression. The term usually refers to the GNU Project's implementation, "gzip" standing for GNU zip. It is based on the DEFLATE algorithm, which is a combination of Lempel-Ziv and Huffman coding...
, bzip2Bzip2bzip2 is a free and open source implementation of the Burrows–Wheeler algorithm. It is developed and maintained by Julian Seward. Seward made the first public release of bzip2, version 0.15, in July 1996.-Compression efficiency:...
, and more. - MillScript VFS - virtual filesystem for Java, influenced by the KIO subsystem in KDE, Steve Leach's work on a VFS in JSpice and to a limited extent the Apache Commons VFS.
- KIO - (KDEKDEKDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Microsoft Windows, Solaris and Mac OS X systems...
IOInput/outputIn computing, input/output, or I/O, refers to the communication between an information processing system , and the outside world, possibly a human, or another information processing system. Inputs are the signals or data received by the system, and outputs are the signals or data sent from it...
) - a network-enabled file management system - flipcode - Programming a Virtual File System
- ZX-VFS - A free and open source single-file virtual filesystem for Java.
- Dokan - A free and open source virtual filesystem for Windows (includes C, .NET, and Ruby bindings).