ReiserFS
Encyclopedia
ReiserFS is a general-purpose, journaled computer file system
designed and implemented by a team at Namesys
led by Hans Reiser
. ReiserFS is currently supported on Linux
(without quota support). Introduced in version 2.4.1 of the Linux kernel
, it was the first journaling file system to be included in the standard kernel. ReiserFS is the default file system on the Elive
, Xandros
, Linspire
, GoboLinux
, and Yoper
Linux distribution
s. ReiserFS was the default file system in Novell
's SUSE Linux Enterprise until Novell decided to move to ext3
on October 12, 2006 for future releases.
Namesys considered ReiserFS (now occasionally referred to as Reiser3) stable and feature-complete and, with the exception of security updates and critical bug fixes, ceased development on it to concentrate on its successor, Reiser4
. Namesys went out of business in 2008 after Reiser was charged for the murder of his wife. However, volunteers continue to work on the open source project.
and ext3
in version 2.4 of the Linux kernel, when dealing with files under 4 KiB and with tail packing enabled, ReiserFS may be faster. This was said to be of great benefit in Usenet
news spools, HTTP caches, mail delivery systems and other applications where performance with small files is critical. However, in practice news spools use a feature called cycbuf, which holds articles in one large file; fast HTTP caches and several revision control
systems use similar approach, nullifying these performance advantages. For email servers, reiserfs was problematic due to semantic problems explained below. Also, ReiserFS had a problem with very fast filesystem aging when compared to other filesystems – in several usage scenarios filesystem performance decreased dramatically with time.
Before Linux 2.6.33, ReiserFS heavily used the big kernel lock
(BKL) — a global kernel-wide lock — which does not scale very well for systems with multiple cores, as the critical code parts are only ever executed by one core at a time.
on ReiserFS, which can result in data corruption with applications relying heavily on file-based locks (such as mail transfer agent
s qmail
and Postfix
) if the machine halts before it has synchronized the disk.
There are no programs to specifically defragment
a ReiserFS file system, although tools have been written to automatically copy the contents of fragmented files hoping that more contiguous blocks of free space can be found. However, a "repacker" tool was planned for the next Reiser4 file system to deal with file fragmentation.
has attracted much criticism: if the file system becomes so badly corrupted that its internal tree is unusable, performing a tree rebuild operation may further corrupt existing files or introduce new entries with unexpected contents, but this action is not part of normal operation or a normal file system check and has to be explicitly initiated and confirmed by the administrator.
ReiserFS v3 images should not be stored on a ReiserFS v3 partition (e.g. backups or disk image
s for emulators) without transforming them (e.g., by compressing or encrypting) in order to avoid confusing the rebuild. Reformatting an existing ReiserFS v3 partition can also leave behind data that could confuse the rebuild operation and make files from the old system reappear. This also allows malicious users to intentionally store files that will confuse the rebuilder. As the metadata is always in a consistent state after a file system check, corruption here means that contents of files are merged in unexpected ways with the contained file system's metadata. The ReiserFS successor, Reiser4, fixes this problem.
Early implementations of ReiserFS (prior to that in Linux 2.6.2) were also susceptible to out-of-order write hazards. But the current journaling implementation in ReiserFS is now on par with that of ext3
's "ordered" journaling level.
for the default installation file system. Some reasons he mentioned were scalability, "performance problems with extended attributes and ACL
s", "a small and shrinking development community", and that "Reiser4
is not an incremental update and requires a reformat, which is unreasonable for most people." On October 4 he wrote a response comment on a blog in order to clear up some issues. He wrote that his proposal for the switch was unrelated to Reiser's "legal troubles" (i.e., Hans Reiser's prosecution for murdering his wife
) Mahoney wrote he "was concerned that people would make a connection where none existed" and that "the timing is entirely coincidental and the motivation is unrelated."
On Oct 12, 2006, Novell
similarly announced that SuSE Linux Enterprise would switch from ReiserFS to ext3.
entries ("directory items"), inode
block lists ("indirect items"), and tails of files
("direct items") in a single, combined B+ tree
keyed by a universal object ID. Disk blocks allocated to nodes of the tree are "formatted internal blocks". Blocks for leaf nodes (in which items are packed end-to-end) are "formatted leaf blocks". All other blocks are "unformatted blocks" containing file contents. Directory items with too many entries or indirect items which are too long to fit into a node spill over into the right leaf neighbour. Block allocation is tracked by free space bitmap
s in fixed locations.
By contrast, ext2
and other Berkeley FFS-like file systems of that time simply used a fixed formula for computing inode locations, hence limiting the number of files they may contain. Most such file systems also store directories as simple lists of entries, which makes directory lookups and updates linear time operations and degrades performance on very large directories. The single B+ tree
design in ReiserFS avoids both of these problems due to better scalability properties.
Journaling file system
A journaling file system is a file system that keeps track of the changes that will be made in a journal before committing them to the main file system...
designed and implemented by a team at Namesys
Namesys
Namesys was a California corporation responsible for the design and implementation of the ReiserFS and Reiser4 filesystems. It has been inactive since late 2007, and as of 2010, is listed with the State of California with a status of "Suspended". Owned by Hans Reiser, Namesys was based in Oakland,...
led by Hans Reiser
Hans Reiser
Hans Thomas Reiser is an American computer programmer, entrepreneur, and convicted murderer. He is the creator and primary developer of the ReiserFS computer file system, which is contained within the Linux kernel, as well as its attempted successor, Reiser4. In 2004 he founded Namesys, a...
. ReiserFS is currently supported on Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
(without quota support). Introduced in version 2.4.1 of the Linux kernel
Linux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
, it was the first journaling file system to be included in the standard kernel. ReiserFS is the default file system on the Elive
Elive
Elive is a GNU/Linux distribution based on Debian for legacy and commodity hardware with Intel processors. Elive uses the Enlightenment X window manager instead of GNOME or KDE...
, Xandros
Xandros
Xandros is the name of both a line of operating systems and Xandros Corporation, the company which creates them. Xandros Desktop was a Linux distribution. The name Xandros is derived from the X Window System and the Greek island of Andros....
, Linspire
Linspire
Linspire, previously known as LindowsOS, was a commercial operating system based on Debian GNU/Linux and later Ubuntu. Linspire was published by Linspire, Inc. and focused on ease-of-use, targeting home PC users...
, GoboLinux
GoboLinux
GoboLinux is a free and open source operating system whose most prominent feature is a reorganization of the traditional Linux file system. Rather than following the Filesystem Hierarchy Standard like most Unix-like systems, each program in a GoboLinux system has its own subdirectory tree, where...
, and Yoper
YOPER
Yoper Linux—Your Operating System—is a Linux distribution for PCs with i686 or higher processor types. It can be used for both personal computers and server use and uses hardware recognition tools known from Knoppix...
Linux distribution
Linux distribution
A Linux distribution is a member of the family of Unix-like operating systems built on top of the Linux kernel. Such distributions are operating systems including a large collection of software applications such as word processors, spreadsheets, media players, and database applications...
s. ReiserFS was the default file system in Novell
Novell
Novell, Inc. is a multinational software and services company. It is a wholly owned subsidiary of The Attachmate Group. It specializes in network operating systems, such as Novell NetWare; systems management solutions, such as Novell ZENworks; and collaboration solutions, such as Novell Groupwise...
's SUSE Linux Enterprise until Novell decided to move to ext3
Ext3
The ext3 or third extended filesystem is a journaled file system that is commonly used by the Linux kernel. It is the default file system for many popular Linux distributions, including Debian...
on October 12, 2006 for future releases.
Namesys considered ReiserFS (now occasionally referred to as Reiser3) stable and feature-complete and, with the exception of security updates and critical bug fixes, ceased development on it to concentrate on its successor, Reiser4
Reiser4
Reiser4 is a computer file system, successor to the ReiserFS file system, developed from scratch by Namesys and sponsored by DARPA as well as Linspire...
. Namesys went out of business in 2008 after Reiser was charged for the murder of his wife. However, volunteers continue to work on the open source project.
Features
At the time of its introduction, ReiserFS offered features that had not been available in existing Linux file systems:- MetadataMetadataThe term metadata is an ambiguous term which is used for two fundamentally different concepts . Although the expression "data about data" is often used, it does not apply to both in the same way. Structural metadata, the design and specification of data structures, cannot be about data, because at...
-only journalingJournaling file systemA journaling file system is a file system that keeps track of the changes that will be made in a journal before committing them to the main file system...
(also block journaling, since Linux 2.6.8), its most-publicized advantage over what was the stock Linux file system at the time, ext2Ext2The ext2 or second extended filesystem is a file system for the Linux kernel. It was initially designed by Rémy Card as a replacement for the extended file system ....
. - Online resizing (growth only), with or without an underlying volume manager such as LVMLogical Volume Manager (Linux)LVM is a logical volume manager for the Linux kernel; it manages disk drives and similar mass-storage devices, in particular large ones. The term "volume" refers to a disk drive or partition thereof...
. Since then, Namesys has also provided tools to resize (both grow and shrink) ReiserFS file systems offline. - Tail packing, a scheme to reduce internal fragmentation. Tail packing, however, can have a significant performance impact. Reiser4 may have improved this by packing tails where it does not hurt performance.
Performance
Compared with ext2Ext2
The ext2 or second extended filesystem is a file system for the Linux kernel. It was initially designed by Rémy Card as a replacement for the extended file system ....
and ext3
Ext3
The ext3 or third extended filesystem is a journaled file system that is commonly used by the Linux kernel. It is the default file system for many popular Linux distributions, including Debian...
in version 2.4 of the Linux kernel, when dealing with files under 4 KiB and with tail packing enabled, ReiserFS may be faster. This was said to be of great benefit in Usenet
Usenet
Usenet is a worldwide distributed Internet discussion system. It developed from the general purpose UUCP architecture of the same name.Duke University graduate students Tom Truscott and Jim Ellis conceived the idea in 1979 and it was established in 1980...
news spools, HTTP caches, mail delivery systems and other applications where performance with small files is critical. However, in practice news spools use a feature called cycbuf, which holds articles in one large file; fast HTTP caches and several revision control
Revision control
Revision control, also known as version control and source control , is the management of changes to documents, programs, and other information stored as computer files. It is most commonly used in software development, where a team of people may change the same files...
systems use similar approach, nullifying these performance advantages. For email servers, reiserfs was problematic due to semantic problems explained below. Also, ReiserFS had a problem with very fast filesystem aging when compared to other filesystems – in several usage scenarios filesystem performance decreased dramatically with time.
Before Linux 2.6.33, ReiserFS heavily used the big kernel lock
Giant lock
In operating systems, a giant lock, also known as a big-lock or kernel-lock, is a lock which may be used in the kernel to provide the concurrency control required by symmetric multiprocessing systems....
(BKL) — a global kernel-wide lock — which does not scale very well for systems with multiple cores, as the critical code parts are only ever executed by one core at a time.
Criticism
Some directory operations (including unlink(2)) are not synchronousSynchronization
Synchronization is timekeeping which requires the coordination of events to operate a system in unison. The familiar conductor of an orchestra serves to keep the orchestra in time....
on ReiserFS, which can result in data corruption with applications relying heavily on file-based locks (such as mail transfer agent
Mail transfer agent
Within Internet message handling services , a message transfer agent or mail transfer agent or mail relay is software that transfers electronic mail messages from one computer to another using a client–server application architecture...
s qmail
Qmail
qmail is a mail transfer agent that runs on Unix. It was written, starting December 1995, by Daniel J. Bernstein as a more secure replacement for the popular Sendmail program...
and Postfix
Postfix (software)
In computing, Postfix is a free and open-source mail transfer agent that routes and delivers electronic mail. It is intended as a fast, easier-to-administer, and secure alternative to the widely-used Sendmail MTA....
) if the machine halts before it has synchronized the disk.
There are no programs to specifically defragment
Defragmentation
In the maintenance of file systems, defragmentation is a process that reduces the amount of fragmentation. It does this by physically organizing the contents of the mass storage device used to store files into the smallest number of contiguous regions . It also attempts to create larger regions of...
a ReiserFS file system, although tools have been written to automatically copy the contents of fragmented files hoping that more contiguous blocks of free space can be found. However, a "repacker" tool was planned for the next Reiser4 file system to deal with file fragmentation.
fsck
The tree rebuild process of ReiserFS's fsckFsck
The system utility fsck is a tool for checking the consistency of a file system in Unix and Unix-like operating systems such as Linux.-Use:...
has attracted much criticism: if the file system becomes so badly corrupted that its internal tree is unusable, performing a tree rebuild operation may further corrupt existing files or introduce new entries with unexpected contents, but this action is not part of normal operation or a normal file system check and has to be explicitly initiated and confirmed by the administrator.
ReiserFS v3 images should not be stored on a ReiserFS v3 partition (e.g. backups or disk image
Disk image
A disk image is a single file or storage device containing the complete contents and structure representing a data storage medium or device, such as a hard drive, tape drive, floppy disk, CD/DVD/BD, or USB flash drive, although an image of an optical disc may be referred to as an optical disc image...
s for emulators) without transforming them (e.g., by compressing or encrypting) in order to avoid confusing the rebuild. Reformatting an existing ReiserFS v3 partition can also leave behind data that could confuse the rebuild operation and make files from the old system reappear. This also allows malicious users to intentionally store files that will confuse the rebuilder. As the metadata is always in a consistent state after a file system check, corruption here means that contents of files are merged in unexpected ways with the contained file system's metadata. The ReiserFS successor, Reiser4, fixes this problem.
Earlier issues
ReiserFS in versions of the Linux kernel before 2.4.16 were considered unstable by Namesys and not recommended for production use, especially in conjunction with NFS.Early implementations of ReiserFS (prior to that in Linux 2.6.2) were also susceptible to out-of-order write hazards. But the current journaling implementation in ReiserFS is now on par with that of ext3
Ext3
The ext3 or third extended filesystem is a journaled file system that is commonly used by the Linux kernel. It is the default file system for many popular Linux distributions, including Debian...
's "ordered" journaling level.
Novell / SuSE move away from ReiserFS to ext3
Jeff Mahoney of SuSE wrote a post on Sep 14 2006 proposing to move from ReiserFS to ext3Ext3
The ext3 or third extended filesystem is a journaled file system that is commonly used by the Linux kernel. It is the default file system for many popular Linux distributions, including Debian...
for the default installation file system. Some reasons he mentioned were scalability, "performance problems with extended attributes and ACL
Access control list
An access control list , with respect to a computer file system, is a list of permissions attached to an object. An ACL specifies which users or system processes are granted access to objects, as well as what operations are allowed on given objects. Each entry in a typical ACL specifies a subject...
s", "a small and shrinking development community", and that "Reiser4
Reiser4
Reiser4 is a computer file system, successor to the ReiserFS file system, developed from scratch by Namesys and sponsored by DARPA as well as Linspire...
is not an incremental update and requires a reformat, which is unreasonable for most people." On October 4 he wrote a response comment on a blog in order to clear up some issues. He wrote that his proposal for the switch was unrelated to Reiser's "legal troubles" (i.e., Hans Reiser's prosecution for murdering his wife
Hans Reiser
Hans Thomas Reiser is an American computer programmer, entrepreneur, and convicted murderer. He is the creator and primary developer of the ReiserFS computer file system, which is contained within the Linux kernel, as well as its attempted successor, Reiser4. In 2004 he founded Namesys, a...
) Mahoney wrote he "was concerned that people would make a connection where none existed" and that "the timing is entirely coincidental and the motivation is unrelated."
On Oct 12, 2006, Novell
Novell
Novell, Inc. is a multinational software and services company. It is a wholly owned subsidiary of The Attachmate Group. It specializes in network operating systems, such as Novell NetWare; systems management solutions, such as Novell ZENworks; and collaboration solutions, such as Novell Groupwise...
similarly announced that SuSE Linux Enterprise would switch from ReiserFS to ext3.
Design
ReiserFS stores file metadata ("stat items"), directoryDirectory (file systems)
In computing, a folder, directory, catalog, or drawer, is a virtual container originally derived from an earlier Object-oriented programming concept by the same name within a digital file system, in which groups of computer files and other folders can be kept and organized.A typical file system may...
entries ("directory items"), inode
Inode
In computing, an inode is a data structure on a traditional Unix-style file system such as UFS. An inode stores all the information about a regular file, directory, or other file system object, except its data and name....
block lists ("indirect items"), and tails of files
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
("direct items") in a single, combined B+ tree
B+ tree
In computer science, a B+ tree or B plus tree is a type of tree which represents sorted data in a way that allows for efficient insertion, retrieval and removal of records, each of which is identified by a key. It is a dynamic, multilevel index, with maximum and minimum bounds on the number of...
keyed by a universal object ID. Disk blocks allocated to nodes of the tree are "formatted internal blocks". Blocks for leaf nodes (in which items are packed end-to-end) are "formatted leaf blocks". All other blocks are "unformatted blocks" containing file contents. Directory items with too many entries or indirect items which are too long to fit into a node spill over into the right leaf neighbour. Block allocation is tracked by free space bitmap
Free space bitmap
Free space bitmaps are one method used to track allocated sectors by some file systems. While the most simplistic design is highly inefficient, advanced or hybrid implementations of free space bitmaps are used by some modern file systems.-Example:...
s in fixed locations.
By contrast, ext2
Ext2
The ext2 or second extended filesystem is a file system for the Linux kernel. It was initially designed by Rémy Card as a replacement for the extended file system ....
and other Berkeley FFS-like file systems of that time simply used a fixed formula for computing inode locations, hence limiting the number of files they may contain. Most such file systems also store directories as simple lists of entries, which makes directory lookups and updates linear time operations and degrades performance on very large directories. The single B+ tree
B+ tree
In computer science, a B+ tree or B plus tree is a type of tree which represents sorted data in a way that allows for efficient insertion, retrieval and removal of records, each of which is identified by a key. It is a dynamic, multilevel index, with maximum and minimum bounds on the number of...
design in ReiserFS avoids both of these problems due to better scalability properties.
External links
- Namesys web site (archived version from 2007 October)
- reiserfsprogs, new $HOME
- Theodore Ts'o critiques the ReiserFS block journaling implementation
- convertfs, a utility which performs in-place conversion between any two file systems with sparse file support
- An overview of the ReiserFS on-disk format
- Florian Buchholz's The structure of the Reiser file system, a very detailed description of the ReiserFS disk structures.
- Gentoo Forum Link – Discussion on ReiserFS fragmentation, including a script for measuring fragmentation and defragmenting files.
- Reiser4 Future Vision
- Windows utilities to access ReiserFS: RFSTOOL, YAReG – Yet Another R(eiser)FStool GUI, rfsd – ReiserDriver.