Home      Discussion      Topics      Dictionary      Almanac
Signup       Login
Backup

Backup

Overview
In information technology
Information technology
Information technology is the acquisition, processing, storage and dissemination of vocal, pictorial, textual and numerical information by a microelectronics-based combination of computing and telecommunications...

, a backup or the process of backing up is making copies of data
Data
The term data refers to qualitative or quantitative attributes of a variable or set of variables. Data are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which...

 which may be used to restore the original after a data loss
Data loss
Data loss is an error condition in information systems in which information is destroyed by failures or neglect in storage, transmission, or processing. Information systems implement backup and disaster recovery equipment and processes to prevent data loss or restore lost data.Data loss is...

 event. The verb form is back up in two words, whereas the noun is backup.

Backups have two distinct purposes. The primary purpose is to recover data after its loss, be it by data deletion or corruption
Data corruption
Data corruption refers to errors in computer data that occur during writing, reading, storage, transmission, or processing, which introduce unintended changes to the original data...

. Data loss is a very common experience of computer users.
Discussion
Ask a question about 'Backup'
Start a new discussion about 'Backup'
Answer questions from other users
Full Discussion Forum
 
Quotations

Backup: The duplicate copy of crucial data that no one bothered to make, used only in the abstract.

Anonymous

Only wimps use tape backup: _real_ men just upload their important stuff on FTP|ftp, and let the rest of the world mirror it ;)

Linus Torvalds

Back up my files?! Are you kidding? Is that a real thing you have to do? I always thought that that was just like... you know, a figure of speech.

"Strong Bad" of Homestar Runner

BANANA - Backups Are Not Archives, NOT ARCHIVES

Anderson McCammont, quoting someone else on the veritas-bu mailing list (3/27/2007).

I didn't lose my mind, I've got it backed up on cd somewhere.

Unknown Category:Themes
Encyclopedia
In information technology
Information technology
Information technology is the acquisition, processing, storage and dissemination of vocal, pictorial, textual and numerical information by a microelectronics-based combination of computing and telecommunications...

, a backup or the process of backing up is making copies of data
Data
The term data refers to qualitative or quantitative attributes of a variable or set of variables. Data are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which...

 which may be used to restore the original after a data loss
Data loss
Data loss is an error condition in information systems in which information is destroyed by failures or neglect in storage, transmission, or processing. Information systems implement backup and disaster recovery equipment and processes to prevent data loss or restore lost data.Data loss is...

 event. The verb form is back up in two words, whereas the noun is backup.

Backups have two distinct purposes. The primary purpose is to recover data after its loss, be it by data deletion or corruption
Data corruption
Data corruption refers to errors in computer data that occur during writing, reading, storage, transmission, or processing, which introduce unintended changes to the original data...

. Data loss is a very common experience of computer users. 67% of Internet users have suffered serious data loss. The secondary purpose of backups is to recover data from an earlier time, according to a user-defined data retention
Data retention
Data retention defines the policies of persistent data and records management for meeting legal and business data archival requirements. A data retention policy weighs legal and privacy concerns against economics and need to know concerns to determine both the retention time, archival rules, data...

 policy, typically configured within a backup application for how long copies of data are required.

Though backups popularly represent a simple form of disaster recovery
Disaster recovery
Disaster recovery is the process, policies and procedures related to preparing for recovery or continuation of technology infrastructure critical to an organization after a natural or human-induced disaster. Disaster recovery is a subset of business continuity...

, and should be part of a disaster recovery plan, by themselves, backups should not alone be considered disaster recovery. Not all backup systems or backup applications are able to reconstitute a computer system, or in turn other complex configurations such as a computer cluster, active directory
Active Directory
Active Directory is a directory service created by Microsoft for Windows domain networks. It is included in most Windows Server operating systems. Server computers on which Active Directory is running are called domain controllers....

 servers, or a database server
Database server
A database server is a computer program that provides database services to other computer programs or computers, as defined by the client–server model. The term may also refer to a computer dedicated to running such a program...

, by restoring only data from a backup.

Since a backup system contains at least one copy of all data worth saving, the data storage requirements are considerable. Organizing this storage space and managing the backup process is a complicated undertaking. A data repository model can be used to provide structure to the storage. In the modern era of computing there are many different types of data storage device
Data storage device
thumb|200px|right|A reel-to-reel tape recorder .The magnetic tape is a data storage medium. The recorder is data storage equipment using a portable medium to store the data....

s that are useful for making backups. There are also many different ways in which these devices can be arranged to provide geographic redundancy, data security
Data security
Data security is the means of ensuring that data is kept safe from corruption and that access to it is suitably controlled. Thus data security helps to ensure privacy. It also helps in protecting personal data. Data security is part of the larger practice of Information security.- Disk Encryption...

, and portability.

Before data is sent to its storage location, it is selected, extracted, and manipulated. Many different techniques have been developed to optimize the backup procedure. These include optimizations for dealing with open files and live data sources as well as compression, encryption, and de-duplication, among others. Many organizations and individuals try to have confidence that the process is working as expected and work to define measurements and validation techniques. It is also important to recognize the limitations and human factors involved in any backup scheme.

Data repository models


Any backup strategy starts with a concept of a data repository. The backup data needs to be stored somehow and probably should be organized to a degree. It can be as simple as a sheet of paper with a list of all backup tapes and the dates they were written or a more sophisticated setup with a computerized index, catalog, or relational database. Different repository models have different advantages. This is closely related to choosing a backup rotation scheme
Backup rotation scheme
A backup rotation scheme is a method for effectively backing up data where multiple media are used in the backup process. The scheme determines how and when each piece of removable storage is used for a backup job and how long it is retained once it has backup data stored on it...

.

Unstructured : An unstructured repository may simply be a stack of floppy disks or CD-R/DVD-R media with minimal information about what was backed up and when. This is the easiest to implement, but probably the least likely to achieve a high level of recoverability.
Full only / System imaging
System image
A system image in computing is a copy of the entire state of a computer system stored in some non-volatile form such as a file. A system is said to be capable of using system images if it can be shut down and later restored to exactly the same state...

: A repository of this type contains complete system images from one or more specific points in time. This technology is frequently used by computer technicians to record known good configurations. Imaging is generally more useful for deploying a standard configuration to many systems rather than as a tool for making ongoing backups of diverse systems.
Incremental
Incremental backup
An incremental backup preserves data by not creating multiple copies that are based on the differences in those data: a successive copy of the data contains only that portion which has changed since the preceding copy has been created.-Incremental:...

 / Differential
: An incremental style repository aims to make it more feasible to store backups from more points in time by organizing the data into increments of change between points in time. This eliminates the need to store duplicate copies of unchanged data. Typically to start out, a full backup (of all files) is made. After that, any number of incremental or differential backups can be made. Restoring a whole system to a certain point in time would require locating the last full backup taken previous to that time and all the incremental / differential backups that cover the period of time between the full backup and the particular point in time to which the system is supposed to be restored. Additionally, some backup systems can reorganize the repository to synthesize full backups from a series of incrementals.
Note: Different implementations of backup systems frequently use specialized or conflicting definitions of these terms. The most relevant characteristic of a type of incremental backup is which reference point is it using to check for changes. By a common definition, a differential backup copies files that have been created or changed since the last full backup, regardless of whether any other backups have been since then and an incremental backup refers only to a backup that includes just the changes made since the most recent backup of any type. Other variations include multi-level incrementals and incremental backups that compare parts of files instead of just the whole file.

Reverse delta : A reverse delta type repository stores a recent "mirror" of the source data and a series of differences between the mirror in its current state and its previous states. A reverse delta backup will start with a normal full backup. After the full backup is performed, the system will periodically synchronize the full backup with the live copy, while storing the data necessary to reconstruct older versions. This can either be done using hard links, or using binary diffs. This system works particularly well for large, slowly changing, data sets. Examples of programs that use this method are rdiff-backup and Time Machine
Time Machine (Apple software)
Time Machine is a backup utility developed by Apple. It is included with Mac OS X and was introduced with the 10.5 "Leopard" release of Mac OS X. The software is designed to work with the Time Capsule as well as other internal or external drives.-Overview:...

.
Continuous data protection
Continuous data protection
Continuous data protection , also called continuous backup or real-time backup, refers to backup of computer data by automatically saving a copy of every change made to that data, essentially capturing every version of the data that the user saves...

: Instead of scheduling periodic backups, the system immediately logs every change on the host system. This is generally done by saving byte or block-level differences rather than file-level differences. It differs from simple disk mirroring
Disk mirroring
In data storage, disk mirroring or RAID1 is the replication of logical disk volumes onto separate physical hard disks in real time to ensure continuous availability...

 in that it enables a roll-back of the log and thus restoration of old image of data.

Storage media


Regardless of the repository model that is used, the data has to be stored on some data storage medium somewhere.

Magnetic tape
Magnetic tape data storage
Magnetic tape data storage uses digital recording on to magnetic tape to store digital information. Modern magnetic tape is most commonly packaged in cartridges and cassettes. The device that performs actual writing or reading of data is a tape drive...

 : Magnetic tape has long been the most commonly used medium for bulk data storage, backup, archiving, and interchange. Tape has typically had an order of magnitude better capacity/price ratio when compared to hard disk, but recently the ratios for tape and hard disk have become a lot closer. There are myriad formats, many of which are proprietary or specific to certain markets like mainframes or a particular brand of personal computer. Tape is a sequential access medium, so even though access times may be poor, the rate of continuously writing or reading data can actually be very fast. Some new tape drives are even faster than modern hard disks. A principal advantage of tape is that it has been used for this purpose for decades (much longer than any alternative) and its characteristics are well understood.
Hard disk
Hard disk
A hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...

 : The capacity/price ratio of hard disk has been rapidly improving for many years. This is making it more competitive with magnetic tape as a bulk storage medium. The main advantages of hard disk storage are low access times, availability, capacity and ease of use. External disks can be connected via local interfaces like SCSI
SCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard disks and tape drives, but it...

, USB, FireWire, or eSATA, or via longer distance technologies like Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....

, iSCSI
ISCSI
In computing, iSCSI , is an abbreviation of Internet Small Computer System Interface, an Internet Protocol -based storage networking standard for linking data storage facilities. By carrying SCSI commands over IP networks, iSCSI is used to facilitate data transfers over intranets and to manage...

, or Fibre Channel
Fibre Channel
Fibre Channel, or FC, is a gigabit-speed network technology primarily used for storage networking. Fibre Channel is standardized in the T11 Technical Committee of the InterNational Committee for Information Technology Standards , an American National Standards Institute –accredited standards...

. Some disk-based backup systems, such as Virtual Tape Libraries, support data deduplication
Data deduplication
In computing, data deduplication is a specialized data compression technique for eliminating coarse-grained redundant data. The technique is used to improve storage utilization and can also be applied to network data transfers to reduce the number of bytes that must be sent across a link...

 which can dramatically reduce the amount of disk storage capacity consumed by daily and weekly backup data. The main disadvantages of hard disk backups are that they are easily damaged, especially while being transported (e.g., for off-site backups), and that their stability over periods of years is a relative unknown.
Optical storage
Optical storage
Optical storage is a term from engineering referring to the storage of data on an optically readable medium. Data is recorded by making marks in a pattern that can be read back with the aid of light, usually a beam of laser light precisely focused on a spinning disc. An older example, that does...

 : Recordable CDs, DVD
DVD
A DVD is an optical disc storage media format, invented and developed by Philips, Sony, Toshiba, and Panasonic in 1995. DVDs offer higher storage capacity than Compact Discs while having the same dimensions....

s, and Blu-ray Disc
Blu-ray Disc
Blu-ray Disc is an optical disc storage medium designed to supersede the DVD format. The plastic disc is 120 mm in diameter and 1.2 mm thick, the same size as DVDs and CDs. Blu-ray Discs contain 25 GB per layer, with dual layer discs being the norm for feature-length video discs...

s are commonly used with personal computers and generally have low media unit costs. However, the capacities and speeds of these and other optical discs are typically an order of magnitude lower than hard disk or tape. Many optical disk formats are WORM
Write Once Read Many
A Write Once Read Many or WORM drive is a data storage device where information, once written, cannot be modified. On ordinary data storage devices, the number of times data can be modified is not limited, except by the rated lifespan of the device, as modification involves physical changes that...

 type, which makes them useful for archival purposes since the data cannot be changed. The use of an auto-changer or jukebox can make optical discs a feasible option for larger-scale backup systems. Some optical storage systems allow for cataloged data backups without human contact with the discs, allowing for longer data integrity.
Floppy disk
Floppy disk
A floppy disk is a disk storage medium composed of a disk of thin and flexible magnetic storage medium, sealed in a rectangular plastic carrier lined with fabric that removes dust particles...

 : During the 1980s and early 1990s, many personal/home computer users associated backing up mostly with copying to floppy disks. However the data capacity of floppy disks failed to catch up with growing demands, rendering them unpopular and obsolete.
Solid state storage : Also known as flash memory
Flash memory
Flash memory is a non-volatile computer storage chip that can be electrically erased and reprogrammed. It was developed from EEPROM and must be erased in fairly large blocks before these can be rewritten with new data...

, thumb drives, USB flash drive
USB flash drive
A flash drive is a data storage device that consists of flash memory with an integrated Universal Serial Bus interface. flash drives are typically removable and rewritable, and physically much smaller than a floppy disk. Most weigh less than 30 g...

s, CompactFlash
CompactFlash
CompactFlash is a mass storage device format used in portable electronic devices. Most CompactFlash devices contain flash memory in a standardized enclosure. The format was first specified and produced by SanDisk in 1994...

, SmartMedia
SmartMedia
SmartMedia is a flash memory card standard owned by Toshiba, with capacities ranging from 2 MB to 128 MB. SmartMedia memory cards are no longer manufactured.- History :...

, Memory Stick
Memory Stick
Memory Stick is a removable flash memory card format, launched by Sony in October 1998, and is also used in general to describe the whole family of Memory Sticks...

, Secure Digital card
Secure Digital card
Secure Digital is a non-volatile memory card format developed by the SD Card Association for use in portable devices. The SD technology is used by more than 400 brands across dozens of product categories and more than 8,000 models, and is considered the de-facto industry standard.Secure Digital...

s, etc., these devices are relatively expensive for their low capacity. A solid state drive does not contain any movable parts unlike its magnetic drive counterpart and can have huge throughput in the order of 500mbps to 1Gbps. SSD drives are now available in the order of 500GB to TBs.
Remote backup service
Remote backup service
A remote, online, or managed backup service is a service that provides users with a system for the backup and storage of computer files. Online backup providers are companies that provide this type of service to end users ....

 : As broadband internet access
Broadband Internet access
Broadband Internet access, often shortened to just "broadband", is a high data rate, low-latency connection to the Internet— typically contrasted with dial-up access using a 56 kbit/s modem or satellite Internet with inherently high latency....

 becomes more widespread, remote backup services are gaining in popularity. Backing up via the internet to a remote location can protect against some worst-case scenarios such as fires, floods, or earthquakes which would destroy any backups in the immediate vicinity along with everything else. There are, however, a number of drawbacks to remote backup services. First, Internet connections are usually slower than local data storage devices. Residential broadband is especially problematic as routine backups must use an upstream link that's usually much slower than the downstream link used only occasionally to retrieve a file from backup. This tends to limit the use of such services to relatively small amounts of high value data. Secondly, users must trust a third party service provider to maintain the privacy and integrity of their data, although confidentiality can be assured by encrypting the data before transmission to the backup service with an encryption key
Key (cryptography)
In cryptography, a key is a piece of information that determines the functional output of a cryptographic algorithm or cipher. Without a key, the algorithm would produce no useful result. In encryption, a key specifies the particular transformation of plaintext into ciphertext, or vice versa...

 known only to the user. Ultimately the backup service must itself use one of the above methods so this could be seen as a more complex way of doing traditional backups.

Managing the data repository


Regardless of the data repository model or data storage media used for backups, a balance needs to be struck between accessibility, security and cost. These media management methods are not mutually exclusive and are frequently combined to meet the needs of the situation. Using on-line disks for staging data before it is sent to a near-line tape library
Tape library
In computer storage, a tape library, sometimes called a tape silo, tape robot or tape jukebox, is a storage device which contains one or more tape drives, a number of slots to hold tape cartridges, a barcode reader to identify tape cartridges and an automated method for loading tapes...

 is a common example.

On-line
ONLINE
ONLINE is a magazine for information systems first published in 1977. The publisher Online, Inc. was founded the year before. In May 2002, Information Today, Inc. acquired the assets of Online Inc....

 : On-line backup storage is typically the most accessible type of data storage, which can begin restore in milliseconds time. A good example would be an internal hard disk or a disk array
Disk array
A disk array is a disk storage system which contains multiple disk drives. It is differentiated from a disk enclosure, in that an array has cache memory and advanced functionality, like RAID and virtualization.Components of a typical disk array include:...

 (maybe connected to SAN
Storage area network
A storage area network is a dedicated network that provides access to consolidated, block level data storage. SANs are primarily used to make storage devices, such as disk arrays, tape libraries, and optical jukeboxes, accessible to servers so that the devices appear like locally attached devices...

). This type of storage is very convenient and speedy, but is relatively expensive. On-line storage is quite vulnerable to being deleted or overwritten, either by accident, by intentional malevolent action, or in the wake of a data-deleting virus
Computer virus
A computer virus is a computer program that can replicate itself and spread from one computer to another. The term "virus" is also commonly but erroneously used to refer to other types of malware, including but not limited to adware and spyware programs that do not have the reproductive ability...

 payload.
Near-line
Nearline storage
Nearline storage is a term used in computer science to describe an intermediate type of data storage that represents a compromise between online storage and offline storage/archiving...

 : Near-line storage is typically less accessible and less expensive than on-line storage, but still useful for backup data storage. A good example would be a tape library
Tape library
In computer storage, a tape library, sometimes called a tape silo, tape robot or tape jukebox, is a storage device which contains one or more tape drives, a number of slots to hold tape cartridges, a barcode reader to identify tape cartridges and an automated method for loading tapes...

 with restore times ranging from seconds to a few minutes. A mechanical device is usually involved in moving media units from storage into a drive where the data can be read or written. Generally it has safety properties similar to on-line storage.
Off-line : Off-line storage requires some direct human action in order to make access to the storage media physically possible. This action is typically inserting a tape into a tape drive or plugging in a cable that allows a device to be accessed. Because the data is not accessible via any computer except during limited periods in which it is written or read back, it is largely immune to a whole class of on-line backup failure modes. Access time will vary depending on whether the media is on-site or off-site.

Off-site data protection
Off-site Data Protection
In computing, off-site data protection, or vaulting, is the strategy of sending critical data out of the main location as part of a disaster recovery plan. Data is usually transported off-site using removable storage media such as magnetic tape or optical storage...

: To protect against a disaster or other site-specific problem, many people choose to send backup media to an off-site vault. The vault can be as simple as a system administrator's home office or as sophisticated as a disaster-hardened, temperature-controlled, high-security bunker that has facilities for backup media storage. Importantly a data replica can be off-site but also on-line (e.g., an off-site RAID
RAID
RAID is a storage technology that combines multiple disk drive components into a logical unit...

 mirror). Such a replica has fairly limited value as a backup, and should not be confused with an off-line backup.
Backup site or disaster recovery center (DR center): In the event of a disaster, the data on backup media will not be sufficient to recover. Computer systems onto which the data can be restored and properly configured networks are necessary too. Some organizations have their own data recovery centers that are equipped for this scenario. Other organizations contract this out to a third-party recovery center. Because a DR site is itself a huge investment, backing up is very rarely considered the preferred method of moving data to a DR site. A more typical way would be remote disk mirroring
Disk mirroring
In data storage, disk mirroring or RAID1 is the replication of logical disk volumes onto separate physical hard disks in real time to ensure continuous availability...

, which keeps the DR data as up to date as possible.

Selection and extraction of data


A successful backup job starts with selecting and extracting coherent units of data. Most data on modern computer systems is stored in discrete units, known as file
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...

s. These files are organized into filesystems. Files that are actively being updated can be thought of as "live" and present a challenge to back up. It is also useful to save metadata that describes the computer or the filesystem being backed up.

Deciding what to back up at any given time is a harder process than it seems. By backing up too much redundant data, the data repository will fill up too quickly. Backing up an insufficient amount of data can eventually lead to the loss of critical information.

Files


Copying files
File copying
In the realm of computer file management, file copying is the creation of a new file which has the same content as an existing file.All computer operating systems include file copying provisions in the user interface, like the command, "cp" in Unix and "copy" in MS-DOS; operating systems with a...

 : Making copies of files is the simplest and most common way to perform a backup. A means to perform this basic function is included in all backup software and all operating systems.

Partial file copying : Instead of copying whole files, one can limit the backup to only the blocks or bytes within a file that have changed in a given period of time. This technique can use substantially less storage space on the backup medium, but requires a high level of sophistication to reconstruct files in a restore situation. Some implementations require integration with the source filesystem.

When backing up over a network, the rsync
Rsync
rsync is a software application and network protocol for Unix-like and Windows systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar...

 utility automatically transmits a minimum set of changes to bring an earlier version of a file at the destination up to date with the current version at the source. Rsync can dramatically reduce the network traffic needed to maintain a remote mirror of a large set of files undergoing small, frequent changes.

Filesystems


Filesystem dump : Instead of copying files within a filesystem, a copy of the whole filesystem itself can be made. This is also known as a raw partition backup and is related to disk imaging
Disk image
A disk image is a single file or storage device containing the complete contents and structure representing a data storage medium or device, such as a hard drive, tape drive, floppy disk, CD/DVD/BD, or USB flash drive, although an image of an optical disc may be referred to as an optical disc image...

. The process usually involves unmounting the filesystem and running a program like dd (Unix)
Dd (Unix)
In computing, dd is a common Unix program whose primary purpose is the low-level copying and conversion of raw data. According to the manual page for Version 7 Unix, it will "convert and copy a file". It is used to copy a specified number of bytes or blocks, performing on-the-fly byte order...

. Because the disk is read sequentially and with large buffers, this type of backup can be much faster than reading every file normally, especially when the filesystem contains many small files, is highly fragmented, or is nearly full. But because this method also reads the free disk blocks that contain no useful data, this method can also be slower than conventional reading, especially when the filesystem is nearly empty. Some filesystems, such as XFS
XFS
XFS is a high-performance journaling file system created by Silicon Graphics, Inc. It is the default file system in IRIX releases 5.3 and onwards and later ported to the Linux kernel. XFS is particularly proficient at parallel IO due to its allocation group based design...

, provide a "dump" utility that reads the disk sequentially for high performance while skipping unused sections. The corresponding restore utility can selectively restore individual files or the entire volume at the operator's choice.

Identification of changes : Some filesystems have an archive bit for each file that says it was recently changed. Some backup software looks at the date of the file and compares it with the last backup to determine whether the file was changed.

Versioning file system
Versioning file system
A versioning file system is any computer file system which allows a computer file to exist in several versions at the same time. Thus it is a form of revision control. Most common versioning file systems keep a number of old copies of the file. Some limit the number of changes per minute or per...

 : A versioning filesystem keeps track of all changes to a file and makes those changes accessible to the user. Generally this gives access to any previous version, all the way back to the file's creation time. An example of this is the Wayback versioning filesystem for Linux.

Live data


If a computer system is in use while it is being backed up, the possibility of files being open for reading or writing is real. If a file is open, the contents on disk may not correctly represent what the owner of the file intends. This is especially true for database files of all kinds. The term fuzzy backup
Fuzzy backup
A fuzzy backup is a secondary copy of data file or directories that were in one state when the backup started, but in a different state by the time the backup completed. This may result in the backup copy being unusable because of the data inconsistencies...

 can be used to describe a backup of live data that looks like it ran correctly, but does not represent the state of the data at any single point in time. This is because the data being backed up changed in the period of time between when the backup started and when it finished. For databases in particular, fuzzy backups are worthless.

Snapshot
Snapshot (computer storage)
In computer systems, a snapshot is the state of a system at a particular point in time. The term was coined as an analogy to that in photography. It can refer to an actual copy of the state of a system or to a capability provided by certain systems....

 backup : A snapshot is an instantaneous function of some storage systems that presents a copy of the file system as if it were frozen at a specific point in time, often by a copy-on-write
Copy-on-write
Copy-on-write is an optimization strategy used in computer programming. The fundamental idea is that if multiple callers ask for resources which are initially indistinguishable, they can all be given pointers to the same resource...

 mechanism. An effective way to back up live data is to temporarily quiesce
Quiesce
Quiesce is used to describe pausing or altering the state of running processes on a computer, particularly those that might modify information stored on disk during a backup, in order to guarantee a consistent and usable backup...

 it (e.g. close all files), take a snapshot, and then resume live operations. At this point the snapshot can be backed up through normal methods. While a snapshot is very handy for viewing a filesystem as it was at a different point in time, it is hardly an effective backup mechanism by itself.

Open file backup : Many backup software packages feature the ability to handle open files in backup operations. Some simply check for openness and try again later. File locking
File locking
File locking is a mechanism that restricts access to a computer file by allowing only one user or process access at any specific time. Systems implement locking to prevent the classic interceding update scenario ....

 is useful for regulating access to open files.
When attempting to understand the logistics of backing up open files, one must consider that the backup process could take several minutes to back up a large file such as a database. In order to back up a file that is in use, it is vital that the entire backup represent a single-moment snapshot of the file, rather than a simple copy of a read-through. This represents a challenge when backing up a file that is constantly changing. Either the database file must be locked to prevent changes, or a method must be implemented to ensure that the original snapshot is preserved long enough to be copied, all while changes are being preserved. Backing up a file while it is being changed, in a manner that causes the first part of the backup to represent data before changes occur to be combined with later parts of the backup after the change results in a corrupted file that is unusable, as most large files contain internal references between their various parts that must remain consistent throughout the file.


Cold database backup : During a cold backup, the database is closed or locked and not available to users. The datafiles do not change during the backup process so the database is in a consistent state when it is returned to normal operation.

Hot database backup : Some database management systems offer a means to generate a backup image of the database while it is online and usable ("hot"). This usually includes an inconsistent image of the data files plus a log of changes made while the procedure is running. Upon a restore, the changes in the log files are reapplied to bring the database in sync.

Metadata


Not all information stored on the computer is stored in files. Accurately recovering a complete system from scratch requires keeping track of this non-file data too.
System description : System specifications are needed to procure an exact replacement after a disaster.
Boot sector
Boot sector
A boot sector or boot block is a region of a hard disk, floppy disk, optical disc, or other data storage device that contains machine code to be loaded into random-access memory by a computer system's built-in firmware...

 : The boot sector can sometimes be recreated more easily than saving it. Still, it usually isn't a normal file and the system won't boot without it.
Partition
Disk partitioning
Disk partitioning is the act of dividing a hard disk drive into multiple logical storage units referred to as partitions, to treat one physical disk drive as if it were multiple disks. Partitions are also termed "slices" for operating systems based on BSD, Solaris or GNU Hurd...

 layout : The layout of the original disk, as well as partition tables and filesystem settings, is needed to properly recreate the original system.
File metadata
Metadata
The term metadata is an ambiguous term which is used for two fundamentally different concepts . Although the expression "data about data" is often used, it does not apply to both in the same way. Structural metadata, the design and specification of data structures, cannot be about data, because at...

 : Each file's permissions, owner, group, ACLs, and any other metadata need to be backed up for a restore to properly recreate the original environment.
System metadata : Different operating systems have different ways of storing configuration information. Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...

 keeps a registry
Windows registry
The Windows Registry is a hierarchical database that stores configuration settings and options on Microsoft Windows operating systems. It contains settings for low-level operating system components as well as the applications running on the platform: the kernel, device drivers, services, SAM, user...

 of system information that is more difficult to restore than a typical file.

Manipulation of data and dataset optimization


It is frequently useful or required to manipulate the data being backed up to optimize the backup process. These manipulations can provide many benefits including improved backup speed, restore speed, data security, media usage and/or reduced bandwidth requirements.
Compression
Data compression
In computer science and information theory, data compression, source coding or bit-rate reduction is the process of encoding information using fewer bits than the original representation would use....

 : Various schemes can be employed to shrink the size of the source data to be stored so that it uses less storage space. Compression is frequently a built-in feature of tape drive hardware.
Deduplication
Data deduplication
In computing, data deduplication is a specialized data compression technique for eliminating coarse-grained redundant data. The technique is used to improve storage utilization and can also be applied to network data transfers to reduce the number of bytes that must be sent across a link...

 : When multiple similar systems are backed up to the same destination storage device, there exists the potential for much redundancy within the backed up data. For example, if 20 Windows workstations were backed up to the same data repository, they might share a common set of system files. The data repository only needs to store one copy of those files to be able to restore any one of those workstations. This technique can be applied at the file level or even on raw blocks of data, potentially resulting in a massive reduction in required storage space. Deduplication can occur on a server before any data moves to backup media, sometimes referred to as source/client side deduplication. This approach also reduces bandwidth required to send backup data to its target media. The process can also occur at the target storage device, sometimes referred to as inline or back-end deduplication.
Duplication
Replication (computer science)
Replication is the process of sharing information so as to ensure consistency between redundant resources, such as software or hardware components, to improve reliability, fault-tolerance, or accessibility. It could be data replication if the same data is stored on multiple storage devices, or...

 : Sometimes backup jobs are duplicated to a second set of storage media. This can be done to rearrange the backup images to optimize restore speed or to have a second copy at a different location or on a different storage medium.
Encryption
Encryption
In cryptography, encryption is the process of transforming information using an algorithm to make it unreadable to anyone except those possessing special knowledge, usually referred to as a key. The result of the process is encrypted information...

 : High capacity removable storage media such as backup tapes present a data security risk if they are lost or stolen. Encrypting the data on these media can mitigate this problem, but presents new problems. Encryption is a CPU intensive process that can slow down backup speeds, and the security of the encrypted backups is only as effective as the security of the key management policy.
Multiplexing
Multiplexing
The multiplexed signal is transmitted over a communication channel, which may be a physical transmission medium. The multiplexing divides the capacity of the low-level communication channel into several higher-level logical channels, one for each message signal or data stream to be transferred...

 : When there are many more computers to be backed up than there are destination storage devices, the ability to use a single storage device with several simultaneous backups can be useful.
Refactoring : The process of rearranging the backup sets in a data repository is known as refactoring. For example, if a backup system uses a single tape each day to store the incremental backups for all the protected computers, restoring one of the computers could potentially require many tapes. Refactoring could be used to consolidate all the backups for a single computer onto a single tape. This is especially useful for backup systems that do incrementals forever style backups.
Staging
Disk staging
Disk staging is using disks as an additional, temporary stage of backup process before finally storing backup to tape. Backups stay on disk typically for a day or a week, before being copied to tape in a background process and deleted afterwards....

 : Sometimes backup jobs are copied to a staging disk before being copied to tape. This process is sometimes referred to as D2D2T, an acronym for Disk to Disk to Tape. This can be useful if there is a problem matching the speed of the final destination device with the source device as is frequently faced in network-based backup systems. It can also serve as a centralized location for applying other data manipulation techniques.

Managing the backup process


It is important to understand that backing up is a process. As long as new data is being created and changes are being made, backups will need to be updated. Individuals and organizations with anything from one computer to thousands (or even millions) of computer systems all have requirements for protecting data. While the scale is different, the objectives and limitations are essentially the same. Likewise, those who perform backups need to know to what extent they were successful, regardless of scale.

Objectives


Recovery point objective
Recovery point objective
-Recovery point objective :When computers used for normal "production" business services are affected by a "Major Incident" that cannot be fixed quickly, then the Information Technology Service Continuity Plan is performed, by the ITSC recovery team...

 (RPO) : The point in time that the restarted infrastructure will reflect. Essentially, this is the roll-back that will be experienced as a result of the recovery. The most desirable RPO would be the point just prior to the data loss event. Making a more recent recovery point achievable requires increasing the frequency of synchronization
File synchronization
File synchronization in computing is the process of ensuring that computer files in two or more locations are updated via certain rules....

 between the source data and the backup repository.
Recovery time objective
Recovery Time Objective
The recovery time objective is the duration of time and a service level within which a business process must be restored after a disaster in order to avoid unacceptable consequences associated with a break in business continuity....

 (RTO) : The amount of time elapsed between disaster and restoration of business functions.
Data security
Data security
Data security is the means of ensuring that data is kept safe from corruption and that access to it is suitably controlled. Thus data security helps to ensure privacy. It also helps in protecting personal data. Data security is part of the larger practice of Information security.- Disk Encryption...

 : In addition to preserving access to data for its owners, data must be restricted from unauthorized access. Backups must be performed in a manner that does not compromise the original owner's undertaking. This can be achieved with data encryption and proper media handling policies.

Limitations


An effective backup scheme will take into consideration the limitations of the situation.
Backup window : The period of time when backups are permitted to run on a system is called the backup window. This is typically the time when the system sees the least usage and the backup process will have the least amount of interference with normal operations. The backup window is usually planned with users' convenience in mind. If a backup extends past the defined backup window, a decision is made whether it is more beneficial to abort the backup or to lengthen the backup window.
Performance impact : All backup schemes have some performance impact on the system being backed up. For example, for the period of time that a computer system is being backed up, the hard drive is busy reading files for the purpose of backing up, and its full bandwidth is no longer available for other tasks. Such impacts should be analyzed.
Costs of hardware, software, labor : All types of storage media have a finite capacity with a real cost. Matching the correct amount of storage capacity (over time) with the backup needs is an important part of the design of a backup scheme. Any backup scheme has some labor requirement, but complicated schemes have considerably higher labor requirements. The cost of commercial backup software can also be considerable.
Network bandwidth : Distributed backup systems can be affected by limited network bandwidth.

Implementation


Meeting the defined objectives in the face of the above limitations can be a difficult task. The tools and concepts below can make that task more achievable.
Scheduling : Using a job scheduler
Job scheduler
A job scheduler is a software application that is in charge of unattended background executions, commonly known for historical reasons as batch processing....

 can greatly improve the reliability and consistency of backups by removing part of the human element. Many backup software packages include this functionality.
Authentication : Over the course of regular operations, the user accounts and/or system agents that perform the backups need to be authenticated at some level. The power to copy all data off of or onto a system requires unrestricted access. Using an authentication mechanism is a good way to prevent the backup scheme from being used for unauthorized activity.
Chain of trust
Chain of trust
In computer security, a chain of trust is established by validating each component of hardware and software from the bottom up. It is intended to ensure that only trusted software and hardware can be used while still remaining flexible.-Introduction:...

 : Removable storage media are physical items and must only be handled by trusted individuals. Establishing a chain of trusted individuals (and vendors) is critical to defining the security of the data.

Measuring the process


To ensure that the backup scheme is working as expected, the process needs to include monitoring key factors and maintaining historical data.

Backup validation
Backup validation
Backup validation is the process whereby owners of computer data may examine how their data was backed up in order to understand what their risk of data loss might be...

 : (also known as "backup success validation") The process by which owners of data can get information about how their data was backed up. This same process is also used to prove compliance to regulatory bodies outside of the organization, for example, an insurance company might be required under HIPAA to show "proof" that their patient data are meeting records retention requirements. Disaster, data complexity, data value and increasing dependence upon ever-growing volumes of data all contribute to the anxiety around and dependence upon successful backups to ensure business continuity
Business continuity
Business continuity is the activity performed by an organization to ensure that critical business functions will be available to customers, suppliers, regulators, and other entities that must have access to those functions. These activities include many daily chores such as project management,...

. For that reason, many organizations rely on third-party or "independent" solutions to test, validate, and optimize their backup operations (backup reporting).
Reporting : In larger configurations, reports are useful for monitoring media usage, device status, errors, vault coordination and other information about the backup process.
Logging : In addition to the history of computer generated reports, activity and change logs are useful for monitoring backup system events.
Validation : Many backup programs make use of checksum
Checksum
A checksum or hash sum is a fixed-size datum computed from an arbitrary block of digital data for the purpose of detecting accidental errors that may have been introduced during its transmission or storage. The integrity of the data can be checked at any later time by recomputing the checksum and...

s or hash
Hash function
A hash function is any algorithm or subroutine that maps large data sets to smaller data sets, called keys. For example, a single integer can serve as an index to an array...

es to validate that the data was accurately copied. These offer several advantages. First, they allow data integrity to be verified without reference to the original file: if the file as stored on the backup medium has the same checksum as the saved value, then it is very probably correct. Second, some backup programs can use checksums to avoid making redundant copies of files, to improve backup speed. This is particularly useful for the de-duplication process.
Monitored backup : Backup processes are monitored by a third party monitoring center. This center alerts users to any errors that occur during automated backups. Monitored backup requires software capable of pinging the monitoring center's servers in the case of errors. Some monitoring services also allow collection of historical meta-data, that can be used for Storage Resource Management purposes like projection of data growth, locating redundant primary storage capacity and reclaimable backup capacity. The Wizards Storage Portal is an example of a solution that monitors IBM's well known Tivoli Storage Manager(TSM) solution.

Confusion


Because of a considerable overlap in technology, backups and backup systems are frequently confused with archive
Archive
An archive is a collection of historical records, or the physical place they are located. Archives contain primary source documents that have accumulated over the course of an individual or organization's lifetime, and are kept to show the function of an organization...

s and fault-tolerant system
Fault-tolerant system
Fault-tolerance or graceful degradation is the property that enables a system to continue operating properly in the event of the failure of some of its components. A newer approach is progressive enhancement...

s. Backups differ from archives in the sense that archives are the primary copy of data, usually put away for future use, while backups are a secondary copy of data, kept on hand to replace the original item. Backup systems differ from fault-tolerant systems in the sense that backup systems assume that a fault will cause a data loss event and fault-tolerant systems assure a fault will not.

Advice


  • The more important the data that is stored on the computer, the greater is the need for backing up this data.
  • A backup is only as useful as its associated restore strategy. For critical systems and data, the restoration process must be tested.
  • Storing the copy near the original is unwise, since many disasters such as fire, flood, theft, and electrical surges are likely to cause damage to the backup at the same time. In these cases, both the original and the backup medium are likely to be lost.
  • Automated backup and scheduling should be considered, as manual backups can be affected by human error.
  • Backups can fail for a wide variety of reasons. A verification or monitoring strategy is an important part of a successful backup plan.
  • Multiple backups on different media, stored in different locations, should be used for all critical information.
  • Backed up archives should be stored in open and standard formats, especially when the goal is long-term archiving. Recovery software and processes may have changed, and software may not be available to restore data saved in proprietary formats.
  • System administrators and others working in the information technology field are routinely fired for not devising and maintaining backup processes suitable to their organization.
  • If you already have a tape backup system, a second backup program may be necessary. Perform an additional backup to an external hard disk with an automatic backup program so you will have doubled the data security, and it is easy to check the backed-up files in the external hard disk.

Events

  • In 1996, during a fire at the headquarters of Crédit Lyonnais
    Crédit Lyonnais
    Crédit Lyonnais is a historic French bank. In the early 1990s it was the largest French bank, majority state-owned at that point. Crédit Lyonnais was the subject of poor management during that period which almost led to its bankruptcy in 1993...

    , a major bank in Paris, system administrators ran into the burning building to rescue backup tapes because they didn't have off-site copies. Crucial bank archives and computer data were lost.
  • Privacy Rights Clearinghouse
    Privacy Rights Clearinghouse
    Privacy Rights Clearinghouse is a project of the , an American 501 non-profit consumer advocacy organization. The Privacy Rights Clearinghouse is devoted to upholding the right to privacy and protecting consumers against identity theft and other privacy crimes.It was established in 1992 by Beth...

     has documented 16 instances of stolen or lost backup tapes (among major organizations) in 2005 & 2006. Affected organizations included Bank of America
    Bank of America
    Bank of America Corporation, an American multinational banking and financial services corporation, is the second largest bank holding company in the United States by assets, and the fourth largest bank in the U.S. by market capitalization. The bank is headquartered in Charlotte, North Carolina...

    , Ameritrade, Citigroup
    Citigroup
    Citigroup Inc. or Citi is an American multinational financial services corporation headquartered in Manhattan, New York City, New York, United States. Citigroup was formed from one of the world's largest mergers in history by combining the banking giant Citicorp and financial conglomerate...

    , and Time Warner
    Time Warner
    Time Warner is one of the world's largest media companies, headquartered in the Time Warner Center in New York City. Formerly two separate companies, Warner Communications, Inc...

    .
  • On 3 January 2008, an email server crashed at TeliaSonera
    TeliaSonera
    TeliaSonera AB is the dominant telephone company and mobile network operator in Sweden and Finland. The company has operations in other countries in Northern, Eastern Europe, Central Asia and Spain, with a total of 150 million mobile customers...

    , a major Nordic telecom company and internet service provider
    Internet service provider
    An Internet service provider is a company that provides access to the Internet. Access ISPs directly connect customers to the Internet using copper wires, wireless or fiber-optic connections. Hosting ISPs lease server space for smaller businesses and host other people servers...

    . It was subsequently discovered that the last serviceable backup set was from 15 December 2007. Three hundred thousand customer email accounts were affected.
  • On 27 February 2011 a software bug on Gmail
    Gmail
    Gmail is a free, advertising-supported email service provided by Google. Users may access Gmail as secure webmail, as well via POP3 or IMAP protocols. Gmail was launched as an invitation-only beta release on April 1, 2004 and it became available to the general public on February 7, 2007, though...

     caused 0.02% of its users to lose all their emails. The messages were successfully restored from tape backups hours after the event.

See also


About backup
  • Backup software
    Backup software
    Backup software are computer programs used to perform backup; they create supplementary exact copies of files, databases or entire computers. These programs may later use the supplementary copies to restore the original contents in the event of data loss....

  • Glossary of backup terms
    Glossary of backup terms
    The subject of computer backups is rife with jargon and highly specialized terminology. This page is a glossary of backup terms that aims to clarify the meaning of such jargon and terminology.- Terms and definitions :...

  • Remote backup service
    Remote backup service
    A remote, online, or managed backup service is a service that provides users with a system for the backup and storage of computer files. Online backup providers are companies that provide this type of service to end users ....

  • Virtual backup appliance
    Virtual Backup Appliance
    -Definition:A Virtual Backup Appliance is a small virtual machine that backs up and restores other virtual machines.Unlike some backup solutions that require a dedicated host to act as a proxy server for the backups, virtual backup appliances use the virtual infrastructure platform itself for the...



Related topics
  • Data proliferation
    Data proliferation
    Data proliferation refers to the prodigious amount of data, structured and unstructured, that businesses and governments continue to generate at an unprecedented rate and the usability problems that result from attempting to store and manage that data...

  • Database dump
    Database dump
    A database dump contains a record of the table structure and/or the data from a database and is usually in the form of a list of SQL statements. A database dump is most often used for backing up a database so that its contents can be restored in the event of data loss. Corrupted databases can often...

  • Disaster recovery and business continuity auditing
    Disaster recovery and business continuity auditing
    Disaster recovery and business continuity refers to an organization’s ability to recover from a disaster and/or unexpected event and resume or continue operations. Organizations should have a plan in place that outlines how this will be accomplished...

  • Digital preservation
    Digital preservation
    Digital preservation is the set of processes, activities and management of digital information over time to ensure its long term accessibility. The goal of digital preservation is to preserve materials resulting from digital reformatting, and particularly information that is born-digital with no...

  • File synchronization
    File synchronization
    File synchronization in computing is the process of ensuring that computer files in two or more locations are updated via certain rules....

  • Information repository
    Information repository
    An information repository is an easy way to deploy a secondary tier of data storage that can comprise multiple, networked data storage technologies running on diverse operating systems, where data that no longer needs to be in primary storage is protected, classified according to captured metadata,...