Digital Dark Age
Encyclopedia
The digital dark age is a possible future situation where it will be difficult or impossible to read historical digital documents and multimedia, because they have been stored in an obsolete and obscure digital format. The name derives from the term "Dark Ages" in the sense that there would be a relative lack of written record.
(IFLA) in 1997.
The term was also mentioned in 1998 at the Time and Bits conference,
which was co-sponsored by the Long Now Foundation
and the Getty Conservation Institute
.
The problem is not limited to text documents, but applies equally to photos, video, audio and other kinds of electronic document
s. The concern leading to the use of the term is that documents are stored on physical media
which require special hardware
in order to be read and that this hardware will not be available in a few decades from the time the document was created. For example, it is already the case that disk drives capable of reading 5¼-inch floppy disk
s are not readily available.
The Digital Dark Age also applies to the problems which arise due to obsolete file format
s. In this case it is the lack of the necessary software which causes problems when retrieving stored documents. This is especially problematic when proprietary formats are used, in which case it might be impossible to write appropriate software to read the file.
A famous real example is with NASA
, whose early space records were suffering from a Dark Age issue: for over a decade, magnetic tapes from the 1976 Viking
Mars landing
were unprocessed. When later analyzed, the data was unreadable as it was in an unknown format and the original programmers had either died or left NASA. The images were eventually extracted following many months of puzzling through the data and examining how the recording machines functioned.
Another example is the BBC Domesday Project
in which a survey of the nation was compiled 900 years after the Domesday Book
was published. While the information in the Domesday Book is still accessible today, there were great fears that the discs of the Domesday Project would become unreadable as computers capable of reading the format had become rare and drives capable of accessing the discs even rarer. However the system was emulated in 2002 using a system called DomesEm by the CAMiLEON
project. This allows the information on the discs to be accessed on modern computers.
Encrypted
data may also prove to be an issue, as the process needed to decode the data is intentionally made as obscure as possible. Historically encrypted data is quite rare but even the very simple means available throughout history have provided many examples of documents that can only be read with great effort. For example, it took the capacity of a distributed computing project to break the mechanically generated code of a single brief World War II submarine tactical message. Modern encryption is being used in many more documents and media due to publishers wanting the promised protections of DRM
. This very widespread use of encryption closes down several of the routes (e.g.: Forgotten in the attic) by which the last few copies of documents and media that are later deemed to be historically significant can be recovered.
created a partnership with The National Archives
of the United States of America to prevent the digital dark age and "unlock millions of unreadable stored computer files". This involves moving files from their old proprietary formats to their open format
Open XML.
The Internet Archive
has stated that one of their goals is to prevent the digital dark age.
One approach is open source
, where the source code
for reading and writing a file format is open. In 2007 the chief information officer of the UK's National Archives stated "We welcome open-source software because it makes our lives easier".
About
An early mention of the term was at a conference of the International Federation of Library Associations and InstitutionsInternational Federation of Library Associations and Institutions
The International Federation of Library Associations and Institutions is the leading international association of library organisations. It is the global voice of the library and information profession, and its annual conference provides a venue for librarians to learn from one another...
(IFLA) in 1997.
The term was also mentioned in 1998 at the Time and Bits conference,
which was co-sponsored by the Long Now Foundation
Long Now Foundation
The Long Now Foundation, established in 1996, is a private organization that seeks to become the seed of a very long-term cultural institution. It aims to provide a counterpoint to what it views as today's "faster/cheaper" mindset and to promote "slower/better" thinking...
and the Getty Conservation Institute
Getty Conservation Institute
The Getty Conservation Institute , located in Los Angeles, California, is a program of the J. Paul Getty Trust. It is headquartered at the Getty Center but also has facilities at the Getty Villa, and commenced operation in 1985. The GCI is a private international research institution dedicated to...
.
The problem is not limited to text documents, but applies equally to photos, video, audio and other kinds of electronic document
Electronic document
An electronic document is any electronic media content that are intended to be used in either an electronic form or as printed output....
s. The concern leading to the use of the term is that documents are stored on physical media
Digital media
Digital media is a form of electronic media where data is stored in digital form. It can refer to the technical aspect of storage and transmission Digital media is a form of electronic media where data is stored in digital (as opposed to analog) form. It can refer to the technical aspect of...
which require special hardware
Computer hardware
Personal computer hardware are component devices which are typically installed into or peripheral to a computer case to create a personal computer upon which system software is installed including a firmware interface such as a BIOS and an operating system which supports application software that...
in order to be read and that this hardware will not be available in a few decades from the time the document was created. For example, it is already the case that disk drives capable of reading 5¼-inch floppy disk
Floppy disk
A floppy disk is a disk storage medium composed of a disk of thin and flexible magnetic storage medium, sealed in a rectangular plastic carrier lined with fabric that removes dust particles...
s are not readily available.
The Digital Dark Age also applies to the problems which arise due to obsolete file format
File format
A file format is a particular way that information is encoded for storage in a computer file.Since a disk drive, or indeed any computer storage, can store only bits, the computer must have some way of converting information to 0s and 1s and vice-versa. There are different kinds of formats for...
s. In this case it is the lack of the necessary software which causes problems when retrieving stored documents. This is especially problematic when proprietary formats are used, in which case it might be impossible to write appropriate software to read the file.
A famous real example is with NASA
NASA
The National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research...
, whose early space records were suffering from a Dark Age issue: for over a decade, magnetic tapes from the 1976 Viking
Viking program
The Viking program consisted of a pair of American space probes sent to Mars, Viking 1 and Viking 2. Each spacecraft was composed of two main parts, an orbiter designed to photograph the surface of Mars from orbit, and a lander designed to study the planet from the surface...
Mars landing
Mars landing
A Mars landing is a landing of a spacecraft on the surface of Mars. Of multiple attempted Mars landings by robotic, unmanned spacecraft, six were successful. There have also been studies for a possible manned mission to Mars, including a landing, but none have been attempted.-Mars probe program:In...
were unprocessed. When later analyzed, the data was unreadable as it was in an unknown format and the original programmers had either died or left NASA. The images were eventually extracted following many months of puzzling through the data and examining how the recording machines functioned.
Another example is the BBC Domesday Project
BBC Domesday Project
The BBC Domesday Project was a partnership between Acorn Computers Ltd, Philips, Logica and the BBC to mark the 900th anniversary of the original Domesday Book, an 11th century census of England...
in which a survey of the nation was compiled 900 years after the Domesday Book
Domesday Book
Domesday Book , now held at The National Archives, Kew, Richmond upon Thames in South West London, is the record of the great survey of much of England and parts of Wales completed in 1086...
was published. While the information in the Domesday Book is still accessible today, there were great fears that the discs of the Domesday Project would become unreadable as computers capable of reading the format had become rare and drives capable of accessing the discs even rarer. However the system was emulated in 2002 using a system called DomesEm by the CAMiLEON
CAMiLEON
The CAMiLEON project was a joint undertaking by the University of Michigan and the University of Leeds to develop and evaluate a range of technical strategies for the long term preservation of digital material suffering from digital obsolescence...
project. This allows the information on the discs to be accessed on modern computers.
Encrypted
Encryption
In cryptography, encryption is the process of transforming information using an algorithm to make it unreadable to anyone except those possessing special knowledge, usually referred to as a key. The result of the process is encrypted information...
data may also prove to be an issue, as the process needed to decode the data is intentionally made as obscure as possible. Historically encrypted data is quite rare but even the very simple means available throughout history have provided many examples of documents that can only be read with great effort. For example, it took the capacity of a distributed computing project to break the mechanically generated code of a single brief World War II submarine tactical message. Modern encryption is being used in many more documents and media due to publishers wanting the promised protections of DRM
Digital rights management
Digital rights management is a class of access control technologies that are used by hardware manufacturers, publishers, copyright holders and individuals with the intent to limit the use of digital content and devices after sale. DRM is any technology that inhibits uses of digital content that...
. This very widespread use of encryption closes down several of the routes (e.g.: Forgotten in the attic) by which the last few copies of documents and media that are later deemed to be historically significant can be recovered.
Prevention
In 2007, MicrosoftMicrosoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
created a partnership with The National Archives
National Archives and Records Administration
The National Archives and Records Administration is an independent agency of the United States government charged with preserving and documenting government and historical records and with increasing public access to those documents, which comprise the National Archives...
of the United States of America to prevent the digital dark age and "unlock millions of unreadable stored computer files". This involves moving files from their old proprietary formats to their open format
Open format
An open file format is a published specification for storing digital data, usually maintained by a standards organization, which can therefore be used and implemented by anyone. For example, an open format can be implementable by both proprietary and free and open source software, using the typical...
Open XML.
The Internet Archive
Internet Archive
The Internet Archive is a non-profit digital library with the stated mission of "universal access to all knowledge". It offers permanent storage and access to collections of digitized materials, including websites, music, moving images, and nearly 3 million public domain books. The Internet Archive...
has stated that one of their goals is to prevent the digital dark age.
One approach is open source
Open source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
, where the source code
Source code
In computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...
for reading and writing a file format is open. In 2007 the chief information officer of the UK's National Archives stated "We welcome open-source software because it makes our lives easier".