File verification
Encyclopedia
File verification is the process of using an algorithm
for verifying the integrity or authenticity
of a computer file
. This can be done by comparing two files bit-by-bit, but requires two copies of the same file, and may miss systematic corruptions which might occur to both files. A more popular approach is to also store checksum
s (hashes) of files for later comparison.
. A file can become corrupted by a variety of ways: faulty storage media, errors in transmission, write errors during copying or moving, software bug
s, and so on.
Hash
-based verification ensures that a file has not been corrupted by comparing the file's hash value to a previously calculated value. If these values match, the file is presumed to be unmodified. Due to the nature of hash functions, hash collision
s may result in false positives, but the likelihood of collisions is often negligible with random corruption.
es or backdoors. To verify the authenticity, a classical hash function is not enough as they are not designed to be collision resistant
; it is computationally trivial for an attacker to cause deliberate hash collisions, meaning that a malicious change in the file is not detected with by a hash comparison. In cryptography, this attack is called the collision attack
.
For this purpose, cryptographic hash function
s are employed often. As long as the hash sums cannot be tampered with — for example, if they are communicated over a secure channel — the files can be presumed to be intact. Alternatively, digital signature
s can be employed to assure tamper-resistance.
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
for verifying the integrity or authenticity
Authentication
Authentication is the act of confirming the truth of an attribute of a datum or entity...
of a computer file
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
. This can be done by comparing two files bit-by-bit, but requires two copies of the same file, and may miss systematic corruptions which might occur to both files. A more popular approach is to also store checksum
Checksum
A checksum or hash sum is a fixed-size datum computed from an arbitrary block of digital data for the purpose of detecting accidental errors that may have been introduced during its transmission or storage. The integrity of the data can be checked at any later time by recomputing the checksum and...
s (hashes) of files for later comparison.
Integrity verification
File integrity can be compromised, usually referred to as the file becoming corruptedData corruption
Data corruption refers to errors in computer data that occur during writing, reading, storage, transmission, or processing, which introduce unintended changes to the original data...
. A file can become corrupted by a variety of ways: faulty storage media, errors in transmission, write errors during copying or moving, software bug
Software bug
A software bug is the common term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. Most bugs arise from mistakes and errors made by people in either a program's...
s, and so on.
Hash
Hash function
A hash function is any algorithm or subroutine that maps large data sets to smaller data sets, called keys. For example, a single integer can serve as an index to an array...
-based verification ensures that a file has not been corrupted by comparing the file's hash value to a previously calculated value. If these values match, the file is presumed to be unmodified. Due to the nature of hash functions, hash collision
Hash collision
Not to be confused with wireless packet collision.In computer science, a collision or clash is a situation that occurs when two distinct pieces of data have the same hash value, checksum, fingerprint, or cryptographic digest....
s may result in false positives, but the likelihood of collisions is often negligible with random corruption.
Authenticity verification
It is often desirable to verify that a file hasn't been modified in transmission or storage by untrusted parties, for example, to include malicious code such as virusVirus
A virus is a small infectious agent that can replicate only inside the living cells of organisms. Viruses infect all types of organisms, from animals and plants to bacteria and archaea...
es or backdoors. To verify the authenticity, a classical hash function is not enough as they are not designed to be collision resistant
Collision resistance
Collision resistance is a property of cryptographic hash functions: a hash function is collision resistant if it is hard to find two inputs that hash to the same output; that is, two inputs a and b such that H = H, and a ≠ b.Every hash function with more inputs than outputs will necessarily have...
; it is computationally trivial for an attacker to cause deliberate hash collisions, meaning that a malicious change in the file is not detected with by a hash comparison. In cryptography, this attack is called the collision attack
Collision attack
In cryptography, a collision attack on a cryptographic hash tries to find two arbitrary inputs that will produce the same hash value, i.e. a hash collision...
.
For this purpose, cryptographic hash function
Cryptographic hash function
A cryptographic hash function is a deterministic procedure that takes an arbitrary block of data and returns a fixed-size bit string, the hash value, such that an accidental or intentional change to the data will change the hash value...
s are employed often. As long as the hash sums cannot be tampered with — for example, if they are communicated over a secure channel — the files can be presumed to be intact. Alternatively, digital signature
Digital signature
A digital signature or digital signature scheme is a mathematical scheme for demonstrating the authenticity of a digital message or document. A valid digital signature gives a recipient reason to believe that the message was created by a known sender, and that it was not altered in transit...
s can be employed to assure tamper-resistance.
Products
- Bitser
- Osiris
- OSSECOSSECOSSEC is a free, open source host-based intrusion detection system . It performs log analysis, integrity checking, Windows registry monitoring, rootkit detection, time-based alerting and active response. It provides intrusion detection for most operating systems, including Linux, OpenBSD, FreeBSD,...
- CimTrakCimTrakCimTrak is a commercially available File integrity monitoring and Regulatory compliance Auditing software solution. CimTrak assists in ensuring the availability and integrity of critical IT assets by instantly detecting the root-cause and responding immediately to any unexpected changes to the...