Sparse distributed memory
Encyclopedia
Sparse distributed memory is a mathematical model of human long-term memory that was introduced by Pentti Kanerva
in 1988. It is used for storing and retrieving large amounts(
bits
) of information without focusing on the accuracy of the information. It uses patterns to serve as memory address
es, where information is retrieved based on similarities between addresses. Memory addresses are all in a list even if they are not related, but are only retrieved based on similar content between themselves.
where n is the number of dimensions of the space, and 
is the number of feasible memory items.
to help model the large amounts of memory that humans. It utilizes the Hamming distance
to measure mismatched bits and read back data between the original write address and one near it. Human memory has a tendency to congregate memories based on similarities between them(although they may not be related), such as firetrucks are red and apples are red.
s are electrically excitable cells that transmit information to and from the brain. Thus, they are used as models for sending and retrieving data in a sparse distributed memory system. Neurons are the cells that recall and send information in a memory system. Neurons are used as address decoders for the brain, where they search for information inside your memory and take that and tell your brain that it is what you want.
(RAM), contrary to sequential access memory
. All of the items in a single list, or array, are stored in RAM. The computer has address decoders, similar to the way neurons work in the brain, and return items from the array that match or are similar. Each address in an array points to an individual line in the memory. That line is then returned if it is similar to other lines.
Imagine each line as a different memory address, an example from Manerva's book:
As a result, all of these addresses are returned
to the user, although these may not be the only addresses in that list.
At the University of Memphis, Uma Ramamurthy, Sidney K. D’Mello, and Stan Franklin created a modified version of the Sparse Distributed Memory system that mathematically represents "realizing forgetting." It uses a decay equation to better show interference in data. The Sparse Distributed Memory system distributes each pattern into approximately one hundredth of the locations, so interference can have detrimental results.
Two possible examples of decay from this modified Sparse Distributed Memory are presented
Exponential Decay Mechanism:
Negated-Translated Sigmoid Decay Mechanism:
In the exponential decay function, it approaches zero more quickly as x increases, and a is a constant(usually between 3-9) and c is a counter. For the negated-translated
sigmoid function
, the decay is similar to the exponential decay function when a is greater than 4.
As the graph approaches 0, it represents how the memory is being forgotten using decay mechanisms.
uses genetic algorithm and sparse distributed memory as an artificial neural network. It has been considered for use in creating artificial life.
uses sparse distributed memory to help model cognition
in biological systems. The sparse distributed memory places space is recalling or recognizing the object that is has to in relation to other objects. It was developed by Stan Franklin, the same creator of the "realizing forgetting" modified sparse distributed memory system.
Pentti Kanerva
Pentti Kanerva is a research affiliate at the Redwood Neuroscience Institute, and is the author of sparse distributed memory. He is responsible for relating the properties of long-term memory to mathematical properties of high-dimensional spaces and compares artificial neural-net associative memory...
in 1988. It is used for storing and retrieving large amounts(
bitsBITS
BITS or bits may refer to:* Plural of bit* Background Intelligent Transfer Service, a file transfer protocol* Birla Institute of Technology and Science, a technology school in Pilani, Rajasthan, India, with campuses in Goa, Hyderabad, and Dubai...
) of information without focusing on the accuracy of the information. It uses patterns to serve as memory address
Memory address
A digital computer's memory, more specifically main memory, consists of many memory locations, each having a memory address, a number, analogous to a street address, at which computer programs store and retrieve, machine code or data. Most application programs do not directly read and write to...
es, where information is retrieved based on similarities between addresses. Memory addresses are all in a list even if they are not related, but are only retrieved based on similar content between themselves.
Formula
The general formula is where n is the number of dimensions of the space, and is the number of feasible memory items.Principle
Sparse Distributed Memory is a mathematical representation of human memory, and uses high-dimensional spaceClustering high-dimensional data
Clustering high-dimensional data is the cluster analysis of data with anywhere from a few dozen to many thousands of dimensions. Such high-dimensional data spaces are often encountered in areas such as medicine, where DNA microarray technology can produce a large number of measurements at once, and...
to help model the large amounts of memory that humans. It utilizes the Hamming distance
Hamming distance
In information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different...
to measure mismatched bits and read back data between the original write address and one near it. Human memory has a tendency to congregate memories based on similarities between them(although they may not be related), such as firetrucks are red and apples are red.
Neurons
NeuronNeuron
A neuron is an electrically excitable cell that processes and transmits information by electrical and chemical signaling. Chemical signaling occurs via synapses, specialized connections with other cells. Neurons connect to each other to form networks. Neurons are the core components of the nervous...
s are electrically excitable cells that transmit information to and from the brain. Thus, they are used as models for sending and retrieving data in a sparse distributed memory system. Neurons are the cells that recall and send information in a memory system. Neurons are used as address decoders for the brain, where they search for information inside your memory and take that and tell your brain that it is what you want.
Computers
Memory inside of a computer is random-access memoryRandom-access memory
Random access memory is a form of computer data storage. Today, it takes the form of integrated circuits that allow stored data to be accessed in any order with a worst case performance of constant time. Strictly speaking, modern types of DRAM are therefore not random access, as data is read in...
(RAM), contrary to sequential access memory
Sequential access memory
In computing, sequential access memory is a class of data storage devices that read their data in sequence. This is in contrast to random access memory where data can be accessed in any order...
. All of the items in a single list, or array, are stored in RAM. The computer has address decoders, similar to the way neurons work in the brain, and return items from the array that match or are similar. Each address in an array points to an individual line in the memory. That line is then returned if it is similar to other lines.
Example
Sparse Distributed Memory is based off of pulling in patterns between different addresses.Imagine each line as a different memory address, an example from Manerva's book:
- "Why are fire engines painted red?
- Firemen's suspenders are red, too.
- Two and two are four.
- Four times three is twelve.
- Twelve inches in a foot.
- A foot is a ruler.
- Queen Mary is a ruler.
- Queen Mary sailed the sea.
- The sea has sharks.
- Sharks have fins.
- The Russians conquered the Finns.
- The Russians' color is red.
- Fire engines are always rushin'.
- So that's why they're painted red!"
As a result, all of these addresses are returned
Return statement
In computer programming, a return statement causes execution to leave the current subroutine and resume at the point in the code immediately after where the subroutine was called, known as its return address. The return address is saved, usually on the process's call stack, as part of the operation...
to the user, although these may not be the only addresses in that list.
"Realizing Forgetting"
|
|
||||
| x | Independent variable | |||
| a | Constant | |||
| c | Counter | |||
At the University of Memphis, Uma Ramamurthy, Sidney K. D’Mello, and Stan Franklin created a modified version of the Sparse Distributed Memory system that mathematically represents "realizing forgetting." It uses a decay equation to better show interference in data. The Sparse Distributed Memory system distributes each pattern into approximately one hundredth of the locations, so interference can have detrimental results.
Two possible examples of decay from this modified Sparse Distributed Memory are presented
Exponential Decay Mechanism:
Negated-Translated Sigmoid Decay Mechanism:
In the exponential decay function, it approaches zero more quickly as x increases, and a is a constant(usually between 3-9) and c is a counter. For the negated-translated
Translation (geometry)
In Euclidean geometry, a translation moves every point a constant distance in a specified direction. A translation can be described as a rigid motion, other rigid motions include rotations and reflections. A translation can also be interpreted as the addition of a constant vector to every point, or...
sigmoid function
Sigmoid function
Many natural processes, including those of complex system learning curves, exhibit a progression from small beginnings that accelerates and approaches a climax over time. When a detailed description is lacking, a sigmoid function is often used. A sigmoid curve is produced by a mathematical...
, the decay is similar to the exponential decay function when a is greater than 4.
As the graph approaches 0, it represents how the memory is being forgotten using decay mechanisms.
Genetic Memory
Genetic memoryGenetic memory (computer science)
In computer science, genetic memory refers to an artificial neural network combination of genetic algorithm and the mathematical model of sparse distributed memory. It can be used to predict weather patterns. Genetic memory and genetic algorithms have also gained an interest in the creation of...
uses genetic algorithm and sparse distributed memory as an artificial neural network. It has been considered for use in creating artificial life.
LIDA
LIDALIDA (cognitive architecture)
The LIDA cognitive architecture is an integrated artificial cognitive system that attempts to model a broad spectrum of cognition in biological systems, from low-level perception/action to high-level reasoning. Developed primarily by Stan Franklin and colleagues at the University of Memphis, the...
uses sparse distributed memory to help model cognition
Cognition
In science, cognition refers to mental processes. These processes include attention, remembering, producing and understanding language, solving problems, and making decisions. Cognition is studied in various disciplines such as psychology, philosophy, linguistics, and computer science...
in biological systems. The sparse distributed memory places space is recalling or recognizing the object that is has to in relation to other objects. It was developed by Stan Franklin, the same creator of the "realizing forgetting" modified sparse distributed memory system.