Sepp Hochreiter
Encyclopedia
Sepp Hochreiter is a
computer scientist
working in the fields of bioinformatics
and
machine learning
. Since 2006 he has been head of the
Institute of Bioinformatics at the
Johannes Kepler University
of
Linz
. Before, he was at the
Technical University
of Berlin
,
at the University of Colorado
at Boulder
,
and at the Technical University
of
Munich
. He founded the
Masters Program in Bioinformatics
at the Johannes Kepler University
of Linz where he is still the acting dean, he founded the
Bioinformatics Working Group at the Austrian Computer Society, he is
the representative of the University Linz at the
Asea Uninet, he is founding board
member of different bioinformatics start-up companies, he was program
chair of the conference
Bioinformatics Research and Development,
he is editor, program committee member, and
reviewer for international journals and conferences.
high-density oligonucleotide
DNA microarrays at probe level to analyze RNA
gene expression
. FARMS is based on a factor analysis
model which is optimized in a Bayesian
framework by maximizing the posterior probability
. On Affymetrix spiked-in and other benchmark data, FARMS outperformed all other methods. A highly relevant feature of FARMS is its informative/ non-informative (I/NI) calls. The I/NI call is a Bayesian filtering technique which separates signal variance from noise variance. The I/NI call offers a solution to the main problem of high dimensionality when analyzing microarray data by selecting genes which are measured with high quality. FARMS has been extended to cn.FARMS
for detecting DNA
structural variants like copy number variations
with a low false discovery rate
.
that is simultaneously clustering rows and columns of a matrix
. A bicluster in transcriptomic data is a pair of a gene set and a sample set for which the genes are similar to each other on the samples and vice versa. In drug design, for example, the effects of compounds may be similar only on a subgroup of genes. FABIA is a multiplicative model that assumes realistic non-Gaussian signal distributions with heavy tails and utilizes well understood model selection techniques like a variational approach in the
Bayesian framework. FABIA supplies the information content
of each bicluster to separate spurious biclusters from true biclusters.
methods used for
classification and regression analysis
by recognizing patterns and regularities in the data. Standard SVMs require a positive definite
kernel to generate a squared kernel matrix from the data. Sepp Hochreiter proposed the "Potential Support Vector Machine" (PSVM), which can be applied to non-square kernel matrices and can be used with kernels that are not positive definite.
For PSVM model selection he developed an efficient
sequential minimal optimization
algorithm.
The PSVM minimizes a new objective which ensures theoretical bounds on the generalization error and automatically selects features which are used for classification or regression.
, especially to
gene selection for microarray
data.
The PSVM and standard
support vector machines were applied to extract features that are indicative
coiled coil
oligomerization.
are different types
of simplified mathematical models of biological neural network
s like
those in human brain
s. If data mining
is based on neural networks, overfitting reduces the network's capability to correctly process future data. To avoid overfitting, Sepp Hochreiter
developed algorithms for finding low complexity neural networks like "Flat Minimum Search"
(FMS), which searches for a "flat" minimum — a large connected region in the parameter space where the
network function is constant. Thus, the network parameters can be given with low precision which
means a low complex network that avoids overfitting.
, which overcomes the problem of previous recurrent networks to forget information about the sequence which was observed at the begin of the sequence.
LSTM learns
from training sequences to solve numerous tasks like automatic music composition, speech recognition
, reinforcement learning
, and robotics
.
LSTM was successfully applied to very fast
protein homology
detection without requiring
a sequence alignment
.
computer scientist
Computer scientist
A computer scientist is a scientist who has acquired knowledge of computer science, the study of the theoretical foundations of information and computation and their application in computer systems....
working in the fields of bioinformatics
Bioinformatics
Bioinformatics is the application of computer science and information technology to the field of biology and medicine. Bioinformatics deals with algorithms, databases and information systems, web technologies, artificial intelligence and soft computing, information and computation theory, software...
and
machine learning
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
. Since 2006 he has been head of the
Institute of Bioinformatics at the
Johannes Kepler University
Johannes Kepler University of Linz
The Johannes Kepler University of Linz is a public institution of higher education in Linz, the capital of Upper Austria...
of
Linz
Linz
Linz is the third-largest city of Austria and capital of the state of Upper Austria . It is located in the north centre of Austria, approximately south of the Czech border, on both sides of the river Danube. The population of the city is , and that of the Greater Linz conurbation is about...
. Before, he was at the
Technical University
Technical University of Berlin
The Technische Universität Berlin is a research university located in Berlin, Germany. Translating the name into English is discouraged by the university, however paraphrasing as Berlin Institute of Technology is recommended by the university if necessary .The TU Berlin was founded...
of Berlin
Berlin
Berlin is the capital city of Germany and is one of the 16 states of Germany. With a population of 3.45 million people, Berlin is Germany's largest city. It is the second most populous city proper and the seventh most populous urban area in the European Union...
,
at the University of Colorado
University of Colorado at Boulder
The University of Colorado Boulder is a public research university located in Boulder, Colorado...
at Boulder
Boulder, Colorado
Boulder is the county seat and most populous city of Boulder County and the 11th most populous city in the U.S. state of Colorado. Boulder is located at the base of the foothills of the Rocky Mountains at an elevation of...
,
and at the Technical University
Technical University of Munich
The Technische Universität München is a research university with campuses in Munich, Garching, and Weihenstephan...
of
Munich
Munich
Munich The city's motto is "" . Before 2006, it was "Weltstadt mit Herz" . Its native name, , is derived from the Old High German Munichen, meaning "by the monks' place". The city's name derives from the monks of the Benedictine order who founded the city; hence the monk depicted on the city's coat...
. He founded the
Masters Program in Bioinformatics
at the Johannes Kepler University
of Linz where he is still the acting dean, he founded the
Bioinformatics Working Group at the Austrian Computer Society, he is
the representative of the University Linz at the
Asea Uninet, he is founding board
member of different bioinformatics start-up companies, he was program
chair of the conference
Bioinformatics Research and Development,
he is editor, program committee member, and
reviewer for international journals and conferences.
Microarray Preprocessing and Summarization
Sepp Hochreiter developed "Factor Analysis for Robust Microarray Summarization" (FARMS). FARMS has been designed for preprocessing and summarizingMicroarray analysis techniques
Microarray analysis techniques are used in interpreting the data generated from experiments on DNA, RNA, and protein microarrays, which allow researchers to investigate the expression state of a large number of genes - in many cases, an organism's entire genome - in a single experiment...
high-density oligonucleotide
Oligonucleotide
An oligonucleotide is a short nucleic acid polymer, typically with fifty or fewer bases. Although they can be formed by bond cleavage of longer segments, they are now more commonly synthesized, in a sequence-specific manner, from individual nucleoside phosphoramidites...
DNA microarrays at probe level to analyze RNA
RNA
Ribonucleic acid , or RNA, is one of the three major macromolecules that are essential for all known forms of life....
gene expression
Gene expression
Gene expression is the process by which information from a gene is used in the synthesis of a functional gene product. These products are often proteins, but in non-protein coding genes such as ribosomal RNA , transfer RNA or small nuclear RNA genes, the product is a functional RNA...
. FARMS is based on a factor analysis
Factor analysis
Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved, uncorrelated variables called factors. In other words, it is possible, for example, that variations in three or four observed variables...
model which is optimized in a Bayesian
Bayesian probability
Bayesian probability is one of the different interpretations of the concept of probability and belongs to the category of evidential probabilities. The Bayesian interpretation of probability can be seen as an extension of logic that enables reasoning with propositions, whose truth or falsity is...
framework by maximizing the posterior probability
Posterior probability
In Bayesian statistics, the posterior probability of a random event or an uncertain proposition is the conditional probability that is assigned after the relevant evidence is taken into account...
. On Affymetrix spiked-in and other benchmark data, FARMS outperformed all other methods. A highly relevant feature of FARMS is its informative/ non-informative (I/NI) calls. The I/NI call is a Bayesian filtering technique which separates signal variance from noise variance. The I/NI call offers a solution to the main problem of high dimensionality when analyzing microarray data by selecting genes which are measured with high quality. FARMS has been extended to cn.FARMS
for detecting DNA
DNA
Deoxyribonucleic acid is a nucleic acid that contains the genetic instructions used in the development and functioning of all known living organisms . The DNA segments that carry this genetic information are called genes, but other DNA sequences have structural purposes, or are involved in...
structural variants like copy number variations
with a low false discovery rate
False discovery rate
False discovery rate control is a statistical method used in multiple hypothesis testing to correct for multiple comparisons. In a list of rejected hypotheses, FDR controls the expected proportion of incorrectly rejected null hypotheses...
.
Biclustering
Sepp Hochreiter developed "Factor Analysis for Bicluster Acquisition" (FABIA) for biclusteringBiclustering
Biclustering, co-clustering, or two-mode clustering is a data mining technique which allows simultaneous clustering of the rows and columns of a matrix....
that is simultaneously clustering rows and columns of a matrix
Matrix (mathematics)
In mathematics, a matrix is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements isMatrices of the same size can be added or subtracted element by element...
. A bicluster in transcriptomic data is a pair of a gene set and a sample set for which the genes are similar to each other on the samples and vice versa. In drug design, for example, the effects of compounds may be similar only on a subgroup of genes. FABIA is a multiplicative model that assumes realistic non-Gaussian signal distributions with heavy tails and utilizes well understood model selection techniques like a variational approach in the
Bayesian framework. FABIA supplies the information content
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
of each bicluster to separate spurious biclusters from true biclusters.
Support Vector Machines
Support vector machines (SVMs) are supervised learningSupervised learning
Supervised learning is the machine learning task of inferring a function from supervised training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object and a desired output value...
methods used for
classification and regression analysis
Regression analysis
In statistics, regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables...
by recognizing patterns and regularities in the data. Standard SVMs require a positive definite
kernel to generate a squared kernel matrix from the data. Sepp Hochreiter proposed the "Potential Support Vector Machine" (PSVM), which can be applied to non-square kernel matrices and can be used with kernels that are not positive definite.
For PSVM model selection he developed an efficient
sequential minimal optimization
Sequential Minimal Optimization
Sequential minimal optimization is an algorithm for efficiently solving the optimization problem which arises during the training of support vector machines. It was invented by John Platt in 1998 at Microsoft Research. SMO is widely used for training support vector machines and is implemented by...
algorithm.
The PSVM minimizes a new objective which ensures theoretical bounds on the generalization error and automatically selects features which are used for classification or regression.
Feature Selection
Sepp Hochreiter applied the PSVM to feature selectionFeature selection
In machine learning and statistics, feature selection, also known as variable selection, feature reduction, attribute selection or variable subset selection, is the technique of selecting a subset of relevant features for building robust learning models...
, especially to
gene selection for microarray
data.
The PSVM and standard
support vector machines were applied to extract features that are indicative
coiled coil
Coiled coil
A coiled coil is a structural motif in proteins, in which 2-7 alpha-helices are coiled together like the strands of a rope . Many coiled coil type proteins are involved in important biological functions such as the regulation of gene expression e.g. transcription factors...
oligomerization.
Low Complex Neural Networks
Neural networksNeural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
are different types
Types of artificial neural networks
There are many types of artificial neural networks . An artificial neural network is a computational simulation of a biological neural network...
of simplified mathematical models of biological neural network
Biological neural network
In neuroscience, a biological neural network describes a population of physically interconnected neurons or a group of disparate neurons whose inputs or signalling targets define a recognizable circuit. Communication between neurons often involves an electrochemical process...
s like
those in human brain
Human brain
The human brain has the same general structure as the brains of other mammals, but is over three times larger than the brain of a typical mammal with an equivalent body size. Estimates for the number of neurons in the human brain range from 80 to 120 billion...
s. If data mining
Data mining
Data mining , a relatively young and interdisciplinary field of computer science is the process of discovering new patterns from large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics and database systems...
is based on neural networks, overfitting reduces the network's capability to correctly process future data. To avoid overfitting, Sepp Hochreiter
developed algorithms for finding low complexity neural networks like "Flat Minimum Search"
(FMS), which searches for a "flat" minimum — a large connected region in the parameter space where the
network function is constant. Thus, the network parameters can be given with low precision which
means a low complex network that avoids overfitting.
Recurrent Neural Networks
Recurrent neural networks scan and process sequences and supply their results to the environment. Sepp Hochreiter developed the long short term memoryLong short term memory
Long short term memory or LSTM is a recurrent neural network architecture published in 1997 by Sepp Hochreiter and Jürgen Schmidhuber...
, which overcomes the problem of previous recurrent networks to forget information about the sequence which was observed at the begin of the sequence.
LSTM learns
Learning
Learning is acquiring new or modifying existing knowledge, behaviors, skills, values, or preferences and may involve synthesizing different types of information. The ability to learn is possessed by humans, animals and some machines. Progress over time tends to follow learning curves.Human learning...
from training sequences to solve numerous tasks like automatic music composition, speech recognition
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
, reinforcement learning
Reinforcement learning
Inspired by behaviorist psychology, reinforcement learning is an area of machine learning in computer science, concerned with how an agent ought to take actions in an environment so as to maximize some notion of cumulative reward...
, and robotics
Robotics
Robotics is the branch of technology that deals with the design, construction, operation, structural disposition, manufacture and application of robots...
.
LSTM was successfully applied to very fast
protein homology
Homology modeling
Homology modeling, also known as comparative modeling of protein refers to constructing an atomic-resolution model of the "target" protein from its amino acid sequence and an experimental three-dimensional structure of a related homologous protein...
detection without requiring
a sequence alignment
Sequence alignment
In bioinformatics, a sequence alignment is a way of arranging the sequences of DNA, RNA, or protein to identify regions of similarity that may be a consequence of functional, structural, or evolutionary relationships between the sequences. Aligned sequences of nucleotide or amino acid residues are...
.
Software Downloads
- Microarray Probe Level Analysis (mRNA, copy numbers):
- Biclustering / Clustering
- Protein Structure
- Support Vector Machines
- Neural Networks