Relevance Vector Machine
Encyclopedia
Relevance vector machine (RVM) is a machine learning
technique that uses Bayesian inference
to obtain parsimonious solutions for regression
and classification. The RVM has an identical functional form to the support vector machine
, but provides probabilistic classification.
It is actually equivalent to a Gaussian process
model with covariance function
:
where φ is the kernel function (usually Gaussian), and x1,…,xN are the input vectors of the training set
.
Compared to that of support vector machine
s (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations)See the comment of this claim in the Discussion of this page. However RVMs use an expectation maximization (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard sequential minimal optimization
(SMO)-based algorithms employed by SVM
s, which are guaranteed to find a global optimum.
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
technique that uses Bayesian inference
Bayesian inference
In statistics, Bayesian inference is a method of statistical inference. It is often used in science and engineering to determine model parameters, make predictions about unknown variables, and to perform model selection...
to obtain parsimonious solutions for regression
Regression analysis
In statistics, regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables...
and classification. The RVM has an identical functional form to the support vector machine
Support vector machine
A support vector machine is a concept in statistics and computer science for a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis...
, but provides probabilistic classification.
It is actually equivalent to a Gaussian process
Gaussian process
In probability theory and statistics, a Gaussian process is a stochastic process whose realisations consist of random values associated with every point in a range of times such that each such random variable has a normal distribution...
model with covariance function
Covariance function
In probability theory and statistics, covariance is a measure of how much two variables change together and the covariance function describes the variance of a random variable process or field...
:
where φ is the kernel function (usually Gaussian), and x1,…,xN are the input vectors of the training set
Training set
A training set is a set of data used in various areas of information science to discover potentially predictive relationships. Training sets are used in artificial intelligence, machine learning, genetic programming, intelligent systems, and statistics...
.
Compared to that of support vector machine
Support vector machine
A support vector machine is a concept in statistics and computer science for a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis...
s (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations)See the comment of this claim in the Discussion of this page. However RVMs use an expectation maximization (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard sequential minimal optimization
Sequential Minimal Optimization
Sequential minimal optimization is an algorithm for efficiently solving the optimization problem which arises during the training of support vector machines. It was invented by John Platt in 1998 at Microsoft Research. SMO is widely used for training support vector machines and is implemented by...
(SMO)-based algorithms employed by SVM
Support vector machine
A support vector machine is a concept in statistics and computer science for a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis...
s, which are guaranteed to find a global optimum.