Fisher kernel

In mathematics, the Fisher kernel, named in honour of Sir Ronald Fisher, is a kernel. It was introduced in 1998 by Tommi Jaakkola ["Exploiting Generative Models in Discriminative Classifiers" (1998) [http://people.csail.mit.edu/tommi/papers/gendisc.ps PS] , [http://citeseer.ist.psu.edu/jaakkola98exploiting.html Citeseer] ] .

The Fisher kernel combines the advantages of generative statistical models (like the Hidden Markov model) and those of discriminative methods (like Support vector machine):
* generative model can process data of variable length (adding or removing data is well-supported)
* discriminative methods can have flexible criteria and yield better results.

Derivation

Fisher score

The Fisher kernel makes use of the Fisher score, defined as

$U\_X\; =\; abla\_\{\; heta\}\; log\; P(X|\; heta)$

with $heta$ being a set (vector) of parameters. $log\; P(X|\; heta)$ is the log-likelihood of the probabilistic model.

Fisher kernel

The Fisher kernel is defined as

$K(X\_i,\; X\_j)\; =\; U\_\{X\_\{i^T\; I^\{-1\}\; U\_\{X\_\{j$

with "I" the Fisher information matrix

Applications

Information retrieval

The Fisher kernel is the kernel for a generative probabilistic model. As such, it constitutes a bridge between generative and probabilistic models of documents ["Generative vs Discriminative Approaches to Entity Recognition from Label-Deficient Data" (2003) [http://www.xrce.xerox.com/Publications/Attachments/2003-079/2003_079.pdf PDF] , [http://citeseer.ist.psu.edu/goutte03generative.html Citeseer] ] . Fisher kernels exist for numerous models, notably tf–idf ["Deriving TF-IDF as a fisher kernel" (2005) [http://www-cse.ucsd.edu/users/elkan/papers/spire05.pdf PDF] [http://cat.inist.fr/?aModele=afficheN&cpsidt=17416010] ] , Naive Bayes and PLSI.

See also

* Fisher information metric

Notes and references