- Independent component analysis
Independent component analysis (ICA) is a computational method for separating a
multivariate signal into additive subcomponents supposing the mutualstatistical independence of the non-Gaussian source signals. It is a special case ofblind source separation .Definition
When the independence assumption is correct, blind ICA separation of a mixed signal gives very good results. It is also used for signals that are not supposed to be generated by a mixing for analysis purposes. A simple application of ICA is the “
cocktail party problem ”, where the underlying speech signals are separated from a sample data consisting of people talking simultaneously in a room. Usually the problem is simplified by assuming no time delays and echoes. An important note to consider is that if "N" sources are present, at least "N" observations (i.e., microphones) are needed to get the original signals. This constitutes the square ("J" = "D", where "D" is the input dimension of the data and "J" is the dimension of the model). Other cases of underdetermined ("J" < "D") and overdetermined ("J" > "D") have been investigated.The statistical method finds the independent components (aka factors, latent variables or sources) by maximizing the statistical independence of the estimated components. Non-Gaussianity, motivated by the
central limit theorem , is one method for measuring the independence of the components. Non-Gaussianity can be measured, for instance, bykurtosis or approximations ofnegentropy .Mutual information is another popular criterion for measuring statistical independence of signals.Typical algorithms for ICA use
centering , whitening (usually with theeigenvalue decomposition ), anddimensionality reduction as preprocessing steps in order to simplify and reduce the complexity of the problem for the actual iterative algorithm. Whitening anddimension reduction can be achieved withprincipal component analysis orsingular value decomposition . Whitening ensures that all dimensions are treated equally "a priori" before the algorithm is run. Algorithms for ICA includeinfomax ,FastICA , and JADE, but there are many others also.Most ICA methods are not able to extract the actual number of source signals, the order of the source signals, nor the signs or the scales of the sources.
ICA is important to
blind signal separation and has many practical applications. It is closely related to (or even a special case of) the search for afactorial code of the data, i.e., a new vector-valued representation of each data vector such that it gets uniquely encoded by the resulting code vector (loss-free coding), but the code components are statistically independent.Mathematical definitions
Linear independent component analysis can be divided into noiseless and noisy cases,where noiseless ICA is a special case of noisy ICA. Nonlinear ICA should be considered as a separate case.
General definition
The data is represented by the
random vector x=(x_1,ldots,x_m) and the components as the random vector s=(s_1,ldots,s_n). The task is to transform the observed data x, using a linear static transformation "W" as:s = W x ,,into maximally independent components s measured by some function F(s_1,ldots,s_n) of independence.Generative model
Linear noiseless ICA
The components x_i of the observed random vector x=(x_1,ldots,x_m)^T are generated as a sum of the independent components s_k, k=1,ldots,n:
x_i = a_{i,1} s_1 + ldots + a_{i,k} s_k + cdots + a_{i,n} s_n
weighted by the mixing weights a_{i,k}.
The same generative model can be written in vectorial form as x=sum_{k=1}^{n} s_k a_k,where the observed random vector x is represented by the basis vectors a_k=(a_{1,k},ldots,a_{m,k})^T.The basis vectors a_k form the columns of the mixing matrix A=(a_1,ldots,a_n) and the generative formula can be writtenas x=As, where s=(s_1,ldots,s_n)^T.
Given the model and realizations (samples) x_1,ldots,x_N of the random vector x, the task is to estimate both the mixing matrix A and the sources s. This is done by adaptively calculating the w vectors and setting up a cost function which either maximizes the nongaussianity of the calculated s_k = (w^T*x) or minimizes the mutual information. In some cases, a priori knowledge of the probability distributions of the sources can be used in the cost function.
The original sources s can be recovered by multiplying the observed signals x with the inverse of the mixing matrix W=A^{-1}, also known as the unmixing matrix. Here it is assumed that the mixing matrix is square (n=m). If the number of basis vectors is greater than the dimensionality of the observed vectors, n
, the task is overcomplete but is still solvable. Linear noisy ICA
With the added assumption of zero-mean and uncorrelated Gaussian noise nsim N(0,operatorname{diag}(Sigma)), the ICA model takes the form x=As+n.
Nonlinear ICA
The mixing of the sources does not need to be linear. Using a nonlinear mixing function f(cdot| heta) with parameters heta the
nonlinear ICA model is x=f(s| heta)+n.Identifiability
The identifiability of independent component analysis requires that:
* At most one of the sources s_k can be Gaussian,
* The number of observed mixtures, m, must be at least as large as the number of estimated components n: m ge n. It is equivalent to say that the mixing matrix A must be of full rank, for inverse to exist.See also
*
Blind deconvolution
* Blind signal separation (BSS)
*Factor analysis
*Factorial code s
*Hilbert spectrum
*Image processing
* Non-negative matrix factorization (NMF)
*Nonlinear dimensionality reduction
* Principal component analysis (PCA)
*Projection pursuit
*Redundancy reduction
*Signal processing
* Singular value decomposition (SVD)
*Varimax rotation References
*Pierre Comon (1994): " [http://www.i3s.unice.fr/~pcomon/SP94.html Independent Component Analysis: a new concept] ", Signal Processing, Elsevier, 36(3):287--314 (The original paper describing the concept of ICA)
*A. Hyvärinen, J. Karhunen, E. Oja (2001): " [http://www.cis.hut.fi/projects/ica/book/ Independent Component Analysis] ", New York: Wiley, ISBN 978-0-471-40540-5
** [http://www.cis.hut.fi/projects/ica/book/intro.pdf Introductory chapter]
*J.V. Stone, (2005): " [ftp://ftp.shef.ac.uk/pub/misc/personal/pc1jvs/papers/ica_encyc_jvs4everrit2005.pdf A Brief Introduction to Independent Component Analysis] " in Encyclopedia of Statistics in Behavioral Science, Volume 2, pp. 907–912, Editors Brian S. Everitt & David C. Howell, John Wiley & Sons, Ltd, Chichester, 2005 ISBN 978-0-470-86080-9
*T.-W. Lee (1998): "Independent component analysis: Theory and applications", Boston, Mass: Kluwer Academic Publishers, ISBN 0 7923 8261 7External links
* [http://www.cs.helsinki.fi/u/ahyvarin/whatisica.shtml What is independent component analysis?] by Aapo Hyvärinen
* [http://www.idsia.ch/~juergen/ica.html Nonlinear ICA, Unsupervised Learning, Redundancy Reduction] byJürgen Schmidhuber , with links to papers
* [http://www.cis.hut.fi/projects/ica/fastica/ FastICA as a package for Matlab, in R language, C++]
* [http://www.bsp.brain.riken.go.jp/ICALAB/ ICALAB Toolboxes] for Matlab, developed atRIKEN
* [http://www.nic.uoregon.edu/hipersat/index.php High Performance Signal Analysis Toolkit] provides C++ implementations of FastICA and Infomax
* [http://jim-stone.staff.shef.ac.uk/ Free software for ICA] by JV Stone.
* [http://mole.imm.dtu.dk/toolbox/ ICA toolbox] Matlab tools for ICA with Bell-Sejnowski, Molgedey-Schuster and mean field ICA. Developed at DTU.
* [http://www.cis.hut.fi/projects/ica/cocktail/cocktail_en.cgi Demonstration of the cocktail party problem]
* [http://sccn.ucsd.edu/eeglab/ EEGLAB Toolbox] ICA of EEG for Matlab, developed at UCSD.
* [http://sccn.ucsd.edu/fmrlab/ FMRLAB Toolbox] ICA offMRI for Matlab, developed at UCSD
* [http://brandon-merkl.blogspot.com/2005/12/independent-component-analysis.html Discussion of ICA used in a biomedical shape-representation context]
Wikimedia Foundation. 2010.