Quadratic form (statistics)

If $epsilon$ is a vector of $n$ random variables, and $Lambda$ is an $n$-dimensional symmetric square matrix, then the scalar quantity $epsilon\text{'}Lambdaepsilon$ is known as a quadratic form in $epsilon$.

Expectation

It can be shown that

:$operatorname\{E\}left\; [epsilon\text{'}Lambdaepsilon\; ight]\; =operatorname\{tr\}left\; [Lambda\; Sigma\; ight]\; +\; mu\text{'}Lambdamu$

where $mu$ and $Sigma$ are the expected value and variance-covariance matrix of $epsilon$, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of $mu$ and $Sigma$; in particular, normality of $epsilon$ is not required.

Variance

In general, the variance of a quadratic form depends greatly on the distribution of $epsilon$. However, if $epsilon$ does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that $Lambda$ is a symmetric matrix. Then,

:$operatorname\{var\}left\; [epsilon\text{'}Lambdaepsilon\; ight]\; =2operatorname\{tr\}left\; [Lambda\; SigmaLambda\; Sigma\; ight]\; +\; 4mu\text{'}LambdaSigmaLambdamu$

In fact, this can be generalized to find the covariance between two quadratic forms on the same $epsilon$ (once again, $Lambda\_1$ and $Lambda\_2$ must both be symmetric):

:$operatorname\{cov\}left\; [epsilon\text{'}Lambda\_1epsilon,epsilon\text{'}Lambda\_2epsilon\; ight]\; =2operatorname\{tr\}left\; [Lambda\; \_1SigmaLambda\_2\; Sigma\; ight]\; +\; 4mu\text{'}Lambda\_1SigmaLambda\_2mu$

Computing the variance in the non-symmetric case

Some texts incorrectly state the above variance or covariance results without enforcing $Lambda$ to be symmetric. The case for general $Lambda$ can be derived by noting that

:$epsilon\text{'}Lambda\text{'}epsilon=epsilon\text{'}Lambdaepsilon$

so

:$epsilon\text{'}Lambdaepsilon=epsilon\text{'}left(Lambda+Lambda\text{'}\; ight)epsilon/2$

But this is a quadratic form in the symmetric matrix $ilde\{Lambda\}=left(Lambda+Lambda\text{'}\; ight)/2$, so the mean and variance expressions are the same, provided $Lambda$ is replaced by $ilde\{Lambda\}$ therein.

Examples of quadratic forms

In the setting where one has a set of observations $y$ and an operator matrix $H$, then the residual sum of squares can be written as a quadratic form in $y$:

:$extrm\{RSS\}=y\text{'}left(I-H\; ight)\text{'}left(I-H\; ight)y$

For procedures where the matrix $H$ is symmetric and idempotent, and the errors are Gaussian with covariance matrix $sigma^2I$, $extrm\{RSS\}/sigma^2$ has a chi-square distribution with $k$ degrees of freedom and noncentrality parameter $lambda$, where

:$k=operatorname\{tr\}left\; [left(I-H\; ight)\text{'}left(I-H\; ight)\; ight]$:$lambda=mu\text{'}left(I-H\; ight)\text{'}left(I-H\; ight)mu/2$

may be found by matching the first two central moments of a noncentral chi-square random variable to the expressions given in the first two sections. If $Hy$ estimates $mu$ with no bias, then the noncentrality $lambda$ is zero and $extrm\{RSS\}/sigma^2$ follows a central chi-square distribution.

ee also

*Quadratic form