- Quadratic form (statistics)
If epsilon is a vector of n
random variable s, and Lambda is an n-dimensional symmetricsquare matrix , then thescalar quantity epsilon'Lambdaepsilon is known as a quadratic form in epsilon.Expectation
It can be shown that
:operatorname{E}left [epsilon'Lambdaepsilon ight] =operatorname{tr}left [Lambda Sigma ight] + mu'Lambdamu
where mu and Sigma are the
expected value and variance-covariance matrix of epsilon, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of mu and Sigma; in particular, normality of epsilon is not required.Variance
In general, the variance of a quadratic form depends greatly on the distribution of epsilon. However, if epsilon does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that Lambda is a symmetric matrix. Then,
:operatorname{var}left [epsilon'Lambdaepsilon ight] =2operatorname{tr}left [Lambda SigmaLambda Sigma ight] + 4mu'LambdaSigmaLambdamu
In fact, this can be generalized to find the
covariance between two quadratic forms on the same epsilon (once again, Lambda_1 and Lambda_2 must both be symmetric)::operatorname{cov}left [epsilon'Lambda_1epsilon,epsilon'Lambda_2epsilon ight] =2operatorname{tr}left [Lambda _1SigmaLambda_2 Sigma ight] + 4mu'Lambda_1SigmaLambda_2mu
Computing the variance in the non-symmetric case
Some texts incorrectly state the above variance or covariance results without enforcing Lambda to be symmetric. The case for general Lambda can be derived by noting that
:epsilon'Lambda'epsilon=epsilon'Lambdaepsilon
so
:epsilon'Lambdaepsilon=epsilon'left(Lambda+Lambda' ight)epsilon/2
But this is a quadratic form in the symmetric matrix ilde{Lambda}=left(Lambda+Lambda' ight)/2, so the mean and variance expressions are the same, provided Lambda is replaced by ilde{Lambda} therein.
Examples of quadratic forms
In the setting where one has a set of observations y and an
operator matrix H, then theresidual sum of squares can be written as a quadratic form in y::extrm{RSS}=y'left(I-H ight)'left(I-H ight)y
For procedures where the matrix H is symmetric and idempotent, and the errors are Gaussian with covariance matrix sigma^2I, extrm{RSS}/sigma^2 has a
chi-square distribution with k degrees of freedom and noncentrality parameter lambda, where:k=operatorname{tr}left [left(I-H ight)'left(I-H ight) ight] :lambda=mu'left(I-H ight)'left(I-H ight)mu/2
may be found by matching the first two
central moment s of a noncentral chi-square random variable to the expressions given in the first two sections. If Hy estimates mu with no bias, then the noncentrality lambda is zero and extrm{RSS}/sigma^2 follows a central chi-square distribution.ee also
*
Quadratic form
Wikimedia Foundation. 2010.