Optimal projection equations

In control, the Linear-Quadratic-Gaussian (LQG) control problem is one of the most fundamental optimal control problems. It concerns uncertain linear systems disturbed by additive white Gaussian noise, incomplete state information (i.e. not all the state variables are measured and available for feedback) also disturbed by additive white Gaussian noise and quadratic costs. Moreover the solution is unique and constitutes a linear dynamic feedback control law that is easily computed and implemented. Finally the LQG controller is also fundamental to the optimal perturbation control of non-linear systems ^[1].

The LQG controller itself is a dynamic system like the system it controls. Both systems have the same state dimension. Therefore implementing the LQG controller may be problematic if the dimension of the system state is large. The reduced-order LQG problem (fixed-order LQG problem) overcomes this by fixing a-priori the number of states of the LQG controller. This problem is more difficult to solve because it is no longer separable. Also the solution is no longer unique. Despite these facts numerical algorithms are available ^[2][3][4][5] to solve the associated optimal projection equations ^[6][7][8] which constitute necessary and sufficient conditions for a locally optimal reduced-order LQG controller ^[2].

Mathematical problem formulation and solution

Continuous-time

The Reduced-order LQG control problem is almost identical to the conventional full-order LQG control problem. Let $\hat{\mathbf{x}}_r(t)$ represent the state of the reduced-order LQG controller. Then the only difference is that the state dimension $n_r=dim(\hat{\mathbf{x}}_r(t))$ of the LQG controller is a-priori fixed to be smaller than $n=dim({\mathbf{x}}(t))$, the state dimension of the controlled system.

The reduced-order LQG controller is represented by the following equations,

$\dot{\hat{\mathbf{x}}}_r(t) = A_r(t)\hat{\mathbf{x}}_r(t) + B_r(t){\mathbf{u}}(t)+K_r(t) \left( {\mathbf{y}}(t)-C_r(t)\hat{\mathbf{x}}_r(t) \right),\hat{\mathbf{x}}_r(0)={\mathbf{x}}_r(0),$

${\mathbf{u}}(t)= -L_r(t) \hat{\mathbf{x}}_r(t).$

These equations are delibarately stated in a format that equals that of the conventional full-order LQG controller. For the reduced-order LQG control problem it is convenient to rewrite them:

$\dot{\hat{\mathbf{x}}}_r(t) = F_r(t)\hat{\mathbf{x}}_r(t) + K_r(t) {\mathbf{y}}(t),\hat{\mathbf{x}}_r(0)={\mathbf{x}}_r(0),$

${\mathbf{u}}(t)= -L_r(t) \hat{\mathbf{x}}_r(t),$

where,

${\mathbf{}}F_r(t)=A_r(t)-B_r(t)L_r(t)-K_r(t)C_r(t)$.

The matrices ${\mathbf{}}F_r(t), K_r(t), L_r(t)$ and ${\mathbf{x}}_r(0)$ of the reduced-order LQG controller are determined by the so called optimal projection equations (OPE)^[8].

The square optimal projection matrix ${\mathbf{}}\tau(t)$ with dimension ${\mathbf{}}n$ is central to the OPE. The rank of this matrix is almost everywhere equal to ${\mathbf{}}n_r.$ The associated projection is an oblique projection: ${\mathbf{}}\tau^2(t)=\tau(t).$ The OPE constitute four matrix differential equations. The first two equations listed below are generalizations of the matrix Riccati differential equations associated to the conventional full-order LQG controller. In these equations ${\mathbf{}}\tau_\perp(t)$ denotes ${\mathbf{}}I_n-\tau(t)$ where ${\mathbf{}}I_n$ is the identity matrix of dimension ${\mathbf{}}n$.

$\dot{P}(t) = A(t)P(t)+P(t)A'(t)-P(t)C'(t)W^{-1}(t) C(t)P(t)+V(t)$

$+\tau_\perp (t)P(t)C'(t)W^{-1}(t) C(t)P(t)\tau'_\perp (t),$

$P(0)= E \left({\mathbf{x}}(0){\mathbf{x}}'(0) \right),$

$-\dot{S}(t) = A'(t)S(t)+S(t)A(t)-S(t)B(t)R^{-1}(t)B'(t)S(t)+Q(t)$

$+ \tau'_\perp (t)S(t)B(t)R^{-1}(t)B'(t)S(t) \tau_\perp (t),$

${\mathbf{}}S(T) = F.$

If the dimension of the LQG controller is not reduced, that is if ${\mathbf{}}n=n_r$, then $\tau(t)=I_n, \tau_\perp(t)=0$ and the two equations above become the uncoupled matrix Riccati differential equations associated to the conventional full-order LQG controller. If ${\mathbf{}}n_r<n$ the two equations are coupled by the oblique projection ${\mathbf{}}\tau(t).$ This reveals why the reduced-order LQG problem is not separable. The oblique projection ${\mathbf{}}\tau(t)$ is determined from two additional matrix differential equations which involve rank conditions. Together with the previous two matrix differential equations these are the OPE. To state the additional two matrix differential equations it is convenient to introduce the following two matrices,

$\Psi_1(t)=(A(t)-B(t)R^{-1}(t)B'(t)S(t))\hat{P}(t)+\hat{P}(t) (A(t)-B(t)R^{-1}(t)B'(t)S(t))'$

${\mathbf{}}+P(t)C'(t)W^{-1}(t)C(t)P(t),$

$\Psi_2(t)=(A(t)-P(t)C'(t)W^{-1}(t) C(t))'\hat{S}(t)+\hat{S}(t)(A(t)-P(t)C'(t)W^{-1}(t)C(t))$

${\mathbf{}}+S(t)B(t)R^{-1}(t)B'(t)S(t).$

Then the two additional matrix differential equations that complete the OPE read as follows,

$\dot{\hat{P}}(t)=1/2 \left(\tau(t)\Psi_1(t)+\Psi_1(t)\tau'(t) \right),\hat{P}(0)=E({\mathbf{x}}(0))E({\mathbf{x}}(0))', rank(\hat{P}(t))=n_r$ almost everywhere,

$-\dot{\hat{S}}(t)=1/2 \left(\tau'(t)\Psi_2(t)+\Psi_2(t)\tau(t) \right),\hat{S}(T)=0, rank(\hat{S}(t))=n_r$ almost everywhere,

with,

${\mathbf{}}\tau(t)= \hat{P}(t) \hat{S}(t) \left( \hat{P}(t) \hat{S}(t) \right)^*.$

Here * denotes the group generalized inverse or Drazin inverse that is unique and given by,

${\mathbf{}}A^*=A(A^3)^+A.$

where + denotes the Moore-Penrose pseudoinverse.

The matrices ${\mathbf{}}P(t),S(t),\hat{P}(t),\hat{S}(t)$ must all be nonnegative symmetric. Then they constitute a solution of the OPE that determines the reduced-order LQG controller matrices ${\mathbf{}}F_r(t), K_r(t), L_r(t)$ and ${\mathbf{x}}_r(0)$:

${\mathbf{}}F_r(t)=H(t)\left( A(t)-P(t)C'(t)W^{-1}(t) C(t)-B(t)R^{-1}(t)B'(t)S(t) \right)G(t)+\dot{H}(t)G'(t),$

${\mathbf{}}K_r(t)=H(t)P(t)C'(t)W^{-1}(t),$

${\mathbf{}}L_r(t)=R^{-1}(t)B'(t)S(t)G'(t),$

${\mathbf{x}}_r(0)=H(0)E({\mathbf{x}}(0)).$

In the equations above the matrices ${\mathbf{}}G(t),H(t)$ are two matrices with the following properties,

${\mathbf{}}G'(t)H(t)=\tau(t),G(t)H'(t)=I_{n_r}$ almost everywhere.

They can be obtained from a projective factorization of ${\mathbf{}}\hat{P}(t)\hat{S}(t)$ ^[2].

The OPE can be stated in many different ways that are all equivalent. To identify the equivalent representations the following identities are especially useful,

${\mathbf{}}\tau(t)\hat{P}(t)=\hat{P}(t)\tau'(t)=\hat{P}(t), \tau'(t)\hat{S}(t)=\hat{S}(t)\tau(t)=\hat{S}(t)$

Using these identities one may for instance rewrite the first two of the optimal projection equations as follows,

$\dot{P}(t) = A(t)P(t)+P(t)A'(t)-P(t)C'(t)W^{-1}(t)C(t)P(t)+V(t)+\tau_\perp(t)\Psi_1(t)\tau'_\perp (t),$

$P(0)= E \left({\mathbf{x}}(0){\mathbf{x}}'(0) \right),$

$-\dot{S}(t) = A'(t)S(t)+S(t)A(t)-S(t)B(t)R^{-1}(t)B'(t)S(t)+Q(t)+\tau'_\perp\Psi_2(t)\tau_\perp(t),$

${\mathbf{}}S(T) = F.$

This representation is both relatively simple and suitable for numerical computations.

If all the matrices in the reduced-order LQG problem formulation are time-invariant and if the horizon ${\mathbf{}}T$ tends to infinity, the optimal reduced-order LQG controller becomes time-invariant and so do the OPE ^[6]. In that case the derivatives on the left hand side of the OPE are zero.

Discrete-time

Similar to the continuous-time case in the discrete-time case the difference with the conventional discrete-time full-order LQG problem is the a-priori fixed reduced-order ${\mathbf{}}n_r<n$ of the LQG controller state dimension. As in continuous-time to state the discrete-time OPE it is convenient to introduce the following two matrices,

${\mathbf{}}\Psi^1_i=\left(A_i-B_i(B'_iS_{i+1}B_i+R_i)^{-1}B'_iS_{i+1}A_i)\right)\hat{P}_i \left(A_i-B_i(B'_iS_{i+1}B_i+R_i)^{-1}B'_iS_{i+1}A_i)\right)'$

${\mathbf{}}+A_iP_iC'_i(C_iP_iC'_i+W_i)^{-1}C_iP_iA'_i$

${\mathbf{}}\Psi^2_{i+1}=\left(A_i-A_iP_iC'_i(C_iP_{i}C'_i+W_i)^{-1}C_i\right)'\hat{S}_{i+1} \left(A_i-A_iP_iC'_i(C_iP_{i}C'_i+W_i)^{-1}C_i\right)$

${\mathbf{}}+A'_iS_{i+1}B_i(B'_iS_{i+1}B_i+R_i)^{-1}B'_iS_{i+1}A_i$

Then the discrete-time OPE read as follows,

${\mathbf{}}P_{i+1} = A_i \left( P_i - P_i C'_i \left( C_i P_i C'_i+W_i \right)^{-1} C_i P_i \right) A'_i+V_i+\tau_{\perp i+1}\Psi^1_i \tau'_{\perp i+1}, P_0=E \left( {\mathbf{x}}_0{\mathbf{x'}}_0 \right)$.

${\mathbf{}}S_{i} = A'_i \left( S_{i+1} - S_{i+1}B_i \left( B'_iS_{i+1}B_i+R_i \right)^{-1} B'_i S_{i+1} \right) A_i+Q_i+\tau'_{\perp i}\Psi^2_{i+1} \tau_{\perp i}, S_N=F$.

${\mathbf{}}\hat{P}_{i+1}=1/2(\tau_{i+1}\Psi_i^1+\Psi_i^1\tau'_{i+1}),\hat{P}_0=E({\mathbf{x}}(0))E({\mathbf{x}}(0))', rank(\hat{P}_i)=n_r$ almost everywhere,

${\mathbf{}}\hat{S}_{i}=1/2(\tau'_{i}\Psi_{i+1}^2+\Psi_{i+1}^2\tau_{i}),\hat{S}_N=0, rank(\hat{S}_i)=n_r$ almost everywhere.

The oblique projection matrix is given by,

${\mathbf{}}\tau_i=\hat{P}_i\hat{S}_i \left(\hat{P}_i\hat{S}_i \right)^*.$

The nonnegative symmetric matrices ${\mathbf{}}P_i,S_i,\hat{P}_i,\hat{S}_i$ that solve the discrete-time OPE determine the reduced-order LQG controller matrices ${\mathbf{}}F_i^r, K_i^r, L_i^r$ and ${\mathbf{x}}_0^r$:

${\mathbf{}}F_i^r=H_{i+1}\left( A_i-P_i C'_i \left( C_i P_i C'_i+W_i \right)^{-1}C_i-B_i\left( B'_iS_{i+1}B_i+R_i \right)^{-1} B'_i S_{i+1}\right)G'_i,$

${\mathbf{}}K_i^r=H_{i+1}P_i C'_i \left( C_i P_i C'_i+W_i \right)^{-1},$

${\mathbf{}}L_i^r=\left( B'_iS_{i+1}B_i+R_i \right)^{-1} B'_i S_{i+1}G'_i,$

${\mathbf{x}}_0^r=H_0E({\mathbf{x}}_0).$

In the equations above the matrices ${\mathbf{}}G_i,H_i$ are two matrices with the following properties,

${\mathbf{}}G'_iH_i=\tau_i, G_iH'_i=I_{n_r}$ almost everywhere.

They can be obtained from a projective factorization of ${\mathbf{}}\hat{P}_i\hat{S}_i$ ^[2]. To identify equivalent representations of the discrete-time OPE the following identities are especially useful,

${\mathbf{}}\tau_i\hat{P}_i=\hat{P}_i\tau'_i=\hat{P}_i, \tau'_i\hat{S}_i=\hat{S}_i\tau_i=\hat{S}_i$

As in the continuous-time case if all the matrices in the problem formulation are time-invariant and if the horizon ${\mathbf{}}N$ tends to infinity the reduced-order LQG controller becomes time-invariant. Then the discrete-time OPE converge to a steady state solution that determines the time-invariant reduced-order LQG controller^[7].

The discrete-time OPE apply also to discrete-time systems with variable state, input and output dimensions (discrete-time systems with time-varying dimensions)^[3]. Such systems arise in the case of digital controller design if the sampling occurs asynchronously.

References

1. ^ Athans M. (1971). "The role and use of the stochastic Linear-Quadratic-Gaussian problem in control system design". IEEE Transaction on Automatic Control AC-16 (6): 529–552. doi:10.1109/TAC.1971.1099818. 
2. ^ ^{a b c d} Van Willigenburg L.G., De Koning W.L. (2000). "Numerical algorithms and issues concerning the discrete-time optimal projection equations". European Journal of Control 6 (1): 93–100.  Associated software download from Matlab Central.
3. ^ ^{a b} Van Willigenburg L.G., De Koning W.L. (1999). "Optimal reduced-order compensators for time-varying discrete-time systems with deterministic and white parameters". Automatica 35: 129–138. doi:10.1016/S0005-1098(98)00138-1.  Associated software download from Matlab Central.
4. ^ Zigic D., Watson L.T., Collins E.G., Haddad W.M., Ying S. (1996). "Homotopy methods for solving the optimal projection equations for the H2 reduced order model problem". International Journal of Control 56 (1): 173–191. doi:10.1080/00207179208934308. 
5. ^ Collins Jr. E.G, Haddad W.M., Ying S. (1996). "A homotopy algorithm for reduced-order dynamic compensation using the Hyland-Bernstein optimal projection equations". Journal of Guidance Control & Dynamics 19 (2): 407–417. doi:10.2514/3.21633. 
6. ^ ^{a b} Hyland D.C, Bernstein D.S. (1984). "The optimal projection equations for fixed order dynamic compensation". IEEE Transaction on Automatic Control AC-29 (11): 1034–1037. doi:10.1109/TAC.1984.1103418. 
7. ^ ^{a b} Bernstein D.S., Davis L.D., Hyland D.C. (1986). "The optimal projection equations for reduced-order discrete-time modeling estimation and control". Journal of Guidance Control and Dynamics 9 (3): 288–293. doi:10.2514/3.20105. 
8. ^ ^{a b} Haddad W.M., Tadmor G. (1993). "Reduced-order LQG controllers for linear time-varying plants". Systems & Control Letters 20 (2): 87–97. doi:10.1016/0167-6911(93)90020-7.