LQR - guidosassaroli/controlbasics GitHub Wiki

Linear Quadratic Regulator (LQR) is a feedback control algorithm that utilizes all state variables for optimal control design, where each state variable is multiplied by a gain and summed to generate a single actuation value. It is a stable control method based on specified performance weightings rather than eigenvalue placement.

Here just some hints in order to derive an LQR controller. Consider a linear time-invariant system in state-space form:

$$ \dot{x} = Ax + Bu $$

and the infinite-horizon cost function given by:

$$ J = \int_0^\infty \left[ x^T {\bf Q} x + u^T {\bf R} u \right] dt, \quad {\bf Q} = {\bf Q}^T \geq {\bf 0}, {\bf R} = {\bf R}^T > 0 $$

with $Q$ and $R$ symmetric and positive semi-definite. The goal is to find the optimal cost-to-go function $J^*(x)$ which satisfies the Hamilton–Jacobi–Bellman (HJB) equation:

$$ \forall x, \quad 0 = \min_u \left[ x^T {\bf Q} x + u^T {\bf R} u + \frac{\partial J^*}{\partial x} \left( {\bf A}x + Bu \right) \right] $$

The full process is skipped and it can be found in Chapter 8 of book \cite{LQRBook}.

The main results are that if the pair $(A,B)$ is reachable and the pair $(A,C_q)$ is observable, with $C_q$ being the partition the matrix $Q=C_q ^TC_q$, then:

  1. The optimal control law is given by

$$ u(t) = -\bar K x(t) $$

With

$$ \bar K = R^{-1}B^T \bar P $$

Where $\bar P$ is the unique positive semidefinite solution of the stationary Riccati equation

$$ 0 = {\bf \bar P} {\bf A} + {\bf A}^T {\bf \bar P} - {\bf \bar P} B {\bf R}^{-1} B^T {\bf \bar P} + {\bf Q}. $$

  1. The closed loop system $\dot x(t) = (A-B\bar K)x(t)$ is asymptotically stable.

Example

The example LQR.jpynb demonstrates how to use a Linear Quadratic Regulator (LQR) to stabilize an inverted pendulum mounted on a cart. The system is linearized around the upright position, and the LQR controller is designed to keep the pendulum balanced while minimizing control effort. A linearized state-space model of the system is used and a force applied to the cart is used to move the cart.