8.5 Linear quadratic regulator (LQR)

Overview of linear quadratic regulator (LQR)

The Linear Quadratic Regulator (LQR) is an optimal control method that designs state feedback controllers by minimizing a quadratic cost function. It gives you a principled way to balance how aggressively you drive states to zero against how much control effort you spend doing it.

LQR shows up across aerospace, robotics, process control, and autonomous systems. The core assumptions are that your system is linear, time-invariant, and fully observable (all states can be measured or estimated).

Definition of LQR

LQR finds the control input that minimizes a quadratic cost function subject to linear system dynamics. The cost function penalizes two things simultaneously: deviations of system states from desired values, and the control effort needed to correct those deviations. The result is an optimal state feedback law where the control input is a linear function of the states.

Applications of LQR in control systems

• Aerospace: Aircraft flight control for stabilizing attitude, altitude, and trajectory
• Robotics: Motion planning, trajectory tracking, and stabilization of manipulators and mobile robots
• Process control: Maintaining operating conditions in chemical plants and manufacturing
• Other domains: Power systems, automotive control, and structural vibration suppression

Mathematical formulation of LQR

The formulation has three pieces: a state-space model of the system, a quadratic cost function encoding your performance goals, and the optimization problem that ties them together.

State-space representation

The system dynamics are described by a set of first-order linear differential equations:

$\dot{x}(t) = Ax(t) + Bu(t)$

• $x(t)$ is the state vector, capturing the internal variables that fully describe the system at time $t$ (e.g., position, velocity)
• $u(t)$ is the control input vector, the external signals you can manipulate (e.g., forces, torques)
• $A$ and $B$ are constant matrices defining how states evolve and how inputs affect the states

Quadratic cost function

The cost function that LQR minimizes is:

$J = \int_{0}^{\infty} \left( x^T(t)\,Q\,x(t) + u^T(t)\,R\,u(t) \right) dt$

• $Q$ is a positive semi-definite matrix that penalizes state deviations. Larger entries in $Q$ mean you care more about driving those states to zero quickly.
• $R$ is a positive definite matrix that penalizes control effort. Larger entries in $R$ mean you want to use less actuator energy.

The ratio between $Q$ and $R$ is what shapes the controller's behavior. You're encoding your design priorities directly into these matrices.

Optimal control problem formulation

The goal: find $u(t)$ that minimizes $J$ subject to the dynamics $\dot{x} = Ax + Bu$. This is a constrained optimization problem, and its solution turns out to be a linear state feedback law of the form $u(t) = -Kx(t)$. The elegance of LQR is that this optimal gain $K$ can be computed directly, without searching over possible controllers.

LQR controller design

Algebraic Riccati equation

The optimal gain comes from solving the algebraic Riccati equation (ARE):

$A^TP + PA - PBR^{-1}B^TP + Q = 0$

Here, $P$ is a symmetric positive definite matrix. Once you solve for $P$, you can compute the optimal gain. Several numerical methods exist for solving the ARE:

• Eigenvector (Potter's) method: Solves an eigenvalue problem involving the Hamiltonian matrix
• Schur method: Exploits the invariant subspace structure of the Hamiltonian matrix
• Newton-Kleinman algorithm: An iterative approach that refines successive approximations of $P$

Optimal state feedback gain

With $P$ in hand, the optimal gain matrix is:

$K = R^{-1}B^TP$

The control law is then:

$u(t) = -Kx(t)$

This means the control input at every instant is just a matrix multiplication of the current state. Each element of $K$ determines how strongly a particular state deviation influences a particular control input.

Closed-loop system stability

Applying $u = -Kx$ transforms the open-loop dynamics into:

$\dot{x}(t) = (A - BK)\,x(t)$

The closed-loop system matrix is $A - BK$. If all eigenvalues of this matrix have negative real parts, the system is asymptotically stable and states converge to zero. A key result: when $Q$ is positive semi-definite, $R$ is positive definite, and the pair $(A, B)$ is stabilizable, the LQR solution always produces a stabilizing controller.

Robustness properties of LQR

LQR controllers come with guaranteed robustness margins. Specifically, continuous-time LQR guarantees:

• Gain margin: at least $\frac{1}{2}$ to $\infty$ (the system remains stable if any single input channel gain is halved or increased without bound)
• Phase margin: at least $\pm 60°$

These are strong guarantees compared to many other controller design methods. The robustness stems from the optimality of the control law with respect to the quadratic cost.

LQR design considerations

Selection of weighting matrices

Choosing $Q$ and $R$ is the primary design decision in LQR. A common starting point is Bryson's rule: set the diagonal entries of $Q$ and $R$ as:

$Q_{ii} = \frac{1}{(\text{maximum acceptable value of } x_i)^2}, \quad R_{jj} = \frac{1}{(\text{maximum acceptable value of } u_j)^2}$

This normalizes each state and input by its allowable range, giving you a reasonable initial design that you then refine through simulation.

Balancing control effort vs state deviation

This is the central trade-off in LQR:

• Larger $Q$ relative to $R$: States converge faster, but control signals are larger. Risk of actuator saturation.
• Larger $R$ relative to $Q$: Control actions are smoother and smaller, but states take longer to reach their targets.

Only the relative scaling between $Q$ and $R$ matters. Multiplying both by the same constant produces the same gain $K$.

Tuning LQR performance

1. Start with Bryson's rule or identity matrices for $Q$ and $R$
2. Simulate the closed-loop response and evaluate settling time, overshoot, steady-state error, and control effort
3. Increase specific diagonal entries of $Q$ for states that need tighter regulation
4. Increase specific diagonal entries of $R$ for inputs that are hitting actuator limits
5. Repeat until the response meets your specifications

Pole placement can also guide initial tuning: choose $Q$ and $R$ so that the closed-loop eigenvalues of $A - BK$ land near desired locations.

Limitations of LQR approach

• Linearity assumption: Real systems have nonlinearities. LQR works well near an operating point but may degrade far from it.
• Full state feedback required: All states must be measured or estimated. If estimation is needed, you typically pair LQR with a Kalman filter (forming an LQG controller).
• No explicit constraint handling: LQR doesn't enforce limits on states or inputs. If constraints matter, you may need model predictive control (MPC) instead.
• Model accuracy dependence: Performance degrades with significant modeling errors, unmodeled dynamics, or large disturbances not captured in the design.
• Actuator saturation and noise: These practical effects aren't accounted for in the standard formulation and can degrade real-world performance.

LQR extensions and variations

Infinite-horizon vs finite-horizon LQR

The standard formulation uses an infinite-horizon cost (integrating from $0$ to $\infty$), which yields a constant gain $K$. For tasks with a defined endpoint, such as trajectory planning over a fixed interval $[0, T]$, finite-horizon LQR is more appropriate.

In the finite-horizon case, the gain $K(t)$ is time-varying. You obtain it by solving the differential Riccati equation backward in time from $t = T$ to $t = 0$. The finite-horizon cost function can also include a terminal cost term $x^T(T)\,S\,x(T)$ that penalizes the final state.

Discrete-time LQR

Most controllers run on digital hardware, so you often need the discrete-time version. The system dynamics become:

$x[k+1] = A_d\,x[k] + B_d\,u[k]$

The cost function sums rather than integrates:

$J = \sum_{k=0}^{\infty} \left( x^T[k]\,Q\,x[k] + u^T[k]\,R\,u[k] \right)$

The design follows the same logic: solve a discrete-time algebraic Riccati equation (DARE) to get $P$, then compute $K$. The DARE has a slightly different form than the continuous-time ARE, but MATLAB's dlqr and Python's scipy.linalg.solve_discrete_are handle it directly.

LQR with state constraints

Standard LQR doesn't enforce constraints. Two common approaches to add them:

• Soft constraints: Add penalty terms to the cost function that grow large when states approach their limits. This keeps the problem tractable but doesn't guarantee hard constraint satisfaction.
• Hard constraints: Use constrained optimization (often leading toward MPC-style formulations). This guarantees constraints are met but is computationally heavier.

LQR with output feedback

When not all states are measurable, you design an observer (state estimator) to reconstruct the missing states from available outputs. The estimated states $\hat{x}(t)$ then replace $x(t)$ in the control law:

$u(t) = -K\hat{x}(t)$

The most common pairing is LQR + Kalman filter, known as LQG (Linear Quadratic Gaussian) control. The separation principle guarantees that you can design the controller and observer independently, and the combined system will be stable, provided the system is both stabilizable and detectable.

Numerical methods for solving LQR

Solving Riccati equation numerically

The ARE is a nonlinear matrix equation, so it requires numerical solution. The main approaches:

• Eigenvector (Potter's) method: Forms the $2n \times 2n$ Hamiltonian matrix and computes its eigendecomposition. The stable eigenspace gives $P$. Complexity: $O(n^3)$.
• Schur method: Uses the real Schur decomposition of the Hamiltonian matrix to extract the stable invariant subspace. Also $O(n^3)$, but often more numerically reliable than the eigenvector method.
• Newton-Kleinman iteration: Starts from a stabilizing initial guess and iteratively solves Lyapunov equations to converge to $P$. Each iteration is $O(n^3)$, and convergence is quadratic near the solution.

MATLAB/Python implementation of LQR

In MATLAB (Control System Toolbox):

</>MATLAB
[K, P, e] = lqr(A, B, Q, R);
% K = optimal gain, P = Riccati solution, e = closed-loop eigenvalues

In Python (SciPy):

</>Python
from scipy.linalg import solve_continuous_are
import numpy as np

P = solve_continuous_are(A, B, Q, R)
K = np.linalg.inv(R) @ B.T @ P

Both environments also support discrete-time LQR (dlqr in MATLAB, solve_discrete_are in SciPy).

Computational complexity of LQR

All standard methods for solving the ARE scale as $O(n^3)$ where $n$ is the number of states. For most control problems (tens to hundreds of states), this is fast. For very large-scale systems (thousands of states), specialized low-rank or structure-exploiting solvers may be needed, especially in real-time or embedded applications where computational resources are limited.

LQR in practical applications

LQR for aircraft control

Aircraft flight control is one of the classic LQR applications. The state vector typically includes position, velocity, orientation angles (roll, pitch, yaw), and angular rates. Control inputs are control surface deflections (ailerons, elevators, rudder) and thrust.

LQR designs autopilots and stability augmentation systems that minimize tracking errors while keeping control surface movements smooth. Practical challenges include handling actuator rate limits, sensor noise, and the fact that aircraft dynamics change significantly with speed, altitude, and weight. Gain scheduling (using different LQR gains at different flight conditions) is a common solution.

LQR in robotics and autonomous systems

In robotic manipulators, LQR controls joint angles and end-effector pose while minimizing energy consumption and tracking error. For mobile robots, LQR handles path following and stability control.

A common approach is to linearize the robot's nonlinear dynamics around a reference trajectory, then apply LQR to the linearized system. This works well when the robot stays close to the reference. For larger deviations, techniques like iterative LQR (iLQR) extend the framework to handle nonlinear dynamics more effectively.