If you see this, something is wrong

LaTex2Web logo

LaTeX2Web, a web authoring and publishing system

LaTex2Web logo

LaTeX2Web, a web authoring and publishing system

Table of contents

Digest for "Direct Data Driven Control Using Noisy Measurements"

Abstract

This paper presents a novel direct data-driven control framework for solving the linear quadratic regulator (LQR) under disturbances and noisy state measurements. The system dynamics are assumed unknown, and the LQR solution is learned using only a single trajectory of noisy input-output data while bypassing system identification. Our approach guarantees mean-square stability (MSS) and optimal performance by leveraging convex optimization techniques that incorporate noise statistics directly into the controller synthesis. First, we establish a theoretical result showing that the MSS of an uncertain data-driven system implies the MSS of the true closed-loop system. Building on this, we develop a robust stability condition using linear matrix inequalities (LMIs) that yields a stabilizing controller gain from noisy measurements. Finally, we formulate a data-driven LQR problem as a semidefinite program (SDP) that computes an optimal gain, minimizing the steady-state covariance. Extensive simulations on benchmark systems—including a rotary inverted pendulum and an active suspension system—demonstrate the superior robustness and accuracy of our method compared to existing data-driven LQR approaches. The proposed framework offers a practical and theoretically grounded solution for controller design in noise-corrupted environments where system identification is infeasible.

Assumption 1

The system’s noise covariance \( W\) and the measurement’s noise covariance \( V\) are known.

Definition 1 (Mean Square Stability (MSS))

For a given control policy \( u_k=h(x_k)\) , the system (1.a) is called MSS if there exists a positive definite \( \Sigma_{\infty} \in \mathbb{R}^{n \times n}\) such that \( \lim_{k \xrightarrow{}\infty}{\left \lVert \mathbb{E}(x_k x_k^T)-\Sigma_{\infty} \right \rVert}=0\) and \( \lim_{k \xrightarrow{}\infty}{\left \lVert \mathbb{E}(x_k) \right \rVert=0}.\) In this case, the control policy is called admissible.

Assumption 2

The collected data matrix \( D_0 = \begin{bmatrix} U_0^T & Y_0^T \end{bmatrix}^T\) is sufficiently rich (informative), i.e., it has full row rank. That is,

\[ \begin{align} rank(D_0) = m+n. \end{align} \]

(7)

Lemma 1

Consider the system (1.a)-(1.b) under Assumptions 1 and 2. Let the collected data be given by (4). Then, under (8.a), the data-based closed-loop system becomes

\[ \begin{align} A+BK &= (Y_1 - \Upsilon_1 - \Omega_0) G + A \Upsilon_0 G, \end{align} \]

(9.a)

\[ \begin{align} Y_0 G &= I, \end{align} \]

(9.b)

Remark 1

The parameterization in Equation (8.a) follows the same structural framework as presented in [1, 2]. However, it is specifically formulated for input-output data with noisy state measurements, as opposed to input-state data. Furthermore, the closed-loop matrix in Equation (9.a) incorporates two additional terms, namely \( -\Upsilon_1 G \) and \( A \Upsilon_0 G \), which arise due to measurement noise. Notably, the closed-loop matrix also exhibits dependence on the unknown system matrix \( A \). This dependence renders the direct application of the parameterization in Equation (9.a) infeasible for data-driven derivations. In [3], a bound on the lumped noise term \( A \Upsilon_0 - \Upsilon_1 - \Omega_0 \) was proposed to address this issue and enhance robustness.

Assumption 3

The pair \( (A, B)\) is unknown but stabilizable.

Theorem 1 (MSS Preservation)

Consider the system (1.a) under noisy measurements given by (1.b). Let the control policy be given by (2). Let \( \Upsilon_0\) and \(G\) be given as (5.a) and (8.a), respectively. Let Assumption 3 hold. If the uncertain system matrix (14) provides MSS, then the actual closed-loop system matrix \( A+BK\) guarantees MSS as well.

Theorem 2 (Data-driven MSS)

Consider the system (1) and let the data collected from this system be given as (4), and Assumptions 1-3 hold. Let there exists a \( \gamma >0\) for which the data-driven control gain \( K = U_0 G\) with \( G=FY^{-1}\) and \( Y=P^{-1}\) solves the following program

\[ \begin{align} &\begin{bmatrix} -Y & (Y_1 F)^T & F^T \\ * & -Y & 0\\ * & * & -\gamma I \end{bmatrix} \preceq 0. \end{align} \]

(17.a)

\[ \begin{align} ~ & Y_0F=Y, \\ ~ &Tr\left((V+W)^{-1}Y\right)-\gamma n^2\ge 0, \end{align} \]

(17.b)

\[ \begin{align} ~ & \gamma > 0, \\\end{align} \]

(17.c)

Then, the closed-loop system is MSS.

Assumption 4

For the system (1.a) and the cost function (3), the pair \( (A, \sqrt{Q})\) is detectable.

Remark 2

The data-based closed-loop covariance expression in (29) includes an additional term, \( \mathrm{Tr}(G \Sigma_k G^\top)W\) , which is absent in the formulation presented in [1]. This discrepancy arises because [1] neglects the contribution of the term \( \Omega_0 G\) in the parameterization of the closed-loop matrix \( A + BK\) . In contrast, our approach explicitly models \( \Omega_0 G\) as a random variable, allowing us to capture the effect of process noise propagation through the gain matrix \( G\) . Moreover, the presence of measurement noise introduces further terms into the covariance structure. This enhanced, uncertainty-aware covariance formulation contributes to the robustness of the synthesized feedback gain.

Theorem 3 (Data-driven LQR Using Noisy Measurements)

Consider the system (1.a) with closed-loop data-based parameterization (27) and the steady state covariance (28). Let Assumptions 1-4 hold and \( x_0\) be a given initial condition. Then, the optimal state feedback matrix \( K=U_0 G\) that minimizes the LQR cost (22) can be computed through the following SDP

\[ \begin{align} \min_{\beta>0, M, \Sigma, H, S, Z, E, F } ~ {\beta} \\\end{align} \]

(30.a)

\[ \begin{align} \textrm{s.t.} &~ Y_1 (H+ Z) Y_1^\top + Tr(H+Z) (W+V) -\Sigma+W \preceq 0 \end{align} \]

(30.b)

\[ \begin{align} & ~ \begin{bmatrix} E & F\\ * & S \end{bmatrix} \succeq 0, \end{align} \]

(30.c)

\[ \begin{align} & ~ \begin{bmatrix} H & F \\ F^\top & \Sigma \end{bmatrix} \succeq 0, \end{align} \]

(30.d)

\[ \begin{align} & ~ \begin{bmatrix} S & \Sigma \\ * & V \end{bmatrix} \succeq 0, \end{align} \]

(30.e)

\[ \begin{align} & ~ U_0 (Z - E) U_0^\top = 0_{m \times m}, \end{align} \]

(30.f)

\[ \begin{align} & ~ Y_0 F = \Sigma, U_0 M = I_m, Y_0 M = 0_{n \times m}, \end{align} \]

(30.g)

\[ \begin{align} & ~ Tr(Q \Sigma)+Tr(R U_0 H U_0^\top) + Tr(RU_0 E U_0^\top) \leq \beta, \end{align} \]

(30.h)

where \( G=F \Sigma^{-1}\) . Then, the gain \( K=U_0F\Sigma^{-1}\) guarantees MSS of the actual closed-loop system (3).

Remark 3

In the derivation of condition (30.f), we assumed that the relation \( U_0 M = I_m \) holds prior to solving the optimization problem. This assumption facilitates the simplification of the nonlinear term in (34). Consequently, before proceeding with the semidefinite program (30), one may verify the feasibility of the linear constraints \( U_0 M = I_m \) and \( Y_0 M = 0_{n \times m} \). If a solution for \( M \) exists, the assumption used in (34) is justified, and the optimization problem (30) can be solved accordingly. Furthermore, since \( Z = E \) is a trivial solution that satisfies (30.f), one may optionally substitute \( Z \) with \( E \) in both (30.f) and (30.b), potentially simplifying the optimization formulation. This observation allows the overall procedure to be modular: first, validate the feasibility of \( M \), and then proceed to solve the remaining part of the optimization problem.

Remark 4

Throughout this work, it is assumed that the noise covariance matrices \( V \) (measurement noise) and \( W \) (process noise) are known. This assumption is common in the data-driven control literature and is also adopted in related works. While knowledge of these covariances facilitates tractable controller synthesis and stability analysis, estimating \( V \) and \( W \) directly from noisy data remains a challenging task. Developing reliable and data-driven methods for covariance estimation constitutes an important direction for future research.

References

[1] C. De Persis and P. Tesi, “Low-complexity learning of linear quadratic regulators from noisy data,'' Automatica, vol. 128, p. 109548, 2021.

[2] R. Esmzad and H. Modares, “Direct data-driven discounted infinite horizon linear quadratic regulator with robustness guarantees,'' Automatica, vol. 175, p. 112197, 2025.

[3] Benita Nortmann and Thulasi Mylvaganam Direct Data-Driven Control of Linear Time-Varying Systems IEEE Transactions on Automatic Control 2023 68 8 4888-4895