Adaptive Dynamic Programming for Minimal Energy Control with Guaranteed Convergence Rate of Linear Systems

다국어 초록 (Multilingual Abstract)

The traditional linear quadratic optimal control can be summarized as finding the state feedback controller, so that the closed-loop system is stable and the performance index is minimum. And it is well known that the solution of the linear quadratic optimal control problem can be obtained by algebraic Riccati equation (ARE) with the standard assumptions. However, results developed for the raditional linear quadratic optimal control problem cannot be directly applied to solve the problem of minimal energy control with guaranteed convergence rate (MECGCR), because the standard assumptions cannot be satisfied in the MECGCR problem. In this paper, we mainly consider the problem of MECGCR and prove that ARE can be applied to solve the MECGCR problem under some conditions. Furthermore, with the assumption that the system dynamics is unknown, we propose a policy iteration (PI) based adaptive dynamic programming (ADP) algorithm to iteratively solve the ARE using the online information of state and input, without requiring the a priori knowledge of the system matrices. Finally, a numerical example is worked out to show the effectiveness of the proposed approach.

참고문헌 (Reference)

1 B. Zhou, "Truncated predictor feedback stabilization of polynomially unstable linear systems with multiple time-varying input delays" 59 (59): 2157-2163, 2014

2 B. Zhou, "Truncated predictor feedback for periodic linear systems with input delays with applications to the elliptical spacecraft rendezvous" 23 (23): 2238-2250, 2015

3 C. Xia, "Steadystate performance improvement for LQR-based PMSM drives" 33 (33): 10622-10632, 2018

4 H. Wu, "Simultaneous policy update algorithms for learning the solution of linear continuous-time H¥ state feedback control" 222 (222): 472-485, 2013

5 Y. Fu, "Robust adaptive dynamic programming of two-player zero-sum games for continuoustime linear systems" 26 (26): 3314-3319, 2015

6 Y. Jiang, "Robust Adaptive Dynamic Programming. Reinforcement Learning and Approximate Dynamic Programming for Feedback Control" John Wiley & Sons, Inc. 2013

7 P. A. Ioannou, "Robust Adaptive Control" Prentice-Hall, Inc. 1995

8 B. Zhou, "Periodic Lyapunov equation based approaches to the stabilization of continuous-time periodic linear systems" 57 (57): 2139-2146, 2012

9 S. Zuo, "Optimal robust output containment of unknown heterogeneous multiagent system using off-policy reinforcement learning" 48 (48): 3197-3207, 2018

10 Yi Jiang, "Optimal Output Regulation of Linear Discrete-Time Systems With Unknown Dynamics Using Reinforcement Learning" Institute of Electrical and Electronics Engineers (IEEE) 1-10, 2019

11 F. L. Lewis, "Optimal Control" John Wiley & Sons, Inc. 2013

12 S. He, "Online adaptive optimal control for continuous-time Markov jump linear systems using a novel policy iteration algorithm" 9 (9): 1536-1543, 2015

13 D. Kleinman, "On an iterative technique for Riccati equation computations" 13 (13): 114-115, 1968

14 J. Fan, "Modelfree linear discrete-time system H¥ control using inputoutput data" 207-212, 2018

15 T. Kaczorek, "Minimum energy control of positive fractional descriptor continuous-time linear systems" 8 (8): 219-225, 2014

16 T. Kaczorek, "Minimum energy control of fractional positive electrical circuits with bounded inputs" 65 (65): 191-201, 2014

17 J. L. Willems, "Minimum energy and maximum accuracy optimal control of linear stochastic systems" 22 (22): 103-112, 1975

18 A. J. Laub, "Matrix Analysis for Scientists and Engineers" Society for Industrial and Applied Mathematics 2004

19 B. Anderson, "Linear system optimization with prescribed degree of stability" 116 (116): 2083-2087, 1969

20 H. Modares, "Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning" 59 (59): 3051-3056, 2014

21 C. T. Chen, "Linear System Theory and Design" Holt, Rinehart, and Winston 1984

22 W. Gao, "Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems" 29 (29): 2614-2624, 2018

23 W. Gao, "Leader-toformation stability of multiagent systems : an adaptive optimal control approach" 63 (63): 3581-3587, 2018

24 H. Zhang, "Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming" 23 (23): 152-163, 2015

25 H. Li, "Integral reinforcement learning for linear continuous-time zero-sum games with completely unknown dynamics" 11 (11): 706-714, 2014

26 M. Stocks, "Guaranteed convergence rate for linear-quadratic optimal time-varying observers" 1653-1658, 2006

27 H. Sun, "Distributed LQR optimal protocol for leader-following consensus" 49 (49): 3532-3546, 2019

28 Y. Jiang, "Datadriven flotation industrial process operational optimal control based on reinforcement learning" 14 (14): 1974-1989, 2018

29 C. Li, "Data-based optimal control for weakly coupled nonlinear systems using policy iteration" 48 (48): 511-521, 2018

30 Huaguang Zhang, "Data-Based Adaptive Dynamic Programming for a Class of Discrete-Time Systems With Multiple Delays" Institute of Electrical and Electronics Engineers (IEEE) 50 (50): 432-441, 2017

31 Y. Jiang, "Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics" 48 (48): 2699-2704, 2012

32 B. Zhou, "Approximation and monotonicity of the maximal invariant ellipsoid for discrete-time systems by bounded controls" 55 (55): 440-446, 2010

33 I. Mareels, "Adaptive systems" 12 (12): 1-26, 1996

34 K. Zhang, "Adaptive optimal control with guaranteed convergence rate for continuous-time linear systems with completely unknown dynamics" 7 : 11526-11532, 2019

35 D. Vrabie, "Adaptive optimal control for continuous-time linear systems based on policy iteration" 45 (45): 477-484, 2009

36 I. Mareels, "Adaptive Systems: An Introduction" DBLP 1996

37 B. Luo, "Adaptive Q learning for data-based optimal output regulation with experience replay" 48 (48): 3337-3348, 2018

38 G. Tao, "Adaptive Control Design and Analysis" Wiley-IEEE Press 2003

39 B. Zhou, "A parametric Lyapunov equation approach to the design of low gain feedback" 53 (53): 1548-1554, 2008