Abstract

This monograph provides an exposition of recently developed reinforcement learning-based techniques for decision and control in human-engineered cognitive systems. The developed methods learn the solution to optimal control, zero-sum, non zero-sum, and graphical game problems completely online by using measured data along the system trajectories and have proved stability, optimality, and robustness. It is true that games have been shown to be important in robust control for disturbance rejection, and in coordinating activities among multiple agents in networked teams. We also consider cases with intermittent (an analogous to triggered control) instead of continuous learning and apply those techniques for optimal regulation and optimal tracking. We also introduce a bounded rational model to quantify the cognitive skills of a reinforcement learning agent. In order to do that, we leverage ideas from behavioral psychology to formulate differential games where the interacting learning agents have different intelligence skills, and we introduce an iterative method of optimal responses that determine the policy of an agent in adversarial environments. Finally, we present applications of reinforcement learning to motion planning and collaborative target tracking of bounded rational unmanned aerial vehicles.

Synchronous Reinforcement Learning-Based Control for Cognitive Autonomy

This monograph describes the use of principles of reinforcement learning (RL) to design feedback policies for continuous-time dynamical systems that combine features of adaptive control and optimal control. In a control engineering context, RL bridges the gap between traditional optimal control and adaptive control algorithms.

The authors give an insightful introduction to reinforcement learning techniques that can address various control problems. In this context, they give a detailed description of techniques such as Game-Theoretic Learning, Q-learning, and Intermittent RL; with each chapter providing a self-contained exposition of the topic and giving the reader suggestions for further reading. Finally, the authors demonstrate the application of the techniques in autonomous vehicles.

This review of a topic that is rapidly becoming ubiquitous in many engineering systems enables to reader dip in and out of the topic to quickly understand the essentials and provides the starting point for further research.