Synopses & Reviews
This book is the first to extend the Markov chain approximation method-typically used for stochastic control and optimal stochastic control problems in continuous time-to stochastic control problems for systems with delays. Numerical methods presented here provide a practical and efficient approach to both the investigation and design of such systems. The book's main focus is on the development of robust and intuitive algorithms that can be conveniently programmed and have good numerical properties.
An excellent resource for graduate students, researchers, and practitioners in stochastic systems and communications, the text is aimed at those whose main interest is in the use of algorithms, as well as those whose primary focus is on the mathematics of stochastic systems with delays. The work may be used as a graduate-level textbook for a special topics course or seminar on numerical methods in stochastic control.
Review
From the reviews: "Overall this is a book entirely devoted to numerics of controlled stochastic systems with delays. In addition to analysis, it contains many numerical and simulation results. The book should be beneficial to both people working in the numerical methods of stochastic controls and people working in various applications who need to use numerical algorithms. It is perhaps the only comprehensive numerical study of controlled diffusions with delays to date...[I]t is conceivable that this book will become a standard reference in the stochastic control literature."
Synopsis
The Markov chain approximation methods are widely used for the numerical solution of nonlinear stochastic control problems in continuous time. This book extends the methods to stochastic systems with delays. The book is the first on the subject and will be of great interest to all those who work with stochastic delay equations and whose main interest is either in the use of the algorithms or in the mathematics. An excellent resource for graduate students, researchers, and practitioners, the work may be used as a graduate-level textbook for a special topics course or seminar on numerical methods in stochastic control.
Table of Contents
Preface Examples and Introduction Weak Convergence and Martingales Stochastic Delay Equations: Models Approximations to the Dynamical Models The Ergodic Cost Problem Markov Chain Approximations: Introduction Markov Chain Approximations: Path and Control Delayed Path and Control Delayed: Continued A Wave Equation Approach References Index Symbol Index