Synopses & Reviews
Using a singular perturbation approach, this is a systematic treatment of those systems that naturally arise in queuing theory, control and optimisation, and manufacturing, gathering a number of ideas which were previously scattered throughout the literature. The book presents results on asymptotic expansions of the corresponding probability distributions, functional occupation measures, exponential upper bounds, and asymptotic normality. To bridge the gap between theory and applications, a large portion of the book is devoted to various applications, thus reducing the dimensionality for problems under Markovian disturbances and providing tools for dealing with large-scale and complex real-world situations. Much of this stems from the authors'recent research, presenting results which have not appeared elsewhere. An important reference for researchers in applied mathematics, probability and stochastic processes, operations research, control theory, and optimisation.
Synopsis
I Prologue and Preliminaries.- 1 Introduction and Overview.- 1.1 Introduction.- 1.2 A Brief Survey.- 1.3 Outline of the Book.- 2 Mathematical Preliminaries.- 2.1 Introduction.- 2.2 Martingales.- 2.3 Markov Chains.- 2.4 Piecewise-Deterministic Processes.- 2.5 Irreducibility and Quasi-Stationary Distributions.- 2.6 Gaussian Processes and Diffusions.- 2.7 Notes.- 3 Markovian Models.- 3.1 Introduction.- 3.2 Birth and Death Processes.- 3.3 Finite-State Space Models.- 3.4 Stochastic Optimization Problems.- 3.5 Linear Systems with Jump Markov Disturbance.- 3.6 Interpretations of Time-Scale Separation.- 3.7 Notes.- II Singularly Perturbed Markov Chains.- 4 Asymptotic Expansion: Irreducible Generators.- 4.1 Introduction.- 4.2 Asymptotic Expansion.- 4.3 Regular Part of the Expansion.- 4.4 Boundary Layer Correction.- 4.5 Asymptotic Validation.- 4.6 Examples.- 4.7 Two-Time-Scale Expansion.- 4.8 Notes.- 5 Asymptotic Normality and Exponential Bounds.- 5.1 Introduction.- 5.2 Occupation Measure.- 5.3 Exponential Bounds.- 5.4 Asymptotic Normality.- 5.5 Extensions.- 5.6 Notes.- 6 Asymptotic Expansion: Weak and Strong Interactions.- 6.1 Introduction.- 6.2 Chains with Recurrent States.- 6.3 Inclusion of Absorbing States.- 6.4 Inclusion of Transient States.- 6.5 Countable-State Space - Part I.- 6.6 Countable-State Space - Part II.- 6.7 Remarks on Singularly Perturbed Diffusions.- 6.8 Notes.- 7 Weak and Strong Interactions: Asymptotic Properties and Ramification.- 7.1 Introduction.- 7.2 Aggregation of Markov Chains.- 7.3 Exponential Bounds.- 7.4 Asymptotic Normality.- 7.5 Measurable Generators.- 7.6 Notes.- III Control and Numerical Methods.- 8 Markov Decision Problems.- 8.1 Introduction.- 8.2 Problem Formulation.- 8.3 Limit Problem.- 8.4 Asymptotic Optimality.- 8.5 Convergence Rate and Error Bound.- 8.6 Long-Run Average Cost.- 8.7 Computational Procedures.- 8.8 Notes.- 9 Stochastic Control of Dynamical Systems.- 9.1 Introduction.- 9.2 Problem Formulation.- 9.3 Properties of the Value Functions.- 9.4 Asymptotic Optimal Controls.- 9.5 Convergence Rate.- 9.6 Weak Convergence Approach.- 9.7 Notes.- 10 Numerical Methods for Control and Optimization.- 10.1 Introduction.- 10.2 Numerical Methods for Optimal Control.- 10.3 Optimization under Threshold Policy.- 10.4 Notes.- A Appendix.- A.1 Properties of Generators.- A.2 Weak Convergence.- A.3 Relaxed Control.- A.4 Viscosity Solutions of HJB Equations.- A.5 Value Functions and Optimal Controls.- A.6 Miscellany.- References.
Synopsis
Using a singular perturbation approach, this book presents a systematic treatment of perturbed systems that naturally arise in queueing theory, control and optimization, and manufacturing systems. The book bridges the gap between theory and applications by coveting various applications in controlled dynamic systems, production planning, and numerical methods for control and optimization.
Synopsis
Using a singular perturbation approach, this is a systematic treatment of those systems that naturally arise in queuing theory, control and optimisation, and manufacturing, gathering a number of ideas which were previously scattered throughout the literature. The book presents results on asymptotic expansions of the corresponding probability distributions, functional occupation measures, exponential upper bounds, and asymptotic normality. To bridge the gap between theory and applications, a large portion of the book is devoted to various applications, thus reducing the dimensionality for problems under Markovian disturbances and providing tools for dealing with large-scale and complex real-world situations. Much of this stems from the authors'recent research, presenting results which have not appeared elsewhere. An important reference for researchers in applied mathematics, probability and stochastic processes, operations research, control theory, and optimisation.
Description
Includes bibliographical references (p. [333]-345) and index.
Table of Contents
Prologue and Preliminaries: Introduction and overview- Mathematical preliminaries. Markovian models.- Singularly perturbed Markov chains: Asymptotic expansion: Irreducible generators. Asymptotic normality and exponential bounds. Asymptotic expansion: Weak and strong interactions. Weak and strong interactions: Asymptotic properties and ramification.- Optimizations and numerical methods: Markov decision problems. Stochastic control of dynamical systems. Numerical methods for control and optimization.