Special Offers see all
More at Powell'sRecently Viewed clear list 
$180.25
New Trade Paper
Ships in 1 to 3 days
available for shipping or prepaid pickup only
Available for Instore Pickup
in 7 to 12 days
Other titles in the Stochastic Modelling and Applied Probability series:
Stochastic Modelling and Applied Probability #35: Stochastic Approximation and Recursive Algorithms and Applicationsby Harold J. Kushner
Synopses & ReviewsPublisher Comments: This revised and expanded second edition presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. There is a complete development of both probability one and weak convergence methods for very general noise processes. The proofs of convergence use the ODE method, the most powerful to date. The assumptions and proof methods are designed to cover the needs of recent applications. The development proceeds from simple to complex problems, allowing the underlying ideas to be more easily understood. Rate of convergence, iterate averaging, highdimensional problems, stabilityODE methods, two time scale, asynchronous and decentralized algorithms, statedependent noise, stability methods for correlated noise, perturbed test function methods, and large deviations methods are covered. Many motivating examples from learning theory, ergodic cost problems for discrete event systems, wireless communications, adaptive control, signal processing, and elsewhere illustrate the applications of the theory.
Synopsis:This book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. This second edition is a thorough revision, although the main features and structure remain unchanged. It contains many additional applications and results as well as more detailed discussion.
Table of Contents Introduction 1 Review of Continuous Time Models 1.1 Martingales and Martingale Inequalities 1.2 Stochastic Integration 1.3 Stochastic Differential Equations: Diffusions 1.4 Reflected Diffusions 1.5 Processes with Jumps 2 Controlled Markov Chains 2.1 Recursive Equations for the Cost 2.2 Optimal Stopping Problems 2.3 Discounted Cost 2.4 Control to a Target Set and Contraction Mappings 2.5 Finite Time Control Problems 3 Dynamic Programming Equations 3.1 Functionals of Uncontrolled Processes 3.2 The Optimal Stopping Problem 3.3 Control Until a Target Set Is Reached 3.4 A Discounted Problem with a Target Set and Reflection 3.5 Average Cost Per Unit Time 4 Markov Chain Approximation Method: Introduction 4.1 Markov Chain Approximation 4.2 Continuous Time Interpolation 4.3 A Markov Chain Interpolation 4.4 A Random Walk Approximation 4.5 A Deterministic Discounted Problem 4.6 Deterministic Relaxed Controls 5 Construction of the Approximating Markov Chains 5.1 One Dimensional Examples 5.2 Numerical Simplifications 5.3 The General Finite Difference Method 5.4 A Direct Construction 5.5 Variable Grids 5.6 Jump Diffusion Processes 5.7 Reflecting Boundaries 5.8 Dynamic Programming Equations 5.9 Controlled and State Dependent Variance 6 Computational Methods for Controlled Markov Chains 6.1 The Problem Formulation 6.2 Classical Iterative Methods 6.3 Error Bounds 6.4 Accelerated Jacobi and GaussSeidel Methods 6.5 Domain Decomposition 6.6 Coarse GridFine Grid Solutions 6.7 A Multigrid Method 6.8 Linear Programming 7 The Ergodic Cost Problem: Formulation and Algorithms 7.1 Formulation of the Control Problem 7.2 A Jacobi Type Iteration 7.3 Approximation in Policy Space 7.4 Numerical Methods 7.5 The Control Problem 7.6 The Interpolated Process 7.7 Computations 7.8 Boundary Costs and Controls 8 Heavy Traffic and Singular Control 8.1 Motivating Examples 8.2 The Heavy Traffic Problem 8.3 Singular Control 9 Weak Convergence and the Characterization of Processes 9.1 Weak Convergence 9.2 Criteria for Tightness in $D^{k}\left [0,\infty \right )$ 9.3 Characterization of Processes 9.4 An Example 9.5 Relaxed Controls 10 Convergence Proofs 10.1 Limit Theorems 10.2 Existence of an Optimal Control 10.3 Approximating the Optimal Control 10.4 The Approximating Markov Chain 10.5 Convergence of the Costs 10.6 Optimal Stopping 11 Convergence for Reflecting Boundaries, Singular Control, and Ergodic Cost Problems 11.1 The Reflecting Boundary Problem 11.2 The Singular Control Problem 11.3 The Ergodic Cost Problem 12 Finite Time Problems and Nonlinear Filtering 12.1 Explicit Approximations: An Example 12.2 General Explicit Approximations 12.3 Implicit Approximations: An Example 12.4 General Implicit Approximations 12.5 Optimal Control Computations 12.6 Solution Methods 12.7 Nonlinear Filtering 13 Controlled Variance and Jumps 13.1 Controlled Variance: Introduction 13.2 Controlled Jumps 14 Problems from the Calculus of Variations: Finite Time Horizon 14.1 Problems with a Continuous Running Cost 14.2 Numerical Schemes and Convergence 14.3 Problems with a Discontinuous Running Cost 15 Problems from the Calculus of Variations: Infinite Time Horizon 15.1 Problems of Interest 15.2 Numerical Schemes for the Case $k(x, \alpha ) \geq k_0 > 0$ 15.3 Numerical Schemes for the Case $k(x, \alpha ) \geq 0$ 15.4 Remarks on Implementation and Examples 16 The Viscosity Solution Approach 16.1 Definitions and Some Properties of Viscosity Solutions 16.2 Numerical Schemes 16.3 Proof of Convergence References Index List of Symbols
What Our Readers Are SayingBe the first to add a comment for a chance to win!Product Details
Related Subjects
Science and Mathematics » Mathematics » Analysis General


