2010 Poster Sessions : Performance Bounds and Suboptimal Policies for Stochastic Control

Student Name : Yang Wang
Advisor : Stephen Boyd
Research Areas: Information Systems
We develop computational bounds on performance for causal state feedback stochastic control with arbitrary noise distribution, and arbitrary input constraint set. This can be very useful as a comparison to the performance of suboptimal control policies, which we can evaluate using Monte Carlo simulation. Our method is based on relaxing the Bellman equation to an inequality, and looking for functions that satisfy this inequality within a given subspace of functions. We can optimize over our family of bounds by solving a convex optimization problem. Numerical experiments show that the lower bound obtained by our method is often close to the performance achieved by several widely-used suboptimal control policies, which shows that both are nearly optimal. As a by-product, our performance bound yields approximate value functions that can be used as approximate value functions for suboptimal control policies.

Yang Wang is a PhD student in the department of Electrical Engineering. His research focuses on the application of convex optimization methods for the design and analysis of feedback control. Yang received his B.A. and M.Eng. in Electrical Engineering from the University of Cambridge, Magdalene College.