Reinforcement Learning State Estimator

@article{citeulike:2880951, abstract = {In this study, we propose a novel use of reinforcement learning for estimating hidden variables and parameters of nonlinear dynamical systems. A critical issue in hidden-state estimation is that we cannot directly observe estimation errors. However, by defining errors of observable variables as a delayed penalty, we can apply a reinforcement learning framework to state estimation problems. Specifically, we derive a method to construct a nonlinear state estimator by finding an appropriate feedback input gain using the policy gradient method. We tested the proposed method on single pendulum dynamics and show that the joint angle variable could be successfully estimated by observing only the angular velocity, and vice versa. In addition, we show that we could acquire a state estimator for the pendulum swing-up task in which a swing-up controller is also acquired by reinforcement learning simultaneously. Furthermore, we demonstrate that it is possible to estimate the dynamics of the pendulum itself while the hidden variables are estimated in the pendulum swing-up task. Application of the proposed method to a two-linked biped model is also presented.}, author = {Morimoto, Jun and Doya, Kenji }, citeulike-article-id = {2880951}, journal = {Neural Comp.}, keywords = {machine\_learning, theory}, month = {March}, number = {3}, pages = {730--756}, posted-at = {2008-06-10 22:14:58}, priority = {3}, title = {Reinforcement Learning State Estimator}, url = {http://neco.mitpress.org/cgi/content/abstract/19/3/730}, volume = {19}, year = {2007} }

TY - JOUR ID - citeulike:2880951 L3 - citeulike-article-id:2880951 TI - Reinforcement Learning State Estimator JF - Neural Comp. VL - 19 IS - 3 SP - 730 EP - 756 N2 - In this study, we propose a novel use of reinforcement learning for estimating hidden variables and parameters of nonlinear dynamical systems. A critical issue in hidden-state estimation is that we cannot directly observe estimation errors. However, by defining errors of observable variables as a delayed penalty, we can apply a reinforcement learning framework to state estimation problems. Specifically, we derive a method to construct a nonlinear state estimator by finding an appropriate feedback input gain using the policy gradient method. We tested the proposed method on single pendulum dynamics and show that the joint angle variable could be successfully estimated by observing only the angular velocity, and vice versa. In addition, we show that we could acquire a state estimator for the pendulum swing-up task in which a swing-up controller is also acquired by reinforcement learning simultaneously. Furthermore, we demonstrate that it is possible to estimate the dynamics of the pendulum itself while the hidden variables are estimated in the pendulum swing-up task. Application of the proposed method to a two-linked biped model is also presented. KW - machine_learning KW - theory AU - Morimoto, Jun AU - Doya, Kenji PY - 2007/03/01/ UR - http://neco.mitpress.org/cgi/content/abstract/19/3/730 ER -