Reinforcement Learning with Efficient Active Feature Acquisition

Reinforcement Learning with Efficient Active Feature Acquisition

Abstract

Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem. To be successful, an agent needs to efficiently gather valuable information about the state of the world for making rewarding decisions. However, in real-life, acquiring valuable information is often highly costly, e.g., in the medical domain, information acquisition might correspond to performing a medical test on a patient. This poses a significant challenge for the agent to perform optimally for the task while reducing the cost for information acquisition. In this paper, we propose a model-based reinforcement learning framework that learns an active feature acquisition policy to solve the exploration-exploitation problem during its execution. Key to the success is a novel sequential variational auto-encoder that learns high-quality representations from partially observed states, which are then used by the policy to maximize the task reward in a cost efficient manner. We demonstrate the efficacy of our proposed framework in a control domain as well as using a medical simulator. In both tasks, our proposed method outperforms conventional baselines and results in policies with greater cost efficiency.

Grafik Top
Authors
  • Yin, Haiyan
  • Li, Yingzhen
  • Pan, Sinno Jialin
  • Zhang, Cheng
  • Tschiatschek, Sebastian
Grafik Top
Shortfacts
Category
Technical Report (Working Paper)
Divisions
Data Mining and Machine Learning
Publisher
CoRR arXiv
Date
2 November 2020
Export
Grafik Top