Abstract: In hierarchical reinforcement learning (HRL), a decision process is decomposed into subproblems, while a semi-Markov decision process (SMDP) chooses among subproblems. Researchers have shown empirically that HRL improves aspects such as exploration, sample efficiency, credit assignment, long-term planning and generalization. This talk discusses several design choices related to HRL, and presents two novel results. The first result is to prove theoretically that HRL is more sample efficient than standard reinforcement learning when the subproblem policies and SMDP policy are learned in parallel. The second result is to develop a theory that enables HRL to compute the globally optimal policy in a sample efficient manner.

Bio: Anders Jonsson is a full professor in the Department of Engineering, where he heads the Artificial Intelligence and Machine Learning research group. His research interests involve sequential decision problems in which one or several agents have to make repeated decisions about what to do, mainly in the form of automated planning and reinforcement learning.