Robust Successor Features
Poster C: Tuesday -- 11:00 - 12:30
Erik Nikulski, Yamen Habib, Vicenç Gomez, Anders Jonsson, Rubén Moreno-Bote, Javier Segovia-Aguas
Keywords: Reinforcement Learning, Robust RL, Transfer Learning, Successor Features, Generalization
Generalization in Reinforcement Learning (RL) refers to the ability to execute close-to-optimal policies in unseen tasks after the agent has been trained on a different set of tasks.
Building on the seminal work of the successor representation and further adaptations with function approximation, Transfer in RL has traditionally focused on generalizing to tasks that only differ in the reward function.
A decade after the introduction of the successor representation, Robust RL emerged simultaneously from several articles in the field of operations research.
In Robust RL, the transition kernel is unknown, and the goal is to maximize the expected reward under this uncertainty.
Our work unifies these two paradigms through robust successor features which generalize across both the reward function and the expected transition kernel, under the assumption that tasks are linear Markov Decision Processes.
These features provide similar theoretical guarantees to other methods in Transfer in RL that leverage successor features combined with Generalized Policy Improvement.
Finally, their generalization capabilities are validated on several grid-based benchmarks and compared to previous alternatives that focus solely on either the reward or the transition kernel.