A central problem in reinforcement learning is training agents that generalize to variations in their task and/or environment, formally defined as changes to the reward and transition kernel in an MDP. What sets this problem apart from standard generalization in supervised learning is that an agent can explore a changed environment and adapt, thereby generalizing at the cost of longer test-time performance. In this talk I will first formally describe this tradeoff under the framework of Bayesian RL. I will then discuss several recent works in RL and imitation learning that achieve strong generalization capabilities by exploiting the power of test-time adaptation.

While Deep RL tackle long-horizon tasks by representation learning, most emergent latent spaces are not interpretable. In this talk, we will introduce two methods in deep reinforcement learning. STAR is a deep hierarchical reinforcement learning algorithm for learning sequences of actions that uses reachability analysis to devise a latent space grounded in the environment both useful for more efficient learning by reinforcement learning, and interpretable for interaction with humans. Further than sequences of actions, we explore complex activities with a richer temporal relationship between primitives. We also use deep reinforcement learning as a framework to learn temporal rules describing long-horizon activities with algorithm AlphaRule. These neurosymbolic methods combine trial-and-error paradigm in continuous settings with formal verification to devise an interpretable representation for human verification.