Reinforcement learning (RL) is an exciting idea as well as a remarkable success story worth sharing. However, RL builds on rather complex interactions between different objects that play out over several cycles and are often best explained with an easily accessible implementation. We present RLLBC-Lib a carefully crafted code library with the goal of lowering the entry barrier for students and other learners of RL in the context of learning-based control. At its heart, RLLBC-Lib comprises a comprehensive libary of tabular RL approaches to enforce a clear understanding of the theoretical foundations, as well as a deep RL library following the same design principles to underscore the parallels between simple tabular and state-of-the-art deep RL approaches. Additionally, RLLBC-Lib provides a collection of implementations illustrating core RL principles and contrasting RL to other learning-based control approaches. Finally, RLLBC-Lib provides an ideal basis for creating programming assignments with automated grading.