Combinatorial sequential decision-making problems are typically modeled as mixed‐integer linear programs (MILPs) and solved via branch‐and‐bound (BnB) algorithms. The inherent difficulty of modeling MILPs that accurately represent stochastic real-world problems leads to suboptimal performance in the real world. Recently, machine‐learning methods have been applied to learn MILP models for decision quality rather than how accurately they model the real-world problem. However, these approaches typically rely on supervised learning, assume access to optimal decisions, and use surrogates for the MILP gradients. In this work, we introduce a proof‐of‐concept CORL framework that end-to-end fine‐tunes an MILP scheme using reinforcement learning (RL) on real‐world data to maximize its operational performance. We enable this by casting an MILP solved by BnB as a differentiable stochastic policy compatible with RL. We validate the CORL method in two illustrative combinatorial sequential decision-making examples.