Revisiting Optimism in Deterministic Bandits
Poster D: Tuesday -- 16:00 - 18:00
Lorenzo Croissant, Yurong Chen
Keywords: deterministic bandits, optimism, bandits
We revisit optimism in deterministic bandits, where querying an action reveals its payoff exactly and the central challenge is geometric extrapolation rather than statistical estimation.
Our starting point is a generic optimistic algorithm based on upper confidence envelopes built from prior knowledge of a local one-sided growth condition.
We introduce the envelope dimension, a complexity measure that captures how objective regularity and the quality of this prior information jointly control regret through the evolution of these envelopes.
We compare this metric with classical packing- and zooming-based complexity measures, showing minimax equivalence while also highlighting structured instances in which informative priors yield genuine improvements.
We also show how discretised optimistic methods fit within this perspective and how it can be used to design optimistic first-order algorithms for (locally) concave problems.
As an illustration, we apply the framework to Stackelberg games, showing how regularity of the follower's response map induces structure in the leader's payoff under both bandit and semi-bandit feedback.