Title
On RL without Lock in Action
Abstract
Most of the literature in reinforcement learning assumes that chosen actions do not affect the future reward distribution. In these classical settings, the optimal policy is typically deterministic and converges to the action that yields the highest average reward. This assumption holds when the reward distribution of an action remains unaffected by how frequently or recently it has been chosen. In such cases, the policy converges to lock-in probability, meaning the optimal action is selected with 100% certainty.
However, in many real-world scenarios, reward outcomes are influenced by the frequency with which actions are selected. For instance, in resource allocation problems, repeatedly choosing the same resource can lead to its depletion, thereby diminishing the reward perceived by a reinforcement learning agent. In these cases, lock-in probability fails to achieve optimal performance.
In this talk, we examine a reinforcement learning scheme known as linear-reward inaction learning automata, which demonstrates favorable properties under such conditions. We also show its effectiveness in game-theoretic environments with limited information, where players are unaware of the existence or actions of others, yet the method still converges to mixed-strategy Nash equilibria. We also demonstrate its effectiveness in game-theoretic environments with limited information, where players are unaware of the existence or actions of others. In such settings, the proposed method still converges to a mixed-strategy Nash equilibrium, whereas classical schemes tend to lock into pure strategy profiles, potentially missing optimal mixed outcomes.
Bio
Anis Yazidi is an Associate Professor in the Department of Informatics at the University of Oslo (UiO), and an Adjunct Professor at OsloMet in Oslo. His current research interests include applied machine learning, reinforcement learning, stochastic optimization, and autonomous computing. He is currently the Principal Investigator (PI) for several projects, including the Horizon 2020 AI-Mind project and Virtual-Eye, a FRIPRO project.
Speaker is presenting in person.
Practical
The Tuesdays seminars series are devoted to various topics relevant for the Integreat´s research focus. Presenters from the Integreat community and beyond have 40 minutes to present, followed by a group discussion.
Seminars are open for attendance for everybody.
For those unable to attend in person: https://uio.zoom.us/j/62535744013
Point of contacts:
- Organiser and moderator: Ali Ramezani-Kebrya, University of Oslo
- Integreat Administrative Leader Maria Dikova, University of Oslo