About this video Over deze video
The field of reinforcement learning has long been dominated by discounted methods, wherein a decision-making agent aims to optimize a potentially-discounted sum of rewards over time. In this talk, we explore a fundamentally different and under-explored decision-making framework, in which a decision-making agent aims to optimize the reward received per time step. Methods associated with this framework are typically referred to as differential or average-reward methods. In particular, we will show how differential methods have unique structural properties that make it possible to circumvent some of the typical challenges and non-trivialities associated with risk-aware decision-making, in which the agent is tasked with learning and/or optimizing a performance-based measure other than the typical (risk-neutral) mean. In the first half of the talk, we will show how the differential framework admits a more scalable family of distributional RL algorithms compared to discounted methods. In the second half of the talk, we will show how we can leverage the unique structural properties of differential RL to optimize, for the first time, the well-known conditional value-at-risk (CVaR) risk measure in a fully-online manner, without the use of an explicit bi-level optimization scheme or an augmented state-space.
Juan Sebastian Rojas is a PhD student at the University of Toronto, where he conducts research as part of the Dynamic Optimization & Reinforcement Learning Lab. His research interests lie in the theory and application of reinforcement learning agents. His current research focuses on developing theoretical frameworks and algorithms that incorporate the notions of risk and longevity into the learning, planning, and decision-making processes of reinforcement learning agents operating in dynamic, uncertain, and safety-critical environments. Juan has over five years of experience in industry, where he’s contributed to projects that span a broad range of topics, including machine learning, robotics, software engineering, and data science.
This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Rahul Narava and Gusti Winata, Leads of our Reinforcement Learning group for their dedication in organizing this event.
If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker.
Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommunityApp).
