Search Results - "Moskovitz, Ted"

Understanding dual process cognition via the minimum description length principle by Moskovitz, Ted, Miller, Kevin J, Sahani, Maneesh, Botvinick, Matthew M

“…Dual-process theories play a central role in both psychology and neuroscience, figuring prominently in domains ranging from executive control to reward-based…”
Get full text

A State Representation for Diminishing Rewards by Moskovitz, Ted, Hromadka, Samo, Touati, Ahmed, Borsa, Diana, Sahani, Maneesh

“…A common setting in multitask reinforcement learning (RL) demands that an agent rapidly adapt to various stationary reward functions randomly sampled from a…”
Get full text

A First-Occupancy Representation for Reinforcement Learning by Moskovitz, Ted, Wilson, Spencer R, Sahani, Maneesh

“…Both animals and artificial agents benefit from state representations that support rapid transfer of learning across tasks and which enable them to efficiently…”
Get full text

Transfer RL via the Undo Maps Formalism by Gupta, Abhi, Moskovitz, Ted, Alvarez-Melis, David, Pacchiano, Aldo

“…Transferring knowledge across domains is one of the most fundamental problems in machine learning, but doing so effectively in the context of reinforcement…”
Get full text

A Unified Theory of Dual-Process Control by Moskovitz, Ted, Miller, Kevin, Sahani, Maneesh, Botvinick, Matthew M

“…Dual-process theories play a central role in both psychology and neuroscience, figuring prominently in fields ranging from executive control to reward-based…”
Get full text

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation by Singh, Aaditya K, Moskovitz, Ted, Hill, Felix, Chan, Stephanie C. Y, Saxe, Andrew M

“…In-context learning is a powerful emergent ability in transformer models. Prior work in mechanistic interpretability has identified a circuit element that may…”
Get full text

Minimum Description Length Control by Moskovitz, Ted, Kao, Ta-Chu, Sahani, Maneesh, Botvinick, Matthew M

“…We propose a novel framework for multitask reinforcement learning based on the minimum description length (MDL) principle. In this approach, which we term…”
Get full text

The Transient Nature of Emergent In-Context Learning in Transformers by Singh, Aaditya K, Chan, Stephanie C. Y, Moskovitz, Ted, Grant, Erin, Saxe, Andrew M, Hill, Felix

“…Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has…”
Get full text

Towards an Understanding of Default Policies in Multitask Policy Optimization by Moskovitz, Ted, Arbel, Michael, Parker-Holder, Jack, Pacchiano, Aldo

“…Much of the recent success of deep reinforcement learning has been driven by regularized policy optimization (RPO) algorithms with strong performance across…”
Get full text

ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPs by Moskovitz, Ted, O'Donoghue, Brendan, Veeriah, Vivek, Flennerhag, Sebastian, Singh, Satinder, Zahavy, Tom

“…In recent years, Reinforcement Learning (RL) has been applied to real-world problems with increasing success. Such applications often require to put…”
Get full text

Confronting Reward Model Overoptimization with Constrained RLHF by Moskovitz, Ted, Singh, Aaditya K, Strouse, DJ, Sandholm, Tuomas, Salakhutdinov, Ruslan, Dragan, Anca D, McAleer, Stephen

“…Large language models are typically aligned with human preferences by optimizing $\textit{reward models}$ (RMs) fitted to human feedback. However, human…”
Get full text

Efficient Wasserstein Natural Gradients for Reinforcement Learning by Moskovitz, Ted, Arbel, Michael, Huszar, Ferenc, Gretton, Arthur

“…A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL). The procedure…”
Get full text

Tactical Optimism and Pessimism for Deep Reinforcement Learning by Moskovitz, Ted, Parker-Holder, Jack, Pacchiano, Aldo, Arbel, Michael, Jordan, Michael I

“…In recent years, deep off-policy actor-critic algorithms have become a dominant approach to reinforcement learning for continuous control. One of the primary…”
Get full text

First-Order Preconditioning via Hypergradient Descent by Moskovitz, Ted, Wang, Rui, Lan, Janice, Kapoor, Sanyam, Miconi, Thomas, Yosinski, Jason, Rawal, Aditya

“…Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter…”
Get full text