Towards robust and domain agnostic reinforcement learning competitions

Reinforcement learning competitions have formed the basis for standard research benchmarks, galvanized advances in the state-of-the-art, and shaped the direction of the field. Despite this, a majority of challenges suffer from the same fundamental problems: participant solutions to the posed challen...

Full description

Saved in:

Bibliographic Details
Main Authors:	Guss, William Hebgen, Milani, Stephanie, Topin, Nicholay, Houghton, Brandon, Mohanty, Sharada, Melnik, Andrew, Harter, Augustin, Buschmaas, Benoit, Jaster, Bjarne, Berganski, Christoph, Heitkamp, Dennis, Henning, Marko, Ritter, Helge, Wu, Chengjie, Hao, Xiaotian, Lu, Yiming, Mao, Hangyu, Mao, Yihuan, Wang, Chao, Opanowicz, Michal, Kanervisto, Anssi, Schraner, Yanick, Scheller, Christian, Zhou, Xiren, Liu, Lu, Nishio, Daichi, Tsuneda, Toi, Ramanauskas, Karolis, Juceviciute, Gabija
Format:	Journal Article
Language:	English
Published:	07-06-2021
Subjects:	Computer Science - Artificial Intelligence Computer Science - Learning Computer Science - Neural and Evolutionary Computing Computer Science - Robotics Statistics - Machine Learning
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Reinforcement learning competitions have formed the basis for standard research benchmarks, galvanized advances in the state-of-the-art, and shaped the direction of the field. Despite this, a majority of challenges suffer from the same fundamental problems: participant solutions to the posed challenge are usually domain-specific, biased to maximally exploit compute resources, and not guaranteed to be reproducible. In this paper, we present a new framework of competition design that promotes the development of algorithms that overcome these barriers. We propose four central mechanisms for achieving this end: submission retraining, domain randomization, desemantization through domain obfuscation, and the limitation of competition compute and environment-sample budget. To demonstrate the efficacy of this design, we proposed, organized, and ran the MineRL 2020 Competition on Sample-Efficient Reinforcement Learning. In this work, we describe the organizational outcomes of the competition and show that the resulting participant submissions are reproducible, non-specific to the competition environment, and sample/resource efficient, despite the difficult competition task.
DOI:	10.48550/arxiv.2106.03748