ENTL: Embodied Navigation Trajectory Learner

• Main Authors: Kotar, Klemen, Walsman, Aaron, Mottaghi, Roozbeh
• Format: Journal Article
• Language: English
• Published: 05-04-2023
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Robotics
• DOI: 10.48550/arxiv.2304.02639

We propose Embodied Navigation Trajectory Learner (ENTL), a method for
extracting long sequence representations for embodied navigation. Our approach
unifies world modeling, localization and imitation learning into a single
sequence prediction task. We train our model using vector-quantized predictions
of future states conditioned on current states and actions. ENTL's generic
architecture enables sharing of the spatio-temporal sequence encoder for
multiple challenging embodied tasks. We achieve competitive performance on
navigation tasks using significantly less data than strong baselines while
performing auxiliary tasks such as localization and future frame prediction (a
proxy for world modeling). A key property of our approach is that the model is
pre-trained without any explicit reward signal, which makes the resulting model
generalizable to multiple tasks and environments.