Zero-Shot Transfer in Imitation Learning

• Main Authors: Cauderan, Alvaro, Boeshertz, Gauthier, Schwarb, Florian, Zhang, Calvin
• Format: Journal Article
• Language: English
• Published: 10-10-2023
• Subjects: Computer Science - Learning
• DOI: 10.48550/arxiv.2310.06710

We present an algorithm that learns to imitate expert behavior and can
transfer to previously unseen domains without retraining. Such an algorithm is
extremely relevant in real-world applications such as robotic learning because
1) reward functions are difficult to design, 2) learned policies from one
domain are difficult to deploy in another domain and 3) learning directly in
the real world is either expensive or unfeasible due to security concerns. To
overcome these constraints, we combine recent advances in Deep RL by using an
AnnealedVAE to learn a disentangled state representation and imitate an expert
by learning a single Q-function which avoids adversarial training. We
demonstrate the effectiveness of our method in 3 environments ranging in
difficulty and the type of transfer knowledge required.