Federated Reconstruction: Partially Local Federated Learning

• Main Authors: Singhal, Karan, Sidahmed, Hakim, Garrett, Zachary, Wu, Shanshan, Rush, Keith, Prakash, Sushant
• Format: Journal Article
• Language: English
• Published: 05-02-2021
• Subjects: Computer Science - Distributed, Parallel, and Cluster Computing; Computer Science - Learning
• DOI: 10.48550/arxiv.2102.03448

Personalization methods in federated learning aim to balance the benefits of
federated and local training for data availability, communication cost, and
robustness to client heterogeneity. Approaches that require clients to
communicate all model parameters can be undesirable due to privacy and
communication constraints. Other approaches require always-available or
stateful clients, impractical in large-scale cross-device settings. We
introduce Federated Reconstruction, the first model-agnostic framework for
partially local federated learning suitable for training and inference at
scale. We motivate the framework via a connection to model-agnostic meta
learning, empirically demonstrate its performance over existing approaches for
collaborative filtering and next word prediction, and release an open-source
library for evaluating approaches in this setting. We also describe the
successful deployment of this approach at scale for federated collaborative
filtering in a mobile keyboard application.