Real-Time Edge Classification: Optimal Offloading under Token Bucket Constraints

• Published in: 2021 IEEE/ACM Symposium on Edge Computing (SEC) pp. 41 - 54
• Main Authors: Chakrabarti, Ayan, Guerin, Roch, Lu, Chenyang, Liu, Jiangnan
• Format: Conference Proceeding
• Language: English
• Published: ACM 01-12-2021
• Subjects: Cameras; Computational modeling; Correlation; edge computing; Machine learning; Markov Decision Process; Markov processes; Measurement uncertainty; Real-time classification; Switches; token bucket
• DOI: 10.1145/3453142.3492329

We consider an edge-computing setting where machine learning-based algorithms are used for real-time classification of inputs acquired by devices, e.g., cameras. Computational resources on the devices are constrained, and therefore only capable of running machine learning models of limited accuracy. A subset of inputs can be offloaded to the edge for processing by a more accurate but resource-intensive machine learning model. Both models process inputs with low-latency, but offloading incurs network delays. To manage these delays and meet application deadlines, a token bucket constrains transmissions from the device. We introduce a Markov Decision Process-based framework to make offload decisions under such constraints. Decisions are based on the local model's confidence and the token bucket state, with the goal of minimizing a specified error measure for the application. We extend the approach to configurations involving multiple devices connected to the same access switch to realize the benefits of a shared token bucket. We evaluate and analyze the policies derived using our framework on the standard ImageNet image classification benchmark.