Deep learning acceleration at the resource-constrained tactical edge

• Published in: 2023 IEEE International Conference on Big Data (BigData) pp. 3857 - 3862
• Main Authors: Geerhart, Billy, Dasari, Venkat R., Wang, Peng, Rapp, Brian
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15-12-2023
• Subjects: Artificial neural networks; Computational efficiency; Computational modeling; computer vision; deep neural networks; inference acceleration; Libraries; Memory management; model reduction; Partitioning algorithms; PyTorch; quantization; Quantization (signal)
• DOI: 10.1109/BigData59044.2023.10386886

This paper outlines how we modified the torch2trt library which allowed us to build a recursive framework that can quantize previously unsupported PyTorch models. The framework partitions the PyTorch model into supported and unsupported modules, and then rebuilds the PyTorch model by replacing the supported PyTorch modules with faster TensorRT modules. The framework allows us to optimize and deploy more advanced Deep Neural Network algorithms that are not natively supported by torch2trt.