Thermally Constrained Codesign of Heterogeneous 3-D Integration of Compute-in-Memory, Digital ML Accelerator, and RISC-V Cores for Mixed ML and Non-ML Workloads

• Published in: IEEE transactions on very large scale integration (VLSI) systems Vol. 32; no. 9; pp. 1718 - 1725
• Main Authors: Luo, Yuan-Chun, Lu, Anni, Sharda, Janak, Scherer, Moritz, Tomas Gomez, Jorge, Shakib Sarwar, Syed, Li, Ziyun, Frederick Pinkham, Reid, De Salvo, Barbara, Yu, Shimeng
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-09-2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Augmented reality; Augmented reality/virtual reality; Co-design; compute-in-memory (CIM); Convolution; Cooling; Form factors; Hardware; Heating systems; heterogenous 3-D integration; In-memory computing; Machine learning; machine learning (ML); Pipelines; Production costs; RISC-V; Runtime; thermally constrained framework; Virtual reality; Workload
• ISSN: 1063-8210; 1557-9999
• DOI: 10.1109/TVLSI.2024.3415481

Heterogeneous 3-D (H3D) integration not only reduces the chip form factor and fabrication cost but also allows the merging of diverse compute paradigms that suit different applications. This is especially attractive when modern algorithms, such as the augmented reality/virtual reality (AR/VR) workloads, consist of mixed machine learning (ML) and non-ML workloads. To date, codesign that considers the thermal, latency, and power constraints of H3D hardware is largely unexplored. In this work, a thermally aware framework for H3D hardware design is developed to evaluate the thermal, latency, and power trade-offs for a heterogeneous system with compute-in-memory (CIM), digital ML cores, and RISC-V cores. The framework solves for runtime tunable operating points described as the optimal speedup factor, the number of activated RISC-V cores, the cooling coefficient, and the activity rate based on user-defined criteria, achieving up to 135 TOPS and 215 TOPS/W under <inline-formula> <tex-math notation="LaTeX">74~^{\circ } </tex-math></inline-formula>C for the AR/VR workloads.