Hyperbolic Deep Neural Networks: A Survey

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 44; no. 12; pp. 10023 - 10044
• Main Authors: Peng, Wei, Varanka, Tuomas, Mostafa, Abdelrahman, Shi, Henglin, Zhao, Guoying
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-12-2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Computational modeling; Computer vision; Deep learning; Euclidean geometry; Geometry; Hyperbolic coordinates; hyperbolic neural networks; Literature reviews; Lorentz model; Manifolds; Mathematical models; Natural language processing; Neural networks on Riemannian manifold; Numerical models; Poincaré model; Task analysis
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2021.3136921

Recently, hyperbolic deep neural networks (HDNNs) have been gaining momentum as the deep representations in the hyperbolic space provide high fidelity embeddings with few dimensions, especially for data possessing hierarchical structure. Such a hyperbolic neural architecture is quickly extended to different scientific fields, including natural language processing, single-cell RNA-sequence analysis, graph embedding, financial analysis, and computer vision. The promising results demonstrate its superior capability, significant compactness of the model, and a substantially better physical interpretability than its counterpart in the euclidean space. To stimulate future research, this paper presents a comprehensive review of the literature around the neural components in the construction of HDNN, as well as the generalization of the leading deep approaches to the hyperbolic space. It also presents current applications of various tasks, together with insightful observations and identifying open questions and promising future directions.