Tensor Linear Regression: Degeneracy and Solution

• Published in: IEEE access Vol. 9; pp. 7775 - 7788
• Main Authors: Zhou, Ya, Wong, Raymond K. W., He, Kejun
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Asymptotic methods; Asymptotic properties; Estimation; High-dimensional regression; Linear regression; low-rank modeling; Matrix decomposition; Optimization; Parameter identification; penalized regression; Regression analysis; Tensors
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3049494

Tensor regression is an important and useful tool for analyzing multidimensional array data. To deal with high dimensionality, CANDECOMP/PARAFAC (CP) low-rank constraints are often imposed on the coefficient tensor parameter in the (penalized) loss functions. However, besides the well-known non-identifiability issue of the CP parameters, we demonstrate that the corresponding optimization may not have any attainable solutions, and thus the estimation of the coefficient tensor is not well-defined when this happens. This is closely related to a phenomenon, called CP degeneracy, in low-rank tensor approximation problems. In this article, we show some useful results of CP degeneracy in the context of tensor regression problems. To overcome the theoretical and numerical issues associated with the degeneracy, we provide a general penalized strategy as a solution to the degeneracy. The related results also explain why some of the existing methods are more stable than the others. The asymptotic properties of the resulting estimation are also studied. Numerical experiments are conducted to illustrate our findings.