Deep learning-based 2D keypoint detection in alpine ski racing – A performance analysis of state-of-the-art algorithms applied to regular skiing and injury situations

• Published in: JSAMS plus Vol. 2; p. 100034
• Main Authors: Zwölfer, Michael, Heinrich, Dieter, Schindelwig, Kurt, Wandt, Bastian, Rhodin, Helge, Spörri, Jörg, Nachbauer, Werner
• Format: Journal Article
• Language: English
• Published: Elsevier 2023
• Subjects: ACL injuries; Alpine skiing; Athletes; Deep learning; Human pose estimation; Deep learning; ACL injuries; Alpine skiing; Human pose estimation; Athletes
• ISSN: 2772-6967; 2772-6967
• DOI: 10.1016/j.jsampl.2023.100034

Objectives
In this study, we examined the practicability of deep learning-based 2D keypoint detection applied to regular skiing and injury situations (i.e., out-of-balance situations and fall situations) on an alpine ski racing track.
Methods
We therefore created a regular skiing- and injury situation-specific dataset (hereinafter called "Injury Ski Dataset"), on which the state-of-the-art keypoint detection algorithms OpenPose, Mask-R-CNN, AlphaPose and DCPose were compared. The performance of each keypoint detector was evaluated by calculating the mean per joint position error (MPJPE) and the percentage of correct keypoints (PCK). Failure cases and common error patterns were further investigated by a visual analysis.
Results
We observed the best results for regular skiing, with 81%–92% of all keypoints detected correctly at an MPJPE of 9 (2) to 14 (3) pixels. In injury situations, self-occlusions and rare poses became more likely, similar to occlusions due to snow spray and motion blur. As a result, the performance in out-of-balance situations decreased to 68%–80% (PCK), while in fall situations, only 35%–54% of all keypoints were detected correctly, with mean errors of 26–36 pixels. Among all algorithms, AlphaPose was the most robust and achieved the best results.
Conclusions
PCK and MPJPE for regular skiing were in the range of manual annotation errors and can be considered low enough for further biomechanical analysis. For fall situations, keypoint detection should be further improved. Regarding the development of a deep learning tool for injury analysis in alpine skiing in the future, we propose to fine-tune a well-performing keypoint detector, such as AlphaPose, on a ski- and injury-specific dataset, such as ours.