Efficient Hand Gesture Recognition using Multi-Task Multi-Modal Learning and Self-Distillation

We propose a fine-grained basketball shot analysis system to analyze "Shot Type", "Shot Result", and "Ball Status" of each shot event in real-time. Multimedia technologies has been applied to sports recent years. Many sport events utilize cameras to capture the information on the court, and then analyze the sport videos with the aid of computer vision and artificial intelligence technologies to achieve further applications. In the proposed system, given a basketball video captured by a single camera as input, it first detects/segments shot candidates by a customized object detector, and then input the shot candidate into our proposed hoop-centric trajectory-aware network to learn the ball trajectory infomation and the spatio-temporal relation between the ball and the hoop. Compared to the existing methods that analyze basketball shots, our algorithm can better handle videos with arbitrary camera movements (e.g., panning, zooming) and captured angle. We evaluate our system by comparing with the existing basketball shot analysis solutions in both training and real basketball game scenarios. Our system outperforms the other applications in the videos of training and basketball game scenarios with the accuracy of 98.02% and 97.35%, respectively in shot detection, and 99.59% and 100.0%, respectively in shot result. To the best of our knowledge, this work is the first system that can analyze fine-grained shot events accurately in real basketball games with arbitrary camera movements, and can be widely applied to game analysis, player training, and highlight generation.