Keyframe-Guided 3D Human Mesh Recovery with Pose Token Completion

Recent research and applications such as virtual reality (VR) systems, motion analysis, and physically simulated humanoid training increasingly rely on accurate 3D human mesh recovery from sports videos. However, existing human mesh recovery (HMR) methods involve heavy per-frame computation, which limits their efficiency and scalability.To address this issue, we propose a keyframe-guided 3D human mesh recovery framework that reconstructs full-body motion sequences from a small number of keyframe meshes. Our approach follows the model-based HMR paradigm using parametric representations like SMPL. We leverage a pose VQ-VAE to learn discrete representations of SMPL body models, referred to as pose tokens. Keyframe meshes are obtained using an existing HMR model, and the masked generative transformer predicts the missing pose tokens to recover the complete SMPL sequence.In the monocular setting, we evaluate our framework on the widely used 3DPW benchmark. Experimental results show that our method achieves accuracy comparable to state-of-the-art approaches while providing the fastest end-toend runtime. To further validate its applicability in complex multi-view sports scenarios, we introduce the NTHU-MultiSports dataset. Results demonstrate that our framework achieves substantial speed-up while maintaining low reconstruction error (less than 20 mm per joint).These findings confirm that our keyframe-guided formulation effectively balances accuracy and efficiency, offering a practical solution for large-scale and realworld 3D human motion reconstruction tasks.