This work combines the silhouettes and skeletons and refines the framewise joint predictions for gait recognition. With temporal information from the silhouette sequences, we show that the refined skeletons can improve gait recognition performance without extra annotations. We compare our methods on four public datasets, CASIA-B, OUMVLP, Gait3D and GREW, and show state-of-the-art performance.

Our proposed architecture for GaitRef and GaitMix. Trapezoids are trainable modules, and modules of the same color in the same model share the weight. Dashed lines are the operation of feature copying. S and J are the input silhouettes and skeletons. F_S represents silhouette features, while F_J and F_J∗ represent skeleton features from input and refined skeletons, respectively. Architecture of the skeleton correction network. F_J^P is the skeleton features after average pooling. We concatenate the joint position J with its feature F_J along with the global feature after pooling F_J^P and the silhouette feature F_S before sending it into the decoder for calculating the position difference ∆J for each frame. Decoders at different timestamps share weights.

Visualization of successful and failure refined skeletons with GaitRef. For each example, from left to right, we have original skeletons, silhouette of the nearby timestamp and corrected skeletons from skeleton correction network.