From Pose to Part: Weakly-Supervised Pose Evolution for Human Part Segmentation

Yifan Zhao,   Jia Li*,   Yu Zhang   Xiaowu Chen

State Key Laboratory of Virtual Reality Technology and Systems, Beihang University

Human part segmentation is a crucial but challenging task in computer vision. Recent researches have achieved progress with the pixel-wise annotations. However, annotating per-pixel masks especially at part level is a tedious and labor-intensive procedure. To overcome this problem, we propose a part evolution framework to learn deep potentials from weak pose annotations, which are much easier to collect. Our framework is composed of two essential modules. The first part adaptation module is designed to learn the deep prior knowledge from three related tasks, i.e., pose estimation, part-level and object-level segmentation. The second module is the part evolution module, which refines the part priors from deep potentials with the boundary-aware optimization algorithm. These two modules are conducted iteratively to evolve pose keypoint annotations into reliable part priors. Experimental evidence shows that our weakly-supervised methods generate comparable results with the state-of-the-art strongly supervised methods on the public PASCAL-Person benchmark, and also validates the potential of notable improvements when combining weak labels with existing part segmentation masks.

 Motivation

Evolution of the proposed weakly-supervised part priors. a): pose annotation of training data. b): generated skeleton prior from pose. c): the initial prediction of PADNet. d): boundary-optimized prior of c). e): the iterative prediction supervised by d). f): ground truth mask.

 Approach

The proposed weakly supervised learning framework is mainly composed of the task adaptation module and a prior evolution module. The task adaptation module is to generate deep prediction with a joint multi-task adaptation framework, which regularizes the part features by additional pose and object-level supervision. The Prior evolution module is to refine the pseudo part priors, which relies on the deep predicted potential of PADNet. These two modules work iteratively to get refined part masks. The red dotted line indicates the update process of part priors.

 Citation

  • From Pose to Part: Weakly-Supervised Pose Evolution for Human Part Segmentation