Ordinal Multi-task Part Segmentation with Recurrent Prior Generation

Yifan Zhao,   Jia Li*,   Yu Zhang,  

State Key Laboratory of Virtual Reality Technology and Systems, Beihang University

Yafei Song,   Yonghong Tian

School of Electronics Engineering and Computer Science, Peking University

Current Status: Early Acess

Semantic object part segmentation is a fundamental task in object understanding and geometric analysis. The clear understanding of part relationships can be of great use to the segmentation process. In this work, we propose a novel Ordinal Multitask Part Segmentation (OMPS) approach which explicitly models the part ordinal relationship to guide the segmentation process in a recurrent manner. Quantitative and qualitative experiments are conducted first to explore the mutual impacts among object parts and then an ordinal part inference algorithm is formulated via experimental observations. Specifically, our framework is mainly composed of two modules, the forward module to segment multiple parts as individual subtasks with prior knowledge, and the recurrent module to generate appropriate part priors with the ordinal inference algorithm. These two modules work iteratively to optimize the segmentation performance and the network parameters. Experimental results show that our approach outperforms the state-of-the-art models on human and vehicle part parsing benchmarks. Comprehensive evaluations are conducted to demonstrate the effectiveness of our approach in object part segmentation.

 Approach

Pipeline of proposed OMPS framework. In the forward module, we adopt a weight-sharing network PartNet to extract image-level features and a task-relevant shallow network PriorNet to encode part-level mask priors. After combining image-level features and prior knowledge, a multitask decoder is constructed to parse each part individually. The recurrent module generates and updates appropriate part priors with an ordinal subtask inference algorithm. These two modules are conducted iteratively to get the optimal results. The arrows between different subtasks indicate the ordinal relationships among subtasks (i.e. the results of the former subtasks serve as prior to the latter).

 Benchmark

Segmentation Results on PASCAL-Car and PASCAL-Aeroplane dataset. Our model shows superior performances, especially in parsing parts with small size and conflicts between classes. DeepLabv3 is the most efficient model with finer results while FCN and DeepLab-LFOV miss many details in this part parsing task

 Citation