Panoptic Segmentation

Panoptic segmentation is a topic that was discussed during our lab seminar recently, because it could potentially improve scene understanding in autonomous vehicles using vision sensors. 

Successful approaches based on convolutional nets have been proposed previously for semantic segmentation task. Further, methods based on object or region proposals have become popular to detect individual objects as well. 

Image source: [1]

The idea behind panoptic segmentation [1] is unifying the tasks of semantic segmentation (studying about 'stuff' such as sky, grass, regions) and instance segmentation using object detectors (studying about countable 'things', E.g., different instances of cars). 

'Panoptic quality (PQ)' metric is proposed as a novel method to evaluate the proposed approach. More details about this can be found here and a simpler version here.  

[1] Panoptic Segmentation, Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, Piotr Dollar