Refinet: A Deep Segmentation Assisted Refinement Network for Salient Object Detection

Journal article, 2019

Deep convolutional neural networks (CNNs) recently have been successfully applied to saliency detection with improved performance on locating salient objects, as comparing to conventional saliency detection by handcrafted features. Unfortunately, due to repeated sub-sampling operations inside CNNs such as pooling and convolution, many CNN-based saliency models fail to maintain fine-grained spatial details and boundary structures of objects. To remedy this issue, this paper proposes a novel end-to-end deep learning-based refinement model named Refinet, which is based on fully convolutional network augmented with segmentation hypotheses. Intermediate saliency maps which are edge-aware are computed from segmentation-based pooling and then cancatenating two streams into a fully convolutional network for effective fusion and refinement, leading to more precise object details and boundaries. In addition, the resolution of feature maps in the proposed Refinet is carefully designed to guarantee sufficient boundary clarity of the refined saliency output. Compared to widely employed dense conditional random field (CRF), Refinet is able to enhance coarse saliency maps generated by existing models with more accurate spatial details, and its effectiveness is demonstrated by experimental results on 7 benchmark datasets.