Semantic Segmentation

network VOC12 VOC12 with COCO Pascal Context CamVid Cityscapes ADE20K Published In
FCN-8s 62.2 37.8 65.3 CVPR 2015
DeepLab 71.6 ICLR 2015
CRF-RNN 72.0 74.7 39.3 ICCV 2015
DeconvNet 72.5 ICCV 2015
DPN 74.1 77.5 ICCV 2015
SegNet 50.2
Dilation8 75.3
Deeplab v2 79.7 45.7 70.4 PAMI
FRRN B 71.8 CVPR 2017
G-FRNet 79.3 68.0 CVPR 2017
GCN 82.2 76.9 CVPR 2017
SegModel 82.5 79.2 CVPR 2017
RefineNet 83.4 47.3 73.6 40.7 CVPR 2017
PSPNet 82.6 85.4 80.2 CVPR 2017
DIS 86.8 ICCV 2017
SAC-multiple 78.1 44.3 ICCV 2017
DeepLabv3 85.7 81.3 arxiv 1706.05587
DUC-HDC 80.1 WACV2018
DDSC 81.2 47.8 70.9 CVPR 2018
EncNet 82.9 85.9 51.7 44.65 CVPR 2018
DFN 82.7 86.2 80.3 CVPR 2018
DenseASPP 80.6 CVPR 2018
UperNet 42.66 ECCV 2018
PSANet 85.7 80.1 43.77 ECCV 2018
DeepLabv3+ 87.8 82.1 ECCV 2018
ExFuse 87.9 ECCV 2018
OCNet 81.2(81.7) 45.08(45.45) arxiv 1809.00916
DAN 52.6 78.2 CVPR 2019
DPC 87.9 82.7 NIPS 2018
CCNet 81.4 45.22 arxiv 1811.11721
GloRe 80.9 CVPR 2019
TKCN 83.2 79.5 ICME 2019
GCU 44.81 NIPS 2018
DUpsampling 85.3 88.1 52.5 CVPR 2019
FastFCN 53.1 44.34 arxiv 1903.11816
GFF 82.3 45.33 arxiv 1904.01803
HRNetV2 54.0 81.6 arxiv 1904.04514
CaseNet 81.9 45.28 arxiv 1904.08170
LDN 83.6 78.1 80.6 arxiv 1904.08170
FDNet 84.2
GSCNN 82.8

Semantic Segmentation论文整理


  • [NYU2] [ECCV2012] Indoor segmentation and support inference from rgbd images
  • [SUN RGB-D] [CVPR2015] SUN RGB-D: A RGB-D scene understanding benchmark suite shuran
  • [Matterport3D] Matterport3D: Learning from RGB-D Data in Indoor Environments [Paper]

2D Semantic Segmentation


  • Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation[1901.02985]
  • Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation [1903.02120]
  • [CVPR 2019] Structured Knowledge Distillation for Semantic Segmentation [1903.04197]
  • [CVPR 2019] Knowledge Adaptation for Efficient Semantic Segmentation [1903.04688]
  • [CVPR 2019] A Cross-Season Correspondence Dataset for Robust Semantic Segmentation [1903.06916]
  • Efficient Smoothing of Dilated Convolutions for Image Segmentation [1903.07992] [Code]
  • [FastFCN] FastFCN:Rethinking Dilated Convolution in the Backbone for Semantic Segmentation [1903.11816] [Code]
  • [GFF] GFF: Gated Fully Fusion for Semantic Segmentation [1904.01803]
  • DADA: Depth-aware Domain Adaptation in Semantic Segmentation [1904.01886]
  • [HRNetV2] High-Resolution Representations for Labeling Pixels and Regions [1904.04514]
  • SparseMask: Differentiable Connectivity Learning for Dense Image Predictions for Image Segmentation [1904.07642] [Code]
  • CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing [1904.08170]
  • Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More [1904.10167]
  • [CVPR 2019] Bidirectional Learning for Domain Adaptation of Semantic Segmentation [1904.10620]
  • [CVPR 2019] Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation [1904.11693]
  • Efficient Ladder-style DenseNets for Semantic Segmentation of Large Images [1905.05661]
  • [AAAI 2019] [FDNet] Learning Fully Dense Neural Networks for Image Semantic Segmentation [1905.08929 ]
  • Spatial Sampling Network for Fast Scene Understanding [1905.09033] [CVPR2019 Workshop on Autonomous Driving]
  • Zero-Shot Semantic Segmentation [1906.00817]
  • RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation [1907.00135]
  • [ICCV 2019] Gated-SCNN: Gated Shape CNNs for Semantic Segmentation [1907.05740] [project] [Towaki Takikawa] [code]
  • Adaptive Context Encoding Module for Semantic Segmentation [1907.06082]
  • Improving Semantic Segmentation via Dilated Affinity [1907.07011]
  • Context-Integrated and Feature-Refined Network for Lightweight Urban Scene Parsing [1907.11474]
  • Interlaced Sparse Self-Attention for Semantic Segmentation [1907.12273]
  • Consensus Feature Network for Scene Parsing [1907.12411]
  • [ICCV 2019] Expectation-Maximization Attention Networks for Semantic Segmentation [1907.13426] [project]
  • I Bet You Are Wrong: Gambling Adversarial Networks for Structured Semantic Segmentation [1908.02711]
  • Benchmarking the Robustness of Semantic Segmentation Models [1908.05005]

CVPR 2019


  • Panoptic Segmentation [Paper]
  • [DeepLabv3+] [ECCV 2018] Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [Paper] [Code]
  • [EncNet] [CVPR 2018] Context Encoding for Semantic Segmentation [Paper] [Code] (Leverages global context to increase accuracy by adding a channel attention module, which triggers attention on certain feature maps based on a newly designed loss function. The loss is based on a network branch which predicts which classes are present in the image (i.e higher level global context))
  • [ECCV 2018] Adaptive Affinity Fields for Semantic Segmentation [Project] [Paper] [Code]
  • [EXFuse] [ECCV 2018] ExFuse: Enhancing Feature Fusion for Semantic Segmentation [Paper] (Uses deep supervision and explicitly combines the multi-scale features from the feature extraction frontend before processing, in order to ensure multi-scale information is processed together at all levels)
  • Vortex Pooling: Improving Context Representation in Semantic Segmentation [Paper]
  • [DFN] [CVPR 2018] Learning a Discriminative Feature Network for Semantic Segmentation [Paper] (Uses deep supervision and attempts to process the smooth and edge portions of the segments separately)
  • Stacked U-Nets: A No-Frills Approach to Natural Image Segmentation [Paper]
  • [BMVC 2018] Pyramid Attention Network for Semantic Segmentation [Paper]
  • [G-FRNet] [CVPR 2017] Gated Feedback Refinement Network for Coarse-to-Fine Dense Semantic Image Labeling [Paper] [code]
  • [CVPR 2018] Context Contrasted Feature and Gated Multi-Scale Aggregation for Scene Segmentation [Paper]
  • [DenseASPP] [CVPR 2018] DenseASPP for Semantic Segmentation in Street Scenes [Paper] [code] (Combines dense connections with atrous convolutions)
  • [CVPR 2018] Dense Decoder Shortcut Connections for Single-Pass Semantic Segmentation [Paper] (Use dense connections in the decoding stage for higher accuracy (previously only done during feature extraction / encoding))
  • Smoothed Dilated Convolutions for Improved Dense Prediction [Paper]
  • [PSANet] [ECCV 2018] PSANet: Point-wise Spatial Attention Network for Scene Parsing [Paper] [project] [code] [slide] (Attention Mechanism)
  • [OCNet] OCNet: Object Context Network for Scene Parsing [Paper] [code] (Attention Mechanism)
  • [DAN] [CVPR 2019] Dual Attention Network for Scene Segmentation [Paper] [code] (Attention Mechanism)
  • [CCNet] CCNet: Criss-Cross Attention for Semantic Segmentation [Paper] [code] (Attention Mechanism)
  • [GloRe] [CVPR 2019] Graph-Based Global Reasoning Networks [Paper] (Graph Convolution)
  • [TKCN] Tree-structured Kronecker Convolutional Networks for Semantic Segmentation [Paper] [code]
  • [GCU] Beyond Grids: Learning Graph Representations for Visual Recognition [Paper] (Graph Convolution)


  • [PixelNet] PixelNet: Representation of the pixels, by the pixels, and for the pixels [Project] [Code-Caffe] [Paper]
  • [DUC-HDC] [WACV 2018]Understanding Convolution for Semantic Segmentation [Model-Mxnet] [Paper] [Code]
  • [GCN] [CVPR2017] Large Kernel Matters - Improve Semantic Segmentation by Global Convolutional Network [Paper]
  • [CVPR 2017] Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade-2017 [Paper]
  • Pixel Deconvolutional Networks-2017 [Code-Tensorflow] [Paper]
  • [DRN] [CVPR 2017] Dilated Residual Networks [Paper] [Code]
  • [Deeplab v3] Deeplab v3: Rethinking Atrous Convolution for Semantic Image Segmentation [Paper]
  • [LinkNet] LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation [Paper]
  • [SDN] Stacked Deconvolutional Network for Semantic Segmentation [Paper]
  • Learning to Segment Every Thing [Paper]




Real-Time Semantic Segmentation

  1. [ENet] ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation-2016 [Paper]
  2. [ICNet] [ECCV 2018] ICNet for Real-Time Semantic Segmentation on High-Resolution Images [Project] [Code] [Paper] [Video] (Uses deep supervision and runs the input image at different scales, each scale through their own subnetwork and progressively combining the results)
  3. [RTSeg] RTSeg: Real-time Semantic Segmentation Comparative Study [Paper]
  4. [ShuffleSeg] ShuffleSeg: Real-time Semantic Segmentation Network [Paper]
  5. [ESPNet] [ECCV 2018] ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation [Paper]
  6. [ContextNet] [BMVC 2018] ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time [Paper]
  7. Guided Upsampling Network for Real-Time Semantic Segmentation [Project] [Paper]
  8. [BiSeNet] [ECCV 2018] BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation [Paper] (Has 2 branches: one is deep for getting semantic information, while the other does very little / minor processing on the input image as to preserve the low-level pixel information)
  9. Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation [Paper]
  10. [BMVC 2018] Light-Weight RefineNet for Real-Time Semantic Segmentation [Paper] [code]
  11. CGNet: A Light-weight Context Guided Network for Semantic Segmentation [Paper] [Code]
  12. ShelfNet for Real-time Semantic Segmentation [Paper]
  13. ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network [Paper][Code]
  14. DSNet for Real-Time Driving Scene Semantic Segmentation [Paper]
  15. [CVPR 2019] Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search [Paper]
  16. Real time backbone for semantic segmentation [Paper]
  17. In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images [Paper]
  18. Residual Pyramid Learning for Single-Shot Semantic Segmentation [Paper]
  19. DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation [Paper]
  20. LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation [Paper] [Code]
  21. ESNet: An Efficient Symmetric Network for Real-time Semantic Segmentation [Paper] [Code]
  22. DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation [Paper]
  23. SqueezeNAS: Fast neural architecture search for faster semantic segmentation [1908.01748]

Loss Fuction

  1. The Lovász Hinge: A Novel Convex Surrogate for Submodular Losses [arxiv] [project]
  2. [CVPR 2017 ] Loss Max-Pooling for Semantic Image Segmentation [Paper]
  3. [CVPR 2018] The Lovász-Softmax loss:A tractable surrogate for the optimization of the intersection-over-union measure in neural networks [Project] [Paper] [Code]
  4. Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations [Paper]
  5. IoU is not submodular [arxiv]
  6. Yes, IoU loss is submodular - as a function of the mispredictions [arxiv]
  7. [BMVC 2018] NeuroIoU: Learning a Surrogate Loss for Semantic Segmentation [Paper] [code]
  8. The Ethical Dilemma when (not) Setting up Cost-based Decision Rules in Semantic Segmentation [Paper]


  • A Survey of Semantic Segmentation [arxiv]
  • A Review on Deep Learning Techniques Applied to Semantic Segmentation [arxiv]
  • Recent progress in semantic image segmentation [arxiv]
  • Survey on semantic segmentation using deep learning techniques [paper]
  • Understanding Deep Learning Techniques for Image Segmentation [arxiv 1907.06119]