RCIL

[CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation
Chang-Bin Zhang¹, Jia-Wen Xiao¹, Xialei Liu¹, Ying-Cong Chen², Ming-Ming Cheng^1
1 _{College of Computer Science, Nankai University}
² _{The Hong Kong University of Science and Technology}

Method

Update

• Coming Soon add data folder
• Coming Soon init code for Classification
• Coming Soon add training scripts for ADE20K and cityscapes
• 09/04/2022 init code for segmentation
• 09/04/2022 init readme

Benchmark and Setting

There are two commonly used settings, disjoint and overlapped. In the disjoint setting, assuming we know all classes in the future, the images in the current training step do not contain any classes in the future. The overlapped setting allows potential classes in the future to appear in the current training images. We call each training on the newly added dataset as a step. Formally, X-Y denotes the continual setting in our experiments, where X denotes the number of classes that we need to train in the first step. In each subsequent learning step, the newly added dataset contains Y classes.

There are some settings reported in our paper. You can also try it on other any custom settings.

• Continual Class Segmentation:

  1. PASCAL VOC 2012 dataset:
     • 15-5 overlapped
     • 15-5 disjoint
     • 15-1 overlapped
     • 15-1 disjoint
     • 10-1 overlapped
     • 10-1 disjoint
  2. ADE20K dataset:
     • 100-50 overlapped
     • 100-10 overlapped
     • 50-50 overlapped
     • 100-5 overlapped

• Continual Domain Segmentation:

  1. Cityscapes:
     • 11-5
     • 11-1
     • 1-1

• Extension Experiments on Continual Classification

  1. ImageNet-100
     • 50-10

Performance

• Continual Class Segmentation on PASCAL VOC 2012

• Continual Class Segmentation on ADE20K

• Continual Domain Segmentation on Cityscapes

Dataset Prepare

• PASCVAL VOC 2012
  sh data/download_voc.sh
• ADE20K
  sh data/download_ade.sh
• Cityscapes
  sh data/download_cityscapes.sh

Environment

1. conda install --yes --file requirements.txt
2. Install inplace-abn

Training

1. Dowload pretrained model from ResNet-101_iabn to pretrained/
2. We have prepared some training scripts in scripts/. You can train the model by

sh scripts/voc/rcil_10-1-overlap.sh

Inference

You can simply modify the bash file by add --test, like

CUDA_VISIBLE_DEVICES=${GPU} python3 -m torch.distributed.launch --master_port ${PORT} --nproc_per_node=${NB_GPU} run.py --data xxx ... --test

Reference

If this work is useful for you, please cite us by:

@inproceedings{zhangCvpr22ContinuSSeg,
  title={Representation Compensation Networks for Continual Semantic Segmentation},
  author={Chang-Bin Zhang and Jiawen Xiao and Xialei Liu and Yingcong Chen and Ming-Ming Cheng},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
  year={2022}
}

Connect

If you have any questions about this work, please feel easy to connect with us (zhangchbin ^ gmail.com).

Thanks

This code is heavily borrowed from [MiB] and [PLOP].

Awesome Continual Segmentation

There is a collection of AWESOME things about continual semantic segmentation, including papers, code, demos, etc. Feel free to pull request and star.

2022

• Representation Compensation Networks for Continual Semantic Segmentation [CVPR 2022] [PyTorch]
• Self-training for Class-incremental Semantic Segmentation [TNNLS 2022] [PyTorch]
• Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation [TPAMI 2022] [[PyTorch]]

2021

• PLOP: Learning without Forgetting for Continual Semantic Segmentation [CVPR 2021] [PyTorch]
• Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations [CVPR2021] [PyTorch]
• An EM Framework for Online Incremental Learning of Semantic Segmentation [ACM MM 2021] [PyTorch]
• SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning [NeurIPS 2021] [PyTorch]

2020

• Modeling the Background for Incremental Learning in Semantic Segmentation [CVPR 2020] [PyTorch]

2019

• Incremental Learning Techniques for Semantic Segmentation [ICCV Workshop 2019] [PyTorch]