ContourletNet: A Generalized Rain Removal Architecture Using Multi-Direction Hierarchical Representation
(Accepted by BMVC'21)

Abstract:

Images acquired from rainy scenes usually suffer from bad visibility which may damage the performance of computer vision applications. The rainy scenarios can be categorized into two classes: moderate rain and heavy rain scenes. Moderate rain scene mainly consists of rain streaks while heavy rain scene contains both rain streaks and the veiling effect (similar to haze). Although existing methods have achieved excellent performance on these two cases individually, it still lacks a general architecture to address both heavy rain and moderate rain scenarios effectively. In this paper, we construct a hierarchical multi-direction representation network by using the contourlet transform (CT) to address both moderate rain and heavy rain scenarios. The CT divides the image into the multi-direction subbands (MS) and the semantic subband (SS). First, the rain streak information is retrieved to the MS based on the multi-orientation property of the CT. Second, a hierarchical architecture is proposed to reconstruct the background information including damaged semantic information and the veiling effect in the SS. Last, the multi-level subband discriminator with the feedback error map is proposed. By this module, all subbands can be well optimized. This is the first architecture that can address both of the two scenarios effectively.

[Paper] [Supplementary Material]

You can also refer our previous works on other low-level vision applications!

Desnowing-[HDCWNet] (ICCV'21) and [JSTASR](ECCV'20)
Dehazing-[PMS-Net](CVPR'19) and [PMHLD](TIP'20)
Image Relighting-[MB-Net] (NTIRE'21 1st solution) and [S3Net] (NTIRE'21 3 rd solution)

Network Architecture

Experimental Results

Quantitative Evaluation

Qualitative Evaluation

Setup and environment

To generate the recovered result you need:

1. Python 3
2. CPU or NVIDIA GPU + CUDA CuDNN
3. Pytorch 1.0+

For moderate rain (trained on Rain100H dataset)

$ python test_real.py --ckpt ckpt/r100h --real_dir input_img/moderate

For heavy rain (trained on Heavy Rain dataset)

$ python test_real.py --ckpt ckpt/heavyrain --real_dir input_img/heavy

Citations

Please cite this paper in your publications if it is helpful for your tasks:

Bibtex:

@inproceedings{chen2021contour,
  title={ContourletNet: A Generalized Rain Removal Architecture Using Multi-Direction Hierarchical Representation},
  author={Chen, Wei-Ting and Tsai, Cheng-Che and Fang, Hao-Yu and and Chen, I-Hsiang and Ding, Jian-Jiun and Kuo, Sy-Yen},
  booktitle={Proceedings of the British Machine Vision Conference},
  year={2021}
}