Offical implementation for "Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation".

Related tags

Deep LearningYTMT-Strategy
Overview

Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation (NeurIPS 2021)

by Qiming Hu, Xiaojie Guo.

Dependencies

  • Python3
  • PyTorch>=1.0
  • OpenCV-Python, TensorboardX, Visdom
  • NVIDIA GPU+CUDA

Network Architecture

figure_arch

🚀 1. Single Image Reflection Separation

Data Preparation

Training dataset

  • 7,643 images from the Pascal VOC dataset, center-cropped as 224 x 224 slices to synthesize training pairs.
  • 90 real-world training pairs provided by Zhang et al.

Tesing dataset

  • 45 real-world testing images from CEILNet dataset.
  • 20 real testing pairs provided by Zhang et al.
  • 454 real testing pairs from SIR^2 dataset, containing three subsets (i.e., Objects (200), Postcard (199), Wild (55)).

Usage

Training

  • For stage 1: python train_sirs.py --inet ytmt_ucs --model ytmt_model_sirs --name ytmt_ucs_sirs --hyper --if_align
  • For stage 2: python train_twostage_sirs.py --inet ytmt_ucs --model twostage_ytmt_model --name ytmt_uct_sirs --hyper --if_align --resume --resume_epoch xx --checkpoints_dir xxx

Testing

python test_sirs.py --inet ytmt_ucs --model twostage_ytmt_model --name ytmt_uct_sirs_test --hyper --if_align --resume --icnn_path ./checkpoints/ytmt_uct_sirs/twostage_unet_68_077_00595364.pt

Trained weights

Google Drive

Visual comparison on real20 and SIR^2

figure_eval

Visual comparison on real45

figure_test

🚀 2. Single Image Denoising

Data Preparation

Training datasets

400 images from the Berkeley segmentation dataset, following DnCNN.

Tesing datasets

BSD68 dataset and Set12.

Usage

Training

python train_denoising.py --inet ytmt_pas --name ytmt_pas_denoising --preprocess True --num_of_layers 9 --mode B --preprocess True

Testing

python test_denoising.py --inet ytmt_pas --name ytmt_pas_denoising_blindtest_25 --test_noiseL 25 --num_of_layers 9 --test_data Set68 --icnn_path ./checkpoints/ytmt_pas_denoising_49_157500.pt

Trained weights

Google Drive

Visual comparison on a sample from BSD68

figure_eval_denoising

🚀 3. Single Image Demoireing

Data Preparation

Training dataset

AIM 2019 Demoireing Challenge

Tesing dataset

100 moireing and clean pairs from AIM 2019 Demoireing Challenge.

Usage

Training

python train_demoire.py --inet ytmt_ucs --model ytmt_model_demoire --name ytmt_uas_demoire --hyper --if_align

Testing

python test_demoire.py --inet ytmt_ucs --model ytmt_model_demoire --name ytmt_uas_demoire_test --hyper --if_align --resume --icnn_path ./checkpoints/ytmt_ucs_demoire/ytmt_ucs_opt_086_00860000.pt

Trained weights

Google Drive

Visual comparison on the validation set of LCDMoire

figure_eval_demoire

You might also like...
Image-to-Image Translation with Conditional Adversarial Networks (Pix2pix) implementation in keras

pix2pix-keras Pix2pix implementation in keras. Original paper: Image-to-Image Translation with Conditional Adversarial Networks (pix2pix) Paper Author

William Falcon 141 Dec 30, 2022
Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

This is a Python implementation of cover trees, a data structure for finding nearest neighbors in a general metric space (e.g., a 3D box with periodic

Patrick Varilly 28 Nov 25, 2022
Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

RGF-team 364 Dec 28, 2022
Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow

xRBM Library Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow Installation Using pip: pip install xrbm Examples Tut

Omid Alemi 55 Dec 29, 2022
A fast Evolution Strategy implementation in Python

Evostra: Evolution Strategy for Python Evolution Strategy (ES) is an optimization technique based on ideas of adaptation and evolution. You can learn

Mika 251 Dec 8, 2022
🌳 A Python-inspired implementation of the Optimum-Path Forest classifier.

OPFython: A Python-Inspired Optimum-Path Forest Classifier Welcome to OPFython. Note that this implementation relies purely on the standard LibOPF. Th

Gustavo Rosa 30 Jan 4, 2023
Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021
Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biom

Phil Wang 59 Nov 24, 2022
Official implementation of AAAI-21 paper
Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

null 101 Nov 25, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
Comments
  • Datasets

    Datasets

    Hi,

    I have been trying to experiment with the model but I'm having trouble finding the correct datasets for testing. The Sirs2 dataset in the provided link doesn't have the images set up with the naming conventions used in the script. Could you please direct me to the correct data sets for testing and training? Is there a separate repository that you have used?

    Thanks so much,

    David

    opened by davidgaddie 3
  • About Training Details

    About Training Details

    Hello, thank you for sharing your wonderful work. I have some question about the triaining details. It says the training epoch is 120 in your paper but the epoch is set to 60 in YTMT-Strategy/options/net_options/train_options.py. Moreover, the best model in your paper is YTMT-UCT which need two stages training. Can you provide the training settings of the YTMT-UCT (epoch, batchsize...)? Look forward to your reply!

    opened by DUT-CSJ 2
  • CUDA vram allocation issue

    CUDA vram allocation issue

    Hi,

    I've been trying to run the reflection test code, but I get this error: RuntimeError: CUDA out of memory. Tried to allocate 15.66 GiB (GPU 0; 22.20 GiB total capacity; 16.09 GiB already allocated; 2.68 GiB free; 17.55 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

    I'm running on an A10G GPU on AWS. I suspect that maybe the dataset is incorrect as each image in the dataset I have is around 800MB. If that's the case can I please be directed to the correct repository for the read20_420 images?

    Thanks so much,

    David

    opened by davidgaddie 1
  • test demoire error

    test demoire error

    Thanks for your great work ,but some error when I run: python test_demoire.py --inet ytmt_ucs --model ytmt_model_demoire --name ytmt_uas_demoire_test --hyper --if_align --resume --icnn_path checkpoints/ytmt_ucs_demoire/ytmt_ucs_demoire_opt_086_00860000.pt

    -------------- End ---------------- [i] initialization method [edsr] Traceback (most recent call last): File "test_demoire.py", line 28, in engine = Engine(opt) File "/nfs_data/code/YTMT-Strategy-main/engine.py", line 19, in init self.__setup() File "/nfs_data/code/YTMT-Strategy-main/engine.py", line 29, in __setup self.model.initialize(opt) File "/nfs_data/code/YTMT-Strategy-main/models/ytmt_model_demoire.py", line 242, in initialize self.load(self, opt.resume_epoch) File "/nfs_data/code/YTMT-Strategy-main/models/ytmt_model_demoire.py", line 413, in load model.net_i.load_state_dict(state_dict['icnn']) File "/opt/conda/envs/torch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1223, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for YTMT_US: Missing key(s) in state_dict: "inc.ytmt_head.fusion_l.weight", "inc.ytmt_head.fusion_l.bias", "inc.ytmt_head.fusion_r.weight", "inc.ytmt_head.fusion_r.bias", "down1.model.ytmt_head.fusion_l.weight", "down1.model.ytmt_head.fusion_l.bias", "down1.model.ytmt_head.fusion_r.weight", "down1.model.ytmt_head.fusion_r.bias", "down2.model.ytmt_head.fusion_l.weight", "down2.model.ytmt_head.fusion_l.bias", "down2.model.ytmt_head.fusion_r.weight", "down2.model.ytmt_head.fusion_r.bias", "down3.model.ytmt_head.fusion_l.weight", "down3.model.ytmt_head.fusion_l.bias", "down3.model.ytmt_head.fusion_r.weight", "down3.model.ytmt_head.fusion_r.bias", "down4.model.ytmt_head.fusion_l.weight", "down4.model.ytmt_head.fusion_l.bias", "down4.model.ytmt_head.fusion_r.weight", "down4.model.ytmt_head.fusion_r.bias", "up1.model.ytmt_head.fusion_l.weight", "up1.model.ytmt_head.fusion_l.bias", "up1.model.ytmt_head.fusion_r.weight", "up1.model.ytmt_head.fusion_r.bias", "up2.model.ytmt_head.fusion_l.weight", "up2.model.ytmt_head.fusion_l.bias", "up2.model.ytmt_head.fusion_r.weight", "up2.model.ytmt_head.fusion_r.bias", "up3.model.ytmt_head.fusion_l.weight", "up3.model.ytmt_head.fusion_l.bias", "up3.model.ytmt_head.fusion_r.weight", "up3.model.ytmt_head.fusion_r.bias", "up4.model.ytmt_head.fusion_l.weight", "up4.model.ytmt_head.fusion_l.bias", "up4.model.ytmt_head.fusion_r.weight", "up4.model.ytmt_head.fusion_r.bias".

    opened by zdyshine 1
Owner
Qiming Hu
Qiming Hu
GitHub Repository
Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting (ICCV, 2021)

DKPNet ICCV 2021 Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting Baseline of DKPNet is availa

19 Oct 14, 2022
Code for KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs

KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs Check out the paper on arXiv: https://arxiv.org/abs/2103.13744 This repo cont

Christian Reiser 373 Dec 20, 2022
Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 03, 2022
A BaSiC Tool for Background and Shading Correction of Optical Microscopy Images

BaSiC Matlab code accompanying A BaSiC Tool for Background and Shading Correction of Optical Microscopy Images by Tingying Peng, Kurt Thorn, Timm Schr

Marr Lab 34 Dec 18, 2022
Code for the Image similarity challenge.

ISC 2021 This repository contains code for the Image Similarity Challenge 2021. Getting started The docs subdirectory has step-by-step instructions on

Facebook Research 173 Dec 12, 2022
MVGCN: a novel multi-view graph convolutional network (MVGCN) framework for link prediction in biomedical bipartite networks.

MVGCN MVGCN: a novel multi-view graph convolutional network (MVGCN) framework for link prediction in biomedical bipartite networks. Developer: Fu Hait

13 Dec 01, 2022
A curated list of resources for Image and Video Deblurring

A curated list of resources for Image and Video Deblurring

Subeesh Vasu 1.7k Jan 01, 2023
Tool cek opsi checkpoint facebook!

tool apa ini? cek_opsi_facebook adalah sebuah tool yang mengecek opsi checkpoint akun facebook yang terkena checkpoint! tujuan dibuatnya tool ini? too

Muhammad Latif Harkat 2 Jul 17, 2022
To propose and implement a multi-class classification approach to disaster assessment from the given data set of post-earthquake satellite imagery.

To propose and implement a multi-class classification approach to disaster assessment from the given data set of post-earthquake satellite imagery.

Kunal Wadhwa 2 Jan 05, 2022
Collection of generative models, e.g. GAN, VAE in Pytorch and Tensorflow.

Generative Models Collection of generative models, e.g. GAN, VAE in Pytorch and Tensorflow. Also present here are RBM and Helmholtz Machine. Note: Gen

Agustinus Kristiadi 7k Jan 02, 2023
Amazing-Python-Scripts - 🚀 Curated collection of Amazing Python scripts from Basics to Advance with automation task scripts.

📑 Introduction A curated collection of Amazing Python scripts from Basics to Advance with automation task scripts. This is your Personal space to fin

Avinash Ranjan 1.1k Dec 29, 2022
Learning Dense Representations of Phrases at Scale (Lee et al., 2020)

DensePhrases DensePhrases provides answers to your natural language questions from the entire Wikipedia in real-time. While it efficiently searches th

Princeton Natural Language Processing 540 Dec 30, 2022
Code for reproducing key results in the paper "InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets"

Status: Archive (code is provided as-is, no updates expected) InfoGAN Code for reproducing key results in the paper InfoGAN: Interpretable Representat

OpenAI 1k Dec 19, 2022
The repo for the paper "I3CL: Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detection".

I3CL: Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detection Updates | Introduction | Results | Usage | Citation |

33 Jan 05, 2023
An inofficial PyTorch implementation of PREDATOR based on KPConv.

PREDATOR: Registration of 3D Point Clouds with Low Overlap An inofficial PyTorch implementation of PREDATOR based on KPConv. The code has been tested

ZhuLifa 14 Aug 03, 2022
Providing the solutions for high-frequency trading (HFT) strategies using data science approaches (Machine Learning) on Full Orderbook Tick Data.

Modeling High-Frequency Limit Order Book Dynamics Using Machine Learning Framework to capture the dynamics of high-frequency limit order books. Overvi

Chang-Shu Chung 1.3k Jan 07, 2023
Parameterized Explainer for Graph Neural Network

PGExplainer This is a Tensorflow implementation of the paper: Parameterized Explainer for Graph Neural Network https://arxiv.org/abs/2011.04573 NeurIP

Dongsheng Luo 89 Dec 12, 2022
Türkiye Canlı Mobese Görüntülerinde Profesyonel Nesne Takip Sistemi

Türkiye Mobese Görüntü Takip Türkiye Mobese görüntülerinde OPENCV ve Yolo ile takip sistemi Multiple Object Tracking System in Turkish Mobese with OPE

15 Dec 22, 2022
Code for CVPR2021 paper "Learning Salient Boundary Feature for Anchor-free Temporal Action Localization"

AFSD: Learning Salient Boundary Feature for Anchor-free Temporal Action Localization This is an official implementation in PyTorch of AFSD. Our paper

Tencent YouTu Research 146 Dec 24, 2022
A Web API for automatic background removal using Deep Learning. App is made using Flask and deployed on Heroku.

Automatic_Background_Remover A Web API for automatic background removal using Deep Learning. App is made using Flask and deployed on Heroku. 👉 https:

Gaurav 16 Oct 29, 2022