code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

Related tags

Deep LearningMMNet
Overview

MMNet

This repo is the official implementation of ICCV 2021 paper "Multi-scale Matching Networks for Semantic Correspondence.".

Pre-requisite

conda create -n mmnet python==3.8.0
conda activate mmnet
conda install torch==1.8.1 torchvision==0.9.1
pip install matplotlib scikit-image pandas

for installation of gluoncvth (fcn-resnet101):

git clone https://github.com/StacyYang/gluoncv-torch.git
cd gluoncv-torch
python setup.py install

Reproduction

for test

Trained models are available on [google drive].

pascal with fcn-resnet101 backbone([email protected]:81.6%):

python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name path\to\ckp_pascal_fcnres101.pth --resize 224,320

spair with fcn-resnet101 backbone([email protected]:46.6%):

python test.py --alpha 0.05 --benchmark spair --backbone fcn-resnet101 --ckp_name path\to\ckp_spair_fcnres101.pth --resize 224,320

Bibtex

If you use this code for your research, please consider citing:

@article{zhao2021multi,
  title={Multi-scale Matching Networks for Semantic Correspondence},
  author={Zhao, Dongyang and Song, Ziyang and Ji, Zhenghao and Zhao, Gangming and Ge, Weifeng and Yu, Yizhou},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021}
}
You might also like...
A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

null 69 Dec 10, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 72 Dec 14, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 26 Sep 26, 2021
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
A PyTorch implementation of
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

Jiaqi Gu 2 Jan 4, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
《Dual-Resolution Correspondence Network》(NeurIPS 2020)
《Dual-Resolution Correspondence Network》(NeurIPS 2020)

Dual-Resolution Correspondence Network Dual-Resolution Correspondence Network, NeurIPS 2020 Dependency All dependencies are included in asset/dualrcne

Active Vision Laboratory 45 Nov 21, 2022
Comments
  • NaN during training

    NaN during training

    Hi, congrats on your paper! I was trying to run your training code (with resnet 101 on pf-pascal) but directly after a couple of iterations, nan appear in the input. Have you ever seen this issue? Thanks

    opened by PruneTruong 2
  • In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    Hello,this paper is very nice,i am very love it. I read your code,in Model.py, def calLayer1(self, feats): sum1 = self.conv1_1_down(self.msblock1_1(feats[1])) +
    self.conv1_2_down(self.msblock1_2(feats[2])) +
    self.conv1_3_down(self.msblock1_3(feats[3])) sum1 = self.wa_1(sum1) return sum1 I do not find where are these operation,self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1,so where are these ,in which document.Thank you,looking forward to your reply.

    opened by liang532 1
  • How to prepare the PF-Pascal dataset?

    How to prepare the PF-Pascal dataset?

    I downloaded the PF-dataset-Pascal.zip from the Proposal Flow paper's web page, extracted it, and run the next line of command, but get errors about missing data files.

    Input:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_pascal.pth --resize 224,320

    Expected output: some results about the benchmark results.

    Actual output:

    currently executing test.py file.
2021-11-19 02:01:59,172 - INFO - Options listed below:----------------
2021-11-19 02:01:59,172 - INFO - name: framework_train
2021-11-19 02:01:59,172 - INFO - benchmark: pfpascal
2021-11-19 02:01:59,172 - INFO - thresh_type: auto
2021-11-19 02:01:59,172 - INFO - backbone_name: fcn-resnet101
2021-11-19 02:01:59,172 - INFO - ms_rate: 4
2021-11-19 02:01:59,173 - INFO - feature_channel: 21
2021-11-19 02:01:59,173 - INFO - batch: 5
2021-11-19 02:01:59,173 - INFO - gpu: 0
2021-11-19 02:01:59,173 - INFO - data_path: /data/SC_Dataset
2021-11-19 02:01:59,173 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 02:01:59,173 - INFO - visualization_path: visualization
2021-11-19 02:01:59,173 - INFO - model_type: MMNet
2021-11-19 02:01:59,173 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_pascal.pth
2021-11-19 02:01:59,173 - INFO - log_path: ./logs/
2021-11-19 02:01:59,173 - INFO - resize: 224,320
2021-11-19 02:01:59,173 - INFO - max_kps_num: 50
2021-11-19 02:01:59,173 - INFO - split_type: test
2021-11-19 02:01:59,173 - INFO - alpha: 0.05
2021-11-19 02:01:59,173 - INFO - resolution: 2
2021-11-19 02:01:59,173 - INFO - Options all listed.------------------
2021-11-19 02:01:59,173 - INFO - ckp file: assets/model/mmnet_fcnresnet101_pascal.pth
Traceback (most recent call last):
  File "/home/runner/MMNet/test.py", line 127, in <module>
    test(logger, options)
  File "/home/runner/MMNet/test.py", line 65, in test
    test_dataset = Dataset.CorrespondenceDataset(
  File "/home/runner/MMNet/data/PascalDataset.py", line 32, in __init__
    self.train_data = pd.read_csv(self.spt_path)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/common.py", line 702, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/data/SC_Dataset/PF-PASCAL/test_pairs.csv'

    P.S. Output of executing ls /data/SC_Dataset/PF-PASCAL/:

    Annotations  html  index.html  JPEGImages  parsePascalVOC.mat  ShowMatchingPairs
    opened by tjyuyao 2
  • How to reproduce the reported test accuracy?

    How to reproduce the reported test accuracy?

    By running given following command with code on the main branch:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_spair.pth --resize 224,320 --benchmark spair

    I expect to get the reported accuracy in the Table.2 of paper, i.e. 50.4 "all" accuracy, or spair with fcn-resnet101 backbone([email protected]:46.6%): as noted in the README.md file. However I get the following output, finding nowhere the related results. Can you point out the steps to reproduce the test accuracy?

    2021-11-19 00:49:54,452 - INFO - Options listed below:----------------
2021-11-19 00:49:54,452 - INFO - name: framework_train
2021-11-19 00:49:54,453 - INFO - benchmark: spair
2021-11-19 00:49:54,453 - INFO - thresh_type: auto
2021-11-19 00:49:54,454 - INFO - backbone_name: fcn-resnet101
2021-11-19 00:49:54,455 - INFO - ms_rate: 4
2021-11-19 00:49:54,455 - INFO - feature_channel: 21
2021-11-19 00:49:54,456 - INFO - batch: 5
2021-11-19 00:49:54,456 - INFO - gpu: 0
2021-11-19 00:49:54,457 - INFO - data_path: /data/SC_Dataset
2021-11-19 00:49:54,457 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 00:49:54,458 - INFO - visualization_path: visualization
2021-11-19 00:49:54,458 - INFO - model_type: MMNet
2021-11-19 00:49:54,459 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:49:54,459 - INFO - log_path: ./logs/
2021-11-19 00:49:54,460 - INFO - resize: 224,320
2021-11-19 00:49:54,460 - INFO - max_kps_num: 50
2021-11-19 00:49:54,461 - INFO - split_type: test
2021-11-19 00:49:54,461 - INFO - alpha: 0.05
2021-11-19 00:49:54,462 - INFO - resolution: 2
2021-11-19 00:49:54,462 - INFO - Options all listed.------------------
2021-11-19 00:49:54,463 - INFO - ckp file: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:50:04,950 - INFO - [    0/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] aeroplane
2021-11-19 00:50:04,953 - INFO - [    1/12234]: 	 [Pair PCK: 0.100]	[Average: 0.217] aeroplane
2021-11-19 00:50:04,956 - INFO - [    2/12234]: 	 [Pair PCK: 0.308]	[Average: 0.247] aeroplane
2021-11-19 00:50:04,958 - INFO - [    3/12234]: 	 [Pair PCK: 0.364]	[Average: 0.276] aeroplane
2021-11-19 00:50:04,960 - INFO - [    4/12234]: 	 [Pair PCK: 0.000]	[Average: 0.221] aeroplane
2021-11-19 00:50:05,575 - INFO - [    5/12234]: 	 [Pair PCK: 0.200]	[Average: 0.217] aeroplane
2021-11-19 00:50:05,577 - INFO - [    6/12234]: 	 [Pair PCK: 0.250]	[Average: 0.222] aeroplane
2021-11-19 00:50:05,580 - INFO - [    7/12234]: 	 [Pair PCK: 0.308]	[Average: 0.233] aeroplane
2021-11-19 00:50:05,583 - INFO - [    8/12234]: 	 [Pair PCK: 0.182]	[Average: 0.227] aeroplane
2021-11-19 00:50:05,585 - INFO - [    9/12234]: 	 [Pair PCK: 0.636]	[Average: 0.268] aeroplane
2021-11-19 00:50:06,153 - INFO - [   10/12234]: 	 [Pair PCK: 0.667]	[Average: 0.304] aeroplane
2021-11-19 00:50:06,156 - INFO - [   11/12234]: 	 [Pair PCK: 0.385]	[Average: 0.311] aeroplane
2021-11-19 00:50:06,158 - INFO - [   12/12234]: 	 [Pair PCK: 0.455]	[Average: 0.322] aeroplane
2021-11-19 00:50:06,160 - INFO - [   13/12234]: 	 [Pair PCK: 0.250]	[Average: 0.317] aeroplane
2021-11-19 00:50:06,163 - INFO - [   14/12234]: 	 [Pair PCK: 0.615]	[Average: 0.337] aeroplane
2021-11-19 00:50:06,731 - INFO - [   15/12234]: 	 [Pair PCK: 0.000]	[Average: 0.316] aeroplane
...
2021-11-19 01:13:47,264 - INFO - [12216/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,265 - INFO - [12217/12234]: 	 [Pair PCK: 0.200]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,266 - INFO - [12218/12234]: 	 [Pair PCK: 0.250]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,268 - INFO - [12219/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,837 - INFO - [12220/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,838 - INFO - [12221/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,848 - INFO - [12222/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,850 - INFO - [12223/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,853 - INFO - [12224/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,422 - INFO - [12225/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,424 - INFO - [12226/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,425 - INFO - [12227/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,427 - INFO - [12228/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,429 - INFO - [12229/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,896 - INFO - [12230/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12231/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12232/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,901 - INFO - [12233/12234]: 	 [Pair PCK: 0.111]	[Average: 0.333] tvmonitor
    opened by tjyuyao 1
Releases(v0.1.0)
Owner
joey zhao
Master in Computer Sciences and Technology at Fudan University
joey zhao
GitHub Repository
Set of models for classifcation of 3D volumes

Classification models 3D Zoo - Keras and TF.Keras This repository contains 3D variants of popular CNN models for classification like ResNets, DenseNet

69 Dec 28, 2022
A pyparsing-based library for parsing SOQL statements

CONTRIBUTORS WANTED!! Installation pip install python-soql-parser or, with poetry poetry add python-soql-parser Usage from python_soql_parser import p

Kicksaw 0 Jun 07, 2022
A unet implementation for Image semantic segmentation

Unet-pytorch a unet implementation for Image semantic segmentation 参考网上的Unet做分割的代码，做了一个针对kaggle地盐识别的，请去以下地址获取数据集: https://www.kaggle.com/c/tgs-salt-id

Rabbit 3 Jun 29, 2022
POT : Python Optimal Transport

POT: Python Optimal Transport This open source Python library provide several solvers for optimization problems related to Optimal Transport for signa

Python Optimal Transport 1.7k Dec 31, 2022
Non-Attentive-Tacotron - This is Pytorch Implementation of Google's Non-attentive Tacotron.

Non-attentive Tacotron - PyTorch Implementation This is Pytorch Implementation of Google's Non-attentive Tacotron, text-to-speech system. There is som

Jounghee Kim 46 Dec 19, 2022
This is a GUI interface which can process forest fire detection, smoke detection and fire segmentation

This is a GUI interface which can process forest fire detection, smoke detection and fire segmentation. Yolov5 is used to detect fire and smoke and unet is used to segment fire.

7 Jan 08, 2023
A set of simple scripts to process the Imagenet-1K dataset as TFRecords and make index files for NVIDIA DALI.

Overview This is a set of simple scripts to process the Imagenet-1K dataset as TFRecords and make index files for NVIDIA DALI. Make TFRecords To run t

8 Nov 01, 2022
The official implementation of Equalization Loss for Long-Tailed Object Recognition (CVPR 2020) based on Detectron2

Equalization Loss for Long-Tailed Object Recognition Jingru Tan, Changbao Wang, Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, Junjie Yan ⚠️ We re

Jingru Tan 197 Dec 25, 2022
On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks We provide the code (in PyTorch) and datasets for our paper "On Size-Orient

Zemin Liu 4 Jun 18, 2022
Lightweight Salient Object Detection in Optical Remote Sensing Images via Feature Correlation

CorrNet This project provides the code and results for 'Lightweight Salient Object Detection in Optical Remote Sensing Images via Feature Correlation'

Gongyang Li 13 Nov 03, 2022
Robbing the FED: Directly Obtaining Private Data in Federated Learning with Modified Models

Robbing the FED: Directly Obtaining Private Data in Federated Learning with Modified Models This repo contains a barebones implementation for the atta

16 Dec 04, 2022
Differential fuzzing for the masses!

NEZHA NEZHA is an efficient and domain-independent differential fuzzer developed at Columbia University. NEZHA exploits the behavioral asymmetries bet

147 Dec 05, 2022
Geometric Vector Perceptron --- a rotation-equivariant GNN for learning from biomolecular structure

Geometric Vector Perceptron Code to accompany Learning from Protein Structure with Geometric Vector Perceptrons by B Jing, S Eismann, P Suriana, RJL T

Dror Lab 85 Dec 29, 2022
PyTorch implementation of the ACL, 2021 paper Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks.

Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks This repo contains the PyTorch implementation of the ACL, 2021 pa

Rabeeh Karimi Mahabadi 98 Dec 28, 2022
SMD-Nets: Stereo Mixture Density Networks

SMD-Nets: Stereo Mixture Density Networks This repository contains a Pytorch implementation of "SMD-Nets: Stereo Mixture Density Networks" (CVPR 2021)

Fabio Tosi 115 Dec 26, 2022
A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

yolov5-helmet-detection-python A Python implementation of Yolov5 to detect head or helmet in the wild in Jetson Xavier nx and Jetson nano. In Jetson X

12 Dec 05, 2022
Official repository of PanoAVQA: Grounded Audio-Visual Question Answering in 360° Videos (ICCV 2021)

Pano-AVQA Official repository of PanoAVQA: Grounded Audio-Visual Question Answering in 360° Videos (ICCV 2021) [Paper] [Poster] [Video] Getting Starte

Heeseung Yun 9 Dec 23, 2022
A simple python stock Predictor

Python Stock Predictor A simple python stock Predictor Demo Run Locally Clone the project git clone https://github.com/yashraj-n/stock-price-predict

Yashraj narke 5 Nov 29, 2021
Learning To Have An Ear For Face Super-Resolution

Learning To Have An Ear For Face Super-Resolution [Project Page] This repository contains demo code of our CVPR2020 paper. Training and evaluation on

50 Nov 16, 2022
Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations

Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations Trevor Ablett, Daniel (Yifan) Zhai, Jonatha

STARS Laboratory 3 Feb 01, 2022