RSC-Net: 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos

Implementation for "3D Human Pose, Shape and Texture from Low-Resolution Images and Videos", TPAMI 2021

Conference version: "3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning", ECCV 2020

Project page

What is new?

• RSC-Net:

  • Resolution-aware structure
  • Self-supervised learning
  • Contrastive learning

• Temporal post-processing for video input

• TexGlo: Global module for 3D texture reconstruction

Brief introduction

Video

Code

Packages

Make sure you have gcc==5.x.x for installing the packages. Then run:

bash install_environment.sh

If you are running the code without a screen, please install OSMesa and the corresponding PyOpenGL. Then uncomment the 2nd line of "utils/renderer.py".

Data preparation

• Download meta data, and unzip it in "./data".

• Download datasets, and unzip it in "./datasets_pkl".

Note that all paths are set in "config.py".

Demo

• Download pretrained RSC-Net, and put it in "./pretrained".
• Run:

python demo.py --checkpoint=./pretrained/RSC-Net.pt --img_path=./examples/im1.png

• Note: if you have trouble in using Pyrender, please try "demo_nr.py":

python demo_nr.py --checkpoint=./pretrained/RSC-Net.pt --img_path=./examples/im1.png

If your neural-renderer has errors, please re-install the package from the source.

Evaluation

python eval.py --checkpoint=./pretrained/RSC-Net.pt 

Training

python train.py --name=RSC-Net 

If you find this work helpful in your research, please cite our paper:

@article{xu20213d,
title={3D Human Pose, Shape and Texture from Low-Resolution Images and Videos},
author={Xu, Xiangyu and Chen, Hao and Moreno-Noguer, Francesc and Jeni, Laszlo A and De la Torre, Fernando},
journal={TPAMI},
year={2021},
}

@inproceedings{xu20203d,
title={3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning},
author={Xu, Xiangyu and Chen, Hao and Moreno-Noguer, Francesc and Jeni, Laszlo A and De la Torre, Fernando},
booktitle={ECCV},
year={2020},
}