Conditional Motion In-Betweening (CMIB)
Official implementation of paper: Conditional Motion In-betweeening.
Paper(arXiv) | Project Page | YouTube
| in-betweening | pose-conditioned |
|---|---|
 |
 |
| walk | jump | dance |
|---|---|---|
 |
 |
 |
Environments
This repo is tested on following environment:
- Ubuntu 20.04
- Python >= 3.7
- PyTorch == 1.10.1
- Cuda V11.3.109
Install
-
Follow
LAFAN1dataset's installation guide. You need to install git lfs first before cloning the dataset repo. -
Run LAFAN1's
evaluate.pyto unzip and validate it. (Installnumpyfirst if you don't have it)$ pip install numpy $ python ubisoft-laforge-animation-dataset/evaluate.py
With this, you will have unpacked LAFAN dataset under
ubisoft-laforge-animation-datasetfolder. -
Install appropriate
pytorchversion depending on your device(CPU/GPU), then install packages listed inrequirements.txt. .
Trained Weights
You can download trained weights from here.
Train from Scratch
Trining script is trainer.py.
python trainer.py \
--processed_data_dir="processed_data_80/" \
--window=90 \
--batch_size=32 \
--epochs=5000 \
--device=0 \
--entity=cmib_exp \
--exp_name="cmib_80" \
--save_interval=50 \
--learning_rate=0.0001 \
--loss_cond_weight=1.5 \
--loss_pos_weight=0.05 \
--loss_rot_weight=2.0 \
--from_idx=9 \
--target_idx=88 \
--interpolation='slerp'
Inference
You can use run_cmib.py for inference. Please refer to help page of run_cmib.py for more details.
python run_cmib.py --help
Reference
- LAFAN1 Dataset
@article{harvey2020robust, author = {Félix G. Harvey and Mike Yurick and Derek Nowrouzezahrai and Christopher Pal}, title = {Robust Motion In-Betweening}, booktitle = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH)}, publisher = {ACM}, volume = {39}, number = {4}, year = {2020} }
Citation
@misc{kim2022conditional,
title={Conditional Motion In-betweening},
author={Jihoon Kim and Taehyun Byun and Seungyoun Shin and Jungdam Won and Sungjoon Choi},
year={2022},
eprint={2202.04307},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
1 Oct 20, 2021
4 Jun 16, 2021
4 Oct 17, 2021
7 Jan 08, 2023
0 Dec 07, 2022
1 Dec 31, 2021
1.1k Jan 02, 2023
25 Nov 30, 2022
221 Jan 07, 2023
6 Aug 24, 2022
47 Nov 26, 2022
42 Nov 01, 2022
93 Dec 18, 2022
4.7k Dec 31, 2022
51 Nov 17, 2022
425 Dec 22, 2022
80 Dec 25, 2022
45 Dec 13, 2022
15 Nov 04, 2022
139 Dec 27, 2022