Conditional Motion In-Betweening (CMIB)
Official implementation of paper: Conditional Motion In-betweeening.
Paper(arXiv) | Project Page | YouTube
| in-betweening | pose-conditioned |
|---|---|
 |
 |
| walk | jump | dance |
|---|---|---|
 |
 |
 |
Environments
This repo is tested on following environment:
- Ubuntu 20.04
- Python >= 3.7
- PyTorch == 1.10.1
- Cuda V11.3.109
Install
-
Follow
LAFAN1dataset's installation guide. You need to install git lfs first before cloning the dataset repo. -
Run LAFAN1's
evaluate.pyto unzip and validate it. (Installnumpyfirst if you don't have it)$ pip install numpy $ python ubisoft-laforge-animation-dataset/evaluate.py
With this, you will have unpacked LAFAN dataset under
ubisoft-laforge-animation-datasetfolder. -
Install appropriate
pytorchversion depending on your device(CPU/GPU), then install packages listed inrequirements.txt. .
Trained Weights
You can download trained weights from here.
Train from Scratch
Trining script is trainer.py.
python trainer.py \
--processed_data_dir="processed_data_80/" \
--window=90 \
--batch_size=32 \
--epochs=5000 \
--device=0 \
--entity=cmib_exp \
--exp_name="cmib_80" \
--save_interval=50 \
--learning_rate=0.0001 \
--loss_cond_weight=1.5 \
--loss_pos_weight=0.05 \
--loss_rot_weight=2.0 \
--from_idx=9 \
--target_idx=88 \
--interpolation='slerp'
Inference
You can use run_cmib.py for inference. Please refer to help page of run_cmib.py for more details.
python run_cmib.py --help
Reference
- LAFAN1 Dataset
@article{harvey2020robust, author = {Félix G. Harvey and Mike Yurick and Derek Nowrouzezahrai and Christopher Pal}, title = {Robust Motion In-Betweening}, booktitle = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH)}, publisher = {ACM}, volume = {39}, number = {4}, year = {2020} }
Citation
@misc{kim2022conditional,
title={Conditional Motion In-betweening},
author={Jihoon Kim and Taehyun Byun and Seungyoun Shin and Jungdam Won and Sungjoon Choi},
year={2022},
eprint={2202.04307},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
122 Dec 11, 2022
286 Dec 21, 2022
0 Nov 09, 2021
2 Mar 16, 2022
110 Nov 02, 2022
30 Nov 28, 2022
160 Jan 04, 2023
1 Apr 30, 2021
26 Nov 20, 2022
13 Dec 13, 2022
748 Jan 07, 2023
10 Nov 11, 2022
48 Nov 16, 2022
9 May 10, 2022
67 Dec 31, 2022
130 Dec 07, 2022
185 Dec 29, 2022
1 Mar 20, 2022
2.3k Dec 30, 2022
23 Dec 02, 2022