InsGen - Data-Efficient Instance Generation from Instance Discrimination

Data-Efficient Instance Generation from Instance Discrimination
Ceyuan Yang, Yujun Shen, Yinghao Xu, Bolei Zhou
arXiv preprint arXiv: 2106.04566

[Paper] [Project Page]

In this work, we develop a novel data-efficient Instance Generation (InsGen) method for training GANs with limited data. With the instance discrimination as an auxiliary task, our method makes the best use of both real and fake images to train the discriminator. The discriminator in turn guides the generator to synthesize as many diverse images as possible. Experiments under different data regimes show that InsGen brings a substantial improvement over the baseline in terms of both image quality and image diversity, and outperforms previous data augmentation algorithms by a large margin.

Qualitative results

Here we provide some synthesized samples with different numbers of training images and correspoding FID. Full codebase and weights are coming soon.

Inference

Here, all pretrained models can be downloaded from Google Drive:

You can download one of them and put it under MODEL_ZOO directory, then synthesize images via

# Generate AFHQ512-CAT with truncation.
python generate.py --network=${MODEL_ZOO}/afhqcat.pkl \
                   --outdir=${TARGET_DIR} \
                   --trunc=0.7 \
                   --seeds=0-10

Training

This repository is built based on styleGAN2-ada-pytorch. Therefore, please prepare datasets first use dataset_tool.py. On top of Generative Adversarial Networks (GANs), we introduce contrastive loss into the training of discriminator, following MoCo. Concretely, the discriminator is used to extract features from images (either real or synthesized) and then trained with an auxiliary task by distinguishing every individual image.

As described in training/contrastive_head.py, we add two addition heads on top of the original discriminator. These two heads are used to project features extracted from real and fake data onto a unit ball respectively. More details can be found in paper. Note that InsGen can be easily applied to any GAN model by merely introducing two contrastive heads. According to MoCo, the feature extractor should be updated in a momentum manner. Here, in InsGen, the contrastive heads are updated in the forward() function, while the discriminator is updated in training/training_loop.py (see D_ema).

Please use the following command to start your own training:

python train.py --gpus=8 \
                --data=${DATA_PATH} \
                --cfg=paper256 \
                --outdir=training_example

In this example, the results are saved to a created director training_example. --cfg specifies the training configuration, which can be further customized with additional options:

• --no_insgen disables InsGen, back to original StyleGAN2-ADA.
• --rqs overrides the number of real image queue size. (default: 5% of the total number of training samples)
• --fqs overrides the number of fake image queue size. More samples are beneficial, especially when the training samples are limited. (default: 5% of the total number of training samples)
• --gamma overrides the R1 gamma (i.e., gradient penalty). As described in styleGAN2-ada-pytorch, training can be sensitive to this hyper-parameter. It would be better to try some different values. Here, we recommend using a smaller one than that in original StyleGAN2-ADA.

More functions would be supported after this projest is merged into our genforce. Please stay tuned!

License

This work is made available under the Nvidia Source Code License.

Acknowledgements

We thank Janne Hellsten and Tero Karras for the pytorch version codebase of their styleGAN2-ada-pytorch.

BibTeX

@article{yang2021insgen,
  title   = {Data-Efficient Instance Generation from Instance Discrimination},
  author  = {Yang, Ceyuan and Shen, Yujun and Xu, Yinghao and Zhou, Bolei},
  journal = {arXiv preprint arXiv:2106.04566},
  year    = {2021}
}