GS-WGAN

This repository contains the implementation for GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators (NeurIPS 2020).

Contact: Dingfan Chen ([email protected])

Requirements

The environment can be set up using Anaconda with the following commands:

conda create --name gswgan-pytorch python=3.6
conda activate gswgan-pytorch
conda install pytorch=1.2.0 
conda install torchvision -c pytorch
pip install -r requirements.txt

Please note that modifications in registering the backward_hook (in source/main.py) may be required if you plan to use a different pytorch version. Please refer to the pytorch document (select pytorch version → torch.nn → Module → search for register_backward_hook) for more information.

Training

Step 1. To warm-start the discriminators:

cd source
sh pretrain.sh

• To run the training in parallel: adjust the 'meta_start' argument and run the script multiple times in parallel.
• Alternatively, you can download the pre-trained models using the links below.

Step 2. To train the differentially private generator:

cd source
python main.py -data 'mnist' -name 'ResNet_default' -ldir '../results/mnist/pretrain/ResNet_default'

• Please refer to source/config.py (or execute python main.py -h) for the complete list of arguments.

• The default setting require ~22G GPU memory. Please allocate multiple GPUs by specifying the '-ngpus' argument if it does not fit in the memory of one GPU.

Evaluation

Privacy

• To compute the privacy cost:

  cd evaluation
  python privacy_analysis.py -data 'mnist' -name 'ResNet_default'
  

Pre-trained Models

Pre-trained model checkpoints can be downloaded using the links below. The discriminators are obtained after the warm-starting step (step 1), while the generators are obtained after the DP training step (step 2). The pre-trained models are stored as .pth files and the corresponding training configurations are stored in params.pkl and params.txt.

Citation

@inproceedings{neurips20chen,
title = {GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators},
author = {Dingfan Chen and Tribhuvanesh Orekondy and Mario Fritz},
year = {2020},
date = {2020-12-06},
booktitle = {Neural Information Processing Systems (NeurIPS)},
pubstate = {published},
tppubtype = {inproceedings}
}

Acknowledgements

Our implementation uses the source code from the following repositories:

• Improved Training of Wasserstein GANs (Pytorch)

• Improved Training of Wasserstein GANs (Tensorflow)

• GANs with Spectral Normalization and Projection Discriminator

• Large Scale GAN Training for High Fidelity Natural Image Synthesis

• Progressive Growing of GANs