Differentiable Annealed Importance Sampling (DAIS)

This repository contains the code to reproduce the DAIS results from the paper Differentiable Annealed Importance Sampling and the Perils of Gradient Noise.

Bayesian Linear Regression

Please first install JAX (https://github.com/google/jax), then reproduce our results by running the following:

python blr/dais.py --gamma 0.0
python blr/dais.py --gamma 0.9
python blr.dais.py --bsize 100

Variational Autoencoder

Please first install PyTorch (https://pytorch.org/).

For training, one example command for DAIS with $K = 10$ and $S = 5$ (to adapt annealing scheme, add --adapt_beta):

python vae/mnist_train.py --lf_step 10 --n_particles 5 --lf_lrate 0.08

After training, you can find the saved checkpoint and evaluate it (e.g., using AIS):

python vae/mnist_eval.py --ais --n_particles 10 --lf_step 10000 --lf_lrate 0.05 --resume xxx

Also, you can reproduce the evaulation results of AIS/HAIS/DAIS by runing vae/mnist_eval_scaling.py:

python vae/mnist_eval_scaling.py --linear_beta --lf_lrate 0.08 --n_particles 10 --resume xxx

Citation

To cite this work, please use

@inproceedings{
	zhang2021differentiable,
	title={Differentiable Annealed Importance Sampling and the Perils of Gradient Noise},
	author={Guodong Zhang and Kyle Hsu and Jianing Li and Chelsea Finn and Roger Baker Grosse},
	booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
	year={2021},
	url={https://openreview.net/forum?id=6rqjgrL7Lq}
}

Differentiable Annealed Importance Sampling (DAIS)

Related tags

Overview

Differentiable Annealed Importance Sampling (DAIS)

Bayesian Linear Regression

Variational Autoencoder

Citation

Owner

Guodong Zhang

PyTorch implementation of "VRT: A Video Restoration Transformer"

[CVPR 2021] Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

N-Omniglot is a large neuromorphic few-shot learning dataset

Official implementation of the RAVE model: a Realtime Audio Variational autoEncoder

一个多模态内容理解算法框架，其中包含数据处理、预训练模型、常见模型以及模型加速等模块。

NEO: Non Equilibrium Sampling on the orbit of a deterministic transform

Automatically erase objects in the video, such as logo, text, etc.

Bayesian Neural Networks in PyTorch

Text to Image Generation with Semantic-Spatial Aware GAN

Little Ball of Fur - A graph sampling extension library for NetworKit and NetworkX (CIKM 2020)

Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy" (ICLR 2022 Spotlight)

Official Pytorch implementation for 2021 ICCV paper "Learning Motion Priors for 4D Human Body Capture in 3D Scenes" and trained models / data

Sub-Cluster AdaCos: Learning Representations for Anomalous Sound Detection.

PyTorch implementation of SCAFFOLD (Stochastic Controlled Averaging for Federated Learning, ICML 2020).

An exploration of log domain "alternative floating point" for hardware ML/AI accelerators.

CVAT is free, online, interactive video and image annotation tool for computer vision

Flybirds - BDD-driven natural language automated testing framework, present by Trip Flight

Unsupervised Foreground Extraction via Deep Region Competition

The implement of papar "Enhanced Graph Learning for Collaborative Filtering via Mutual Information Maximization"

Code base for the paper "Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation"