Dirty Pixels: Towards End-to-End Image Processing and Perception

This repository contains the code for the paper

Dirty Pixels: Towards End-to-End Image Processing and Perception
Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, Felix Heide
Transactions on Graphics, 2021 | To be presented at SIGGRAPH, 2021

Installation

Clone this repository:

git clone [email protected]:princeton-computational-imaging/DirtyPixels.git

The project was developed using Python 3.6, Tensorflow (v1.12) and Slim. We provide an environment file to install all dependencies (creating an envirnoment called dirtypix):

conda env create -f environment.yml
conda activate dirtypix

Running Experiments

We provide code and data and trained models to reproduce the main results presented at the paper, and instructions on how to use this project for further research:

• EVALUATION_INSTRUCTIONS.md provides instructions on how to evaluate our proposed models and reproduce results of the paper.
• TRAINING_INSTRUCTIONS.md gives instructions on how to train new models following our proposed approach.
• ADD_NOISE_INSTRUCTIONS.md explains how to simulate noisy raw images following the image formation model defined in the manuscript.

Citation

If you find our work useful in your research, please cite:

@article{steven:dirtypixels2021,
  title={Dirty Pixels: Towards End-to-End Image Processing and Perception},
  author={Diamond, Steven and Sitzmann, Vincent and Julca-Aguilar, Frank and Boyd, Stephen and Wetzstein, Gordon and Heide, Felix},
  journal={ACM Transactions on Graphics (SIGGRAPH)},
  year={2021},
  publisher={ACM}
}

License

This project is released under MIT License.