vision-transformers-pytorch

Implementation of various Vision Transformers (and other vision models) I found interesting

Models

Currently I have implemented:

NFNet (https://arxiv.org/abs/2102.06171)

Tested and got 83.17 top-1 accuracy with NFNet-F0

Pyramid Vision Transformer (https://arxiv.org/abs/2102.12122)

Tested and got 78.94 top-1 accuracy with PVT-Small

Swin Transformer (https://arxiv.org/abs/2103.14030)

Currently testing Swin-S

Halo Transformer (https://arxiv.org/abs/2103.12731)