|
2 | 2 |
|
3 | 3 | ## What's New
|
4 | 4 |
|
| 5 | +### Oct 13, 2020 |
| 6 | +* Initial impl of Vision Transformer models. Both patch and hybrid (CNN backbone) variants. Currently trying to train... |
| 7 | +* Adafactor and AdaHessian (FP32 only, no AMP) optimizers |
| 8 | +* EdgeTPU-M (`efficientnet_em`) model trained in PyTorch, 79.3 top-1 |
| 9 | +* Pip release, doc updates pending a few more changes... |
| 10 | + |
5 | 11 | ### Sept 18, 2020
|
6 | 12 | * New ResNet 'D' weights. 72.7 (top-1) ResNet-18-D, 77.1 ResNet-34-D, 80.5 ResNet-50-D
|
7 | 13 | * Added a few untrained defs for other ResNet models (66D, 101D, 152D, 200/200D)
|
@@ -124,6 +130,7 @@ A full version of the list below with source links can be found in the [document
|
124 | 130 | * SelecSLS - https://arxiv.org/abs/1907.00837
|
125 | 131 | * Selective Kernel Networks - https://arxiv.org/abs/1903.06586
|
126 | 132 | * TResNet - https://arxiv.org/abs/2003.13630
|
| 133 | +* Vision Transformer - https://openreview.net/forum?id=YicbFdNTTy |
127 | 134 | * VovNet V2 and V1 - https://arxiv.org/abs/1911.06667
|
128 | 135 | * Xception - https://arxiv.org/abs/1610.02357
|
129 | 136 | * Xception (Modified Aligned, Gluon) - https://arxiv.org/abs/1802.02611
|
@@ -162,6 +169,8 @@ Several (less common) features that I often utilize in my projects are included.
|
162 | 169 | * `lookahead` adapted from impl by [Liam](https://github.com/alphadl/lookahead.pytorch) (https://arxiv.org/abs/1907.08610)
|
163 | 170 | * `fused<name>` optimizers by name with [NVIDIA Apex](https://github.com/NVIDIA/apex/tree/master/apex/optimizers) installed
|
164 | 171 | * `adamp` and `sgdp` by [Naver ClovAI](https://github.com/clovaai) (https://arxiv.org/abs/2006.08217)
|
| 172 | + * `adafactor` adapted from [FAIRSeq impl](https://github.com/pytorch/fairseq/blob/master/fairseq/optim/adafactor.py) (https://arxiv.org/abs/1804.04235) |
| 173 | + * `adahessian` by [David Samuel](https://github.com/davda54/ada-hessian) (https://arxiv.org/abs/2006.00719) |
165 | 174 | * Random Erasing from [Zhun Zhong](https://github.com/zhunzhong07/Random-Erasing/blob/master/transforms.py) (https://arxiv.org/abs/1708.04896)
|
166 | 175 | * Mixup (https://arxiv.org/abs/1710.09412)
|
167 | 176 | * CutMix (https://arxiv.org/abs/1905.04899)
|
|
0 commit comments