Skip to content

Commit 8587287

Browse files
committed
Update README again. Add 101x3 BiT-M model to CI ignore since it's starting to fail in GitHub runners.
1 parent f606c45 commit 8587287

File tree

4 files changed

+47
-12
lines changed

4 files changed

+47
-12
lines changed

README.md

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ I'm fortunate to be able to dedicate significant time and money of my own suppor
2323

2424
## What's New
2525

26+
### April 13, 2021
27+
* Add Swin Transformer models and weights from https://github.com/microsoft/Swin-Transformer
28+
2629
### April 12, 2021
2730
* Add ECA-NFNet-L1 (slimmed down F1 w/ SiLU, 41M params) trained with this code. 84% top-1 @ 320x320. Trained at 256x256.
2831
* Add EfficientNet-V2S model (unverified model definition) weights. 83.3 top-1 @ 288x288. Only trained single res 224. Working on progressive training.
@@ -164,17 +167,6 @@ I'm fortunate to be able to dedicate significant time and money of my own suppor
164167
* EdgeTPU-M (`efficientnet_em`) model trained in PyTorch, 79.3 top-1
165168
* Pip release, doc updates pending a few more changes...
166169

167-
### Sept 18, 2020
168-
* New ResNet 'D' weights. 72.7 (top-1) ResNet-18-D, 77.1 ResNet-34-D, 80.5 ResNet-50-D
169-
* Added a few untrained defs for other ResNet models (66D, 101D, 152D, 200/200D)
170-
171-
### Sept 3, 2020
172-
* New weights
173-
* Wide-ResNet50 - 81.5 top-1 (vs 78.5 torchvision)
174-
* SEResNeXt50-32x4d - 81.3 top-1 (vs 79.1 cadene)
175-
* Support for native Torch AMP and channels_last memory format added to train/validate scripts (`--channels-last`, `--native-amp` vs `--apex-amp`)
176-
* Models tested with channels_last on latest NGC 20.08 container. AdaptiveAvgPool in attn layers changed to mean((2,3)) to work around bug with NHWC kernel.
177-
178170

179171
## Introduction
180172

@@ -189,6 +181,7 @@ All model architecture families include variants with pretrained weights. There
189181
A full version of the list below with source links can be found in the [documentation](https://rwightman.github.io/pytorch-image-models/models/).
190182

191183
* Big Transfer ResNetV2 (BiT) - https://arxiv.org/abs/1912.11370
184+
* Bottleneck Transformers - https://arxiv.org/abs/2101.11605
192185
* CspNet (Cross-Stage Partial Networks) - https://arxiv.org/abs/1911.11929
193186
* DeiT (Vision Transformer) - https://arxiv.org/abs/2012.12877
194187
* DenseNet - https://arxiv.org/abs/1608.06993
@@ -205,10 +198,12 @@ A full version of the list below with source links can be found in the [document
205198
* MobileNet-V2 - https://arxiv.org/abs/1801.04381
206199
* Single-Path NAS - https://arxiv.org/abs/1904.02877
207200
* GPU-Efficient Networks - https://arxiv.org/abs/2006.14090
201+
* Halo Nets - https://arxiv.org/abs/2103.12731
208202
* HardCoRe-NAS - https://arxiv.org/abs/2102.11646
209203
* HRNet - https://arxiv.org/abs/1908.07919
210204
* Inception-V3 - https://arxiv.org/abs/1512.00567
211205
* Inception-ResNet-V2 and Inception-V4 - https://arxiv.org/abs/1602.07261
206+
* Lambda Networks - https://arxiv.org/abs/2102.08602
212207
* MobileNet-V3 (MBConvNet w/ Efficient Head) - https://arxiv.org/abs/1905.02244
213208
* NASNet-A - https://arxiv.org/abs/1707.07012
214209
* NFNet-F - https://arxiv.org/abs/2102.06171
@@ -230,6 +225,7 @@ A full version of the list below with source links can be found in the [document
230225
* ReXNet - https://arxiv.org/abs/2007.00992
231226
* SelecSLS - https://arxiv.org/abs/1907.00837
232227
* Selective Kernel Networks - https://arxiv.org/abs/1903.06586
228+
* Swin Transformer - https://arxiv.org/abs/2103.14030
233229
* Transformer-iN-Transformer (TNT) - https://arxiv.org/abs/2103.00112
234230
* TResNet - https://arxiv.org/abs/2003.13630
235231
* Vision Transformer - https://arxiv.org/abs/2010.11929

docs/archived_changes.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,16 @@
11
# Archived Changes
22

3+
### Sept 18, 2020
4+
* New ResNet 'D' weights. 72.7 (top-1) ResNet-18-D, 77.1 ResNet-34-D, 80.5 ResNet-50-D
5+
* Added a few untrained defs for other ResNet models (66D, 101D, 152D, 200/200D)
6+
7+
### Sept 3, 2020
8+
* New weights
9+
* Wide-ResNet50 - 81.5 top-1 (vs 78.5 torchvision)
10+
* SEResNeXt50-32x4d - 81.3 top-1 (vs 79.1 cadene)
11+
* Support for native Torch AMP and channels_last memory format added to train/validate scripts (`--channels-last`, `--native-amp` vs `--apex-amp`)
12+
* Models tested with channels_last on latest NGC 20.08 container. AdaptiveAvgPool in attn layers changed to mean((2,3)) to work around bug with NHWC kernel.
13+
314
### Aug 12, 2020
415
* New/updated weights from training experiments
516
* EfficientNet-B3 - 82.1 top-1 (vs 81.6 for official with AA and 81.9 for AdvProp)

docs/changes.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,33 @@
11
# Recent Changes
22

3+
### April 13, 2021
4+
* Add Swin Transformer models and weights from https://github.com/microsoft/Swin-Transformer
5+
6+
### April 12, 2021
7+
* Add ECA-NFNet-L1 (slimmed down F1 w/ SiLU, 41M params) trained with this code. 84% top-1 @ 320x320. Trained at 256x256.
8+
* Add EfficientNet-V2S model (unverified model definition) weights. 83.3 top-1 @ 288x288. Only trained single res 224. Working on progressive training.
9+
* Add ByoaNet model definition (Bring-your-own-attention) w/ SelfAttention block and corresponding SA/SA-like modules and model defs
10+
* Lambda Networks - https://arxiv.org/abs/2102.08602
11+
* Bottleneck Transformers - https://arxiv.org/abs/2101.11605
12+
* Halo Nets - https://arxiv.org/abs/2103.12731
13+
* Adabelief optimizer contributed by Juntang Zhuang
14+
15+
### April 1, 2021
16+
* Add snazzy `benchmark.py` script for bulk `timm` model benchmarking of train and/or inference
17+
* Add Pooling-based Vision Transformer (PiT) models (from https://github.com/naver-ai/pit)
18+
* Merged distilled variant into main for torchscript compatibility
19+
* Some `timm` cleanup/style tweaks and weights have hub download support
20+
* Cleanup Vision Transformer (ViT) models
21+
* Merge distilled (DeiT) model into main so that torchscript can work
22+
* Support updated weight init (defaults to old still) that closer matches original JAX impl (possibly better training from scratch)
23+
* Separate hybrid model defs into different file and add several new model defs to fiddle with, support patch_size != 1 for hybrids
24+
* Fix fine-tuning num_class changes (PiT and ViT) and pos_embed resizing (Vit) with distilled variants
25+
* nn.Sequential for block stack (does not break downstream compat)
26+
* TnT (Transformer-in-Transformer) models contributed by author (from https://gitee.com/mindspore/mindspore/tree/master/model_zoo/research/cv/TNT)
27+
* Add RegNetY-160 weights from DeiT teacher model
28+
* Add new NFNet-L0 w/ SE attn (rename `nfnet_l0b`->`nfnet_l0`) weights 82.75 top-1 @ 288x288
29+
* Some fixes/improvements for TFDS dataset wrapper
30+
331
### March 7, 2021
432
* First 0.4.x PyPi release w/ NFNets (& related), ByoB (GPU-Efficient, RepVGG, etc).
533
* Change feature extraction for pre-activation nets (NFNets, ResNetV2) to return features before activation.

tests/test_models.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
if 'GITHUB_ACTIONS' in os.environ: # and 'Linux' in platform.system():
2323
# GitHub Linux runner is slower and hits memory limits sooner than MacOS, exclude bigger models
2424
EXCLUDE_FILTERS = [
25-
'*efficientnet_l2*', '*resnext101_32x48d', '*in21k', '*152x4_bitm',
25+
'*efficientnet_l2*', '*resnext101_32x48d', '*in21k', '*152x4_bitm', '*101x3_bitm'
2626
'*nfnet_f3*', '*nfnet_f4*', '*nfnet_f5*', '*nfnet_f6*', '*nfnet_f7*'] + NON_STD_FILTERS
2727
else:
2828
EXCLUDE_FILTERS = NON_STD_FILTERS

0 commit comments

Comments
 (0)