Regarding pretrained models

[ofir1080]

Hi,
Looking at ViT pretrained versions, I have been trying to understand which datasets it was trained on.
As mentioned here, The models' weights are trained on IM (1/21k?)
But here, it says JFT-300M + ImageNet.
A clarification would be very appreciated.
Thanks!

[ofir1080]

@rwightman
Thanks!

[rwightman]

@ofir1080 I thought I replied to this bug I guess not. They are (generally) the 21k pretrained -> 1k fine-tuned weights for all Google sourced models. I am working on multi-weight support (via text tags in the model name and/or extra args) and will have support for both at some point ... ie vit_base_patch16_224:jax_21kft1k vit_base_patch16_224:jax_1k or something along those lines

[rwightman]

my current approach of having different model entry point fns for different pretrain is not scalable so I've resisted using it extensively

[ofir1080]

I think things are more cleared up now.
Thanks for your answer!