Different default behavior of get_intermediate_layers with the official DINOv2 repo

[IcarusWizard]

I am sure it is the right place to put this issue. But I think it is worth to document it somewhere as DINOv2 is a popular pretrained model these days.

The ViT backbone provides an API get_intermediate_layers to compute the feature from intermediate layers same as the official DINOv2 repo. The implementation is almost the same but the default parameter of norm is different between the DINOv2 repo and TIMM implementation (True for DINOv2 repo, False for TIMM repo). This makes the scale of the output different and may infect the performance when transferring one implementation to another.

I am not sure whether we should change the default parameter in TIMM as the API may already been used in some popular repos.

[rwightman]

model.get_intermdiate_layers(x, norm=True) (or False) works for both and forces their equivalence. The timm impl covers a lot of weights aside from just dinov2, I didn't think the norm as a default is desired, it's applying the norm that is trained in most models as a pre-classifier norm to all feature levels.