Hugging Face's logo

zeromodels
/
vit_small_patch16_224_augreg_in1k

Image Classification
ZeroModels
Keras
PyTorch
JAX
TensorFlow
vit
backbone
Model card Files
xet
 Community
IMvision12 commited on
Commit
2652cdc
·
verified ·
1 Parent(s): 37da241

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +102 -102
README.md CHANGED
@@ -1,102 +1,102 @@
1
- ---
2
- pipeline_tag: image-classification
3
- license: apache-2.0
4
- base_model: timm/vit_small_patch16_224.augreg_in1k
5
- library_name: zeromodels
6
- tags:
7
- - keras
8
- - zeromodels
9
- - image-classification
10
- - vit
11
- - backbone
12
- - arxiv:2010.11929
13
- - pytorch
14
- - jax
15
- - tf
16
- ---
17
-
18
- ## ***See [our collection](https://huggingface.co/collections/zeromodels/vit-6a8eae63ab67c7a19fb7a6da) for all versions of ViT.***
19
-
20
- # Run ViT with Keras 3: JAX, PyTorch, or TensorFlow
21
-
22
- [![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-ViT-blue)](https://imvision12.github.io/ZeroModels/classification_backbones/) [![Collection](https://img.shields.io/badge/HF-ViT%20collection-yellow)](https://huggingface.co/collections/zeromodels/vit-6a8eae63ab67c7a19fb7a6da)
23
-
24
- # zeromodels/vit_small_patch16_224_augreg_in1k
25
-
26
- Paper: [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (arXiv:2010.11929)](https://arxiv.org/abs/2010.11929) · [HF Papers](https://huggingface.co/papers/2010.11929)
27
-
28
- Vision Transformer (ViT) patches an image and runs a transformer encoder. Use `ViTImageClassify` for logits or `ViTModel` for tokens / per-block features via `as_backbone=True`.
29
-
30
- For more details on the model, please go to the upstream [model card](https://huggingface.co/timm/vit_small_patch16_224.augreg_in1k).
31
-
32
- Pure-**Keras 3** conversion of [`timm/vit_small_patch16_224.augreg_in1k`](https://huggingface.co/timm/vit_small_patch16_224.augreg_in1k) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
33
-
34
- This is an **image-classification / backbone** checkpoint (`ViTImageClassify` / `ViTModel`).
35
-
36
- ## ✨ Quick start
37
-
38
- ```python
39
- import os
40
- os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
41
-
42
- from PIL import Image
43
- import numpy as np
44
- from zeromodels.models.vit import ViTImageClassify, ViTModel
45
-
46
- model = ViTImageClassify.from_weights("zeromodels/vit_small_patch16_224_augreg_in1k")
47
- backbone = ViTModel.from_weights(
48
- "zeromodels/vit_small_patch16_224_augreg_in1k", as_backbone=True
49
- )
50
-
51
- image = Image.open("your_image.jpg").convert("RGB")
52
- image = image.resize((224, 224))
53
- x = np.asarray(image, dtype="float32")[None] # (1, H, W, 3)
54
- print(model(x).shape) # (1, num_classes)
55
- feats = backbone(x)
56
- print(len(feats), [tuple(f.shape) for f in feats])
57
- ```
58
-
59
- Load any ViT variant the same way with `from_weights("zeromodels/<variant>")`:
60
-
61
- | Variant | Hub |
62
- |---|---|
63
- | `vit_base_patch16_224_augreg_in1k` | [`zeromodels/vit_base_patch16_224_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_224_augreg_in1k) |
64
- | `vit_base_patch16_224_augreg_in21k` | [`zeromodels/vit_base_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_base_patch16_224_augreg_in21k) |
65
- | `vit_base_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_224_augreg_in21k_ft_in1k) |
66
- | `vit_base_patch16_224_orig_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_224_orig_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_224_orig_in21k_ft_in1k) |
67
- | `vit_base_patch16_384_augreg_in1k` | [`zeromodels/vit_base_patch16_384_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_384_augreg_in1k) |
68
- | `vit_base_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_384_augreg_in21k_ft_in1k) |
69
- | `vit_base_patch16_384_orig_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_384_orig_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_384_orig_in21k_ft_in1k) |
70
- | `vit_base_patch32_224_augreg_in1k` | [`zeromodels/vit_base_patch32_224_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_224_augreg_in1k) |
71
- | `vit_base_patch32_224_augreg_in21k` | [`zeromodels/vit_base_patch32_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_base_patch32_224_augreg_in21k) |
72
- | `vit_base_patch32_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch32_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_224_augreg_in21k_ft_in1k) |
73
- | `vit_base_patch32_384_augreg_in1k` | [`zeromodels/vit_base_patch32_384_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_384_augreg_in1k) |
74
- | `vit_base_patch32_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch32_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_384_augreg_in21k_ft_in1k) |
75
- | `vit_large_patch16_224_augreg_in21k` | [`zeromodels/vit_large_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_large_patch16_224_augreg_in21k) |
76
- | `vit_large_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_large_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_large_patch16_224_augreg_in21k_ft_in1k) |
77
- | `vit_large_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_large_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_large_patch16_384_augreg_in21k_ft_in1k) |
78
- | `vit_large_patch32_384_orig_in21k_ft_in1k` | [`zeromodels/vit_large_patch32_384_orig_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_large_patch32_384_orig_in21k_ft_in1k) |
79
- | `vit_small_patch16_224_augreg_in1k` | [`zeromodels/vit_small_patch16_224_augreg_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_224_augreg_in1k) |
80
- | `vit_small_patch16_224_augreg_in21k` | [`zeromodels/vit_small_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_small_patch16_224_augreg_in21k) |
81
- | `vit_small_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_224_augreg_in21k_ft_in1k) |
82
- | `vit_small_patch16_384_augreg_in1k` | [`zeromodels/vit_small_patch16_384_augreg_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_384_augreg_in1k) |
83
- | `vit_small_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_384_augreg_in21k_ft_in1k) |
84
- | `vit_small_patch32_224_augreg_in21k` | [`zeromodels/vit_small_patch32_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_small_patch32_224_augreg_in21k) |
85
- | `vit_small_patch32_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch32_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch32_224_augreg_in21k_ft_in1k) |
86
- | `vit_small_patch32_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch32_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch32_384_augreg_in21k_ft_in1k) |
87
- | `vit_tiny_patch16_224_augreg_in21k` | [`zeromodels/vit_tiny_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_tiny_patch16_224_augreg_in21k) |
88
- | `vit_tiny_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_tiny_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_tiny_patch16_224_augreg_in21k_ft_in1k) |
89
- | `vit_tiny_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_tiny_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_tiny_patch16_384_augreg_in21k_ft_in1k) |
90
-
91
- ## Tips
92
-
93
- - Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
94
- - `ViTImageClassify` returns class logits; `ViTModel` returns features (`as_backbone=True` for multi-scale stages).
95
- - See [docs](https://imvision12.github.io/ZeroModels/classification_backbones/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
96
- - Upstream / timm checkpoints: `ViTImageClassify.from_weights("hf:timm/vit_small_patch16_224.augreg_in1k")`.
97
-
98
- ## Special Thanks
99
-
100
- A huge thank you to the ViT authors and the timm / Hub communities for creating and releasing these models.
101
-
102
- License: see YAML `license` (usually matches the upstream checkpoint).
 
1
+ ---
2
+ pipeline_tag: image-classification
3
+ license: apache-2.0
4
+ base_model: timm/vit_small_patch16_224.augreg_in1k
5
+ library_name: zeromodels
6
+ tags:
7
+ - keras
8
+ - zeromodels
9
+ - image-classification
10
+ - vit
11
+ - backbone
12
+ - arxiv:2010.11929
13
+ - pytorch
14
+ - jax
15
+ - tf
16
+ ---
17
+
18
+ ## ***See [our collection](https://huggingface.co/collections/zeromodels/vit-6a8eae63ab67c7a19fb7a6da) for all versions of ViT.***
19
+
20
+ # Run ViT with Keras 3: JAX, PyTorch, or TensorFlow
21
+
22
+ [![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-ViT-blue)](https://imvision12.github.io/ZeroModels/classification_backbones/) [![Collection](https://img.shields.io/badge/HF-ViT%20collection-yellow)](https://huggingface.co/collections/zeromodels/vit-6a8eae63ab67c7a19fb7a6da)
23
+
24
+ # zeromodels/vit_small_patch16_224_augreg_in1k
25
+
26
+ Paper: [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (arXiv:2010.11929)](https://arxiv.org/abs/2010.11929) · [HF Papers](https://huggingface.co/papers/2010.11929)
27
+
28
+ Vision Transformer (ViT) patches an image and runs a transformer encoder. Use `ViTImageClassify` for logits or `ViTModel` for tokens / per-block features via `as_backbone=True`.
29
+
30
+ For more details on the model, please go to the upstream [model card](https://huggingface.co/timm/vit_small_patch16_224.augreg_in1k).
31
+
32
+ Pure-**Keras 3** conversion of [`timm/vit_small_patch16_224.augreg_in1k`](https://huggingface.co/timm/vit_small_patch16_224.augreg_in1k) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
33
+
34
+ This is an **image-classification / backbone** checkpoint (`ViTImageClassify` / `ViTModel`).
35
+
36
+ ## ✨ Quick start
37
+
38
+ ```python
39
+ import os
40
+
41
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
42
+
43
+ from PIL import Image
44
+ from zeromodels.models.vit import ViTImageClassify, ViTModel, ViTImageProcessor
45
+
46
+ model = ViTImageClassify.from_weights("zeromodels/vit_small_patch16_224_augreg_in1k")
47
+ processor = ViTImageProcessor.from_weights("zeromodels/vit_small_patch16_224_augreg_in1k")
48
+
49
+ image = Image.open("your_image.jpg").convert("RGB")
50
+ pixels = processor(image) # resize + normalize (normalization lives in the processor)
51
+ logits = model(pixels, training=False)
52
+ print(logits.shape) # (1, num_classes)
53
+
54
+ # Feature extraction: the backbone without the classifier head
55
+ backbone = ViTModel.from_weights("zeromodels/vit_small_patch16_224_augreg_in1k", as_backbone=True)
56
+ features = backbone(pixels, training=False)
57
+ ```
58
+
59
+ Load any ViT variant the same way with `from_weights("zeromodels/<variant>")`:
60
+
61
+ | Variant | Hub |
62
+ |---|---|
63
+ | `vit_base_patch16_224_augreg_in1k` | [`zeromodels/vit_base_patch16_224_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_224_augreg_in1k) |
64
+ | `vit_base_patch16_224_augreg_in21k` | [`zeromodels/vit_base_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_base_patch16_224_augreg_in21k) |
65
+ | `vit_base_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_224_augreg_in21k_ft_in1k) |
66
+ | `vit_base_patch16_224_orig_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_224_orig_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_224_orig_in21k_ft_in1k) |
67
+ | `vit_base_patch16_384_augreg_in1k` | [`zeromodels/vit_base_patch16_384_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_384_augreg_in1k) |
68
+ | `vit_base_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_384_augreg_in21k_ft_in1k) |
69
+ | `vit_base_patch16_384_orig_in21k_ft_in1k` | [`zeromodels/vit_base_patch16_384_orig_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch16_384_orig_in21k_ft_in1k) |
70
+ | `vit_base_patch32_224_augreg_in1k` | [`zeromodels/vit_base_patch32_224_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_224_augreg_in1k) |
71
+ | `vit_base_patch32_224_augreg_in21k` | [`zeromodels/vit_base_patch32_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_base_patch32_224_augreg_in21k) |
72
+ | `vit_base_patch32_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch32_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_224_augreg_in21k_ft_in1k) |
73
+ | `vit_base_patch32_384_augreg_in1k` | [`zeromodels/vit_base_patch32_384_augreg_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_384_augreg_in1k) |
74
+ | `vit_base_patch32_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_base_patch32_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_base_patch32_384_augreg_in21k_ft_in1k) |
75
+ | `vit_large_patch16_224_augreg_in21k` | [`zeromodels/vit_large_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_large_patch16_224_augreg_in21k) |
76
+ | `vit_large_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_large_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_large_patch16_224_augreg_in21k_ft_in1k) |
77
+ | `vit_large_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_large_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_large_patch16_384_augreg_in21k_ft_in1k) |
78
+ | `vit_large_patch32_384_orig_in21k_ft_in1k` | [`zeromodels/vit_large_patch32_384_orig_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_large_patch32_384_orig_in21k_ft_in1k) |
79
+ | `vit_small_patch16_224_augreg_in1k` | [`zeromodels/vit_small_patch16_224_augreg_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_224_augreg_in1k) |
80
+ | `vit_small_patch16_224_augreg_in21k` | [`zeromodels/vit_small_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_small_patch16_224_augreg_in21k) |
81
+ | `vit_small_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_224_augreg_in21k_ft_in1k) |
82
+ | `vit_small_patch16_384_augreg_in1k` | [`zeromodels/vit_small_patch16_384_augreg_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_384_augreg_in1k) |
83
+ | `vit_small_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch16_384_augreg_in21k_ft_in1k) |
84
+ | `vit_small_patch32_224_augreg_in21k` | [`zeromodels/vit_small_patch32_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_small_patch32_224_augreg_in21k) |
85
+ | `vit_small_patch32_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch32_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch32_224_augreg_in21k_ft_in1k) |
86
+ | `vit_small_patch32_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_small_patch32_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_small_patch32_384_augreg_in21k_ft_in1k) |
87
+ | `vit_tiny_patch16_224_augreg_in21k` | [`zeromodels/vit_tiny_patch16_224_augreg_in21k`](https://huggingface.co/zeromodels/vit_tiny_patch16_224_augreg_in21k) |
88
+ | `vit_tiny_patch16_224_augreg_in21k_ft_in1k` | [`zeromodels/vit_tiny_patch16_224_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_tiny_patch16_224_augreg_in21k_ft_in1k) |
89
+ | `vit_tiny_patch16_384_augreg_in21k_ft_in1k` | [`zeromodels/vit_tiny_patch16_384_augreg_in21k_ft_in1k`](https://huggingface.co/zeromodels/vit_tiny_patch16_384_augreg_in21k_ft_in1k) |
90
+
91
+ ## Tips
92
+
93
+ - Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
94
+ - `ViTImageClassify` returns class logits; `ViTModel` returns features (`as_backbone=True` for multi-scale stages).
95
+ - See [docs](https://imvision12.github.io/ZeroModels/classification_backbones/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
96
+ - Upstream / timm checkpoints: `ViTImageClassify.from_weights("hf:timm/vit_small_patch16_224.augreg_in1k")`.
97
+
98
+ ## Special Thanks
99
+
100
+ A huge thank you to the ViT authors and the timm / Hub communities for creating and releasing these models.
101
+
102
+ License: see YAML `license` (usually matches the upstream checkpoint).