Gabriele commited on
Commit
c7d0623
·
1 Parent(s): 8e6f865

Update README

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -10,7 +10,7 @@ library_name: transformers
10
  pipeline_tag: zero-shot-image-classification
11
  ---
12
 
13
- # TIPSv2 — G/14
14
 
15
  TIPSv2 (Text-Image Pre-training with Spatial awareness) is a family of contrastive vision-language models that produce spatially rich image features aligned with text embeddings. This is the Giant variant with 1.1B vision params and 389M text params.
16
 
@@ -74,6 +74,17 @@ similarity = cls @ text_emb.T
74
  prediction = similarity.argmax(dim=-1)
75
  ```
76
 
 
 
 
 
 
 
 
 
 
 
 
77
  ### GPU inference
78
 
79
  ```python
 
10
  pipeline_tag: zero-shot-image-classification
11
  ---
12
 
13
+ # TIPSv2 — g/14
14
 
15
  TIPSv2 (Text-Image Pre-training with Spatial awareness) is a family of contrastive vision-language models that produce spatially rich image features aligned with text embeddings. This is the Giant variant with 1.1B vision params and 389M text params.
16
 
 
74
  prediction = similarity.argmax(dim=-1)
75
  ```
76
 
77
+ ### Zero-shot segmentation
78
+
79
+ ```python
80
+ import numpy as np
81
+ from sklearn.decomposition import PCA
82
+
83
+ spatial = out.patch_tokens.reshape(1, 32, 32, 1536)
84
+ feat = spatial[0].detach().numpy().reshape(-1, 1536)
85
+ rgb = PCA(n_components=3).fit_transform(feat).reshape(32, 32, 3)
86
+ ```
87
+
88
  ### GPU inference
89
 
90
  ```python