eaddario/imatrix-calibration
Viewer โข Updated โข 299 โข 9.43k โข 78
How to use ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF")
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium."
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]How to use ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("feature-extraction", model="ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF", device_map="auto")How to use ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF with llama.cpp:
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF --embeddings
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF --embeddings
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF --embeddings
How to use ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF with Ollama:
ollama run hf.co/ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF
How to use ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF with Docker Model Runner:
docker model run hf.co/ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF
How to use ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF with Lemonade:
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF
lemonade run user.Qwen3-Embedding-0.6B-250-v1-GGUF-{{QUANT_TAG}}lemonade list
Many thanks to Ed Addario for an impressive job.
| BPW | PPL correlation | PPL mean ratio | ฮPPL | Mean KLD | Maximum KLD | 99.9% KLD | Mean ฮp | RMS ฮp |
|---|---|---|---|---|---|---|---|---|
| 5.00 | 97.51% | 1.229589 ยฑ 0.004913 | 123.695152 ยฑ 3.455547 | 0.254131 ยฑ 0.001247 | 13.281865 | 3.295894 | -0.945 ยฑ 0.030 % | 7.861 ยฑ 0.073 % |
| 5.25 | 97.93% | 1.198584 ยฑ 0.004387 | 106.990729 ยฑ 3.089464 | 0.201051 ยฑ 0.001020 | 16.191372 | 2.680833 | -0.726 ยฑ 0.027 % | 7.070 ยฑ 0.070 % |
| 5.30 | 98.05% | 1.199072 ยฑ 0.004269 | 107.253986 ยฑ 3.067627 | 0.181907 ยฑ 0.000926 | 11.642469 | 2.371547 | -0.680 ยฑ 0.026 % | 6.811 ยฑ 0.067 % |
| 5.50 | 98.42% | 1.143577 ยฑ 0.003665 | 77.354913 ยฑ 2.488164 | 0.132384 ยฑ 0.000693 | 9.056372 | 1.854777 | -0.483 ยฑ 0.022 % | 5.860 ยฑ 0.061 % |
| 5.75 | 98.67% | 1.107662 ยฑ 0.003264 | 58.005137 ยฑ 2.117964 | 0.097616 ยฑ 0.000599 | 11.806184 | 1.765460 | -0.368 ยฑ 0.019 % | 5.023 ยฑ 0.062 % |
| 5.80 | 98.72% | 1.117510 ยฑ 0.003238 | 63.310457 ยฑ 2.168267 | 0.092513 ยฑ 0.000578 | 10.540298 | 1.608747 | -0.386 ยฑ 0.019 % | 4.906 ยฑ 0.059 % |
| 6.00 | 98.71% | 1.125959 ยฑ 0.003273 | 67.862792 ยฑ 2.234984 | 0.092148 ยฑ 0.000585 | 12.128985 | 1.559407 | -0.435 ยฑ 0.019 % | 4.931 ยฑ 0.061 % |
| 6.25 | 98.89% | 1.082024 ยฑ 0.002912 | 44.191950 ยฑ 1.831029 | 0.067634 ยฑ 0.000431 | 7.849868 | 1.006397 | -0.264 ยฑ 0.016 % | 4.210 ยฑ 0.055 % |
| 6.30 | 98.94% | 1.089430 ยฑ 0.002886 | 48.181944 ยฑ 1.865568 | 0.062879 ยฑ 0.000377 | 7.467944 | 0.950639 | -0.262 ยฑ 0.016 % | 4.112 ยฑ 0.055 % |
| 6.50 | 99.07% | 1.110195 ยฑ 0.002773 | 59.369600 ยฑ 1.962020 | 0.046909 ยฑ 0.000273 | 5.759321 | 0.702734 | -0.293 ยฑ 0.013 % | 3.510 ยฑ 0.044 % |
| 6.75 | 99.22% | 1.071155 ยฑ 0.002479 | 38.336145 ยฑ 1.614146 | 0.027021 ยฑ 0.000217 | 6.292867 | 0.423712 | -0.069 ยฑ 0.011 % | 2.756 ยฑ 0.051 % |
| 6.80 | 99.23% | 1.079566 ยฑ 0.002483 | 42.867759 ยฑ 1.667153 | 0.026098 ยฑ 0.000205 | 6.737493 | 0.397282 | -0.098 ยฑ 0.010 % | 2.714 ยฑ 0.051 % |
| 7.00 | 99.24% | 1.083987 ยฑ 0.002490 | 45.249386 ยฑ 1.706124 | 0.023685 ยฑ 0.000190 | 6.608492 | 0.377540 | -0.084 ยฑ 0.010 % | 2.559 ยฑ 0.049 % |
| 7.25 | 99.27% | 1.088304 ยฑ 0.002447 | 47.575202 ยฑ 1.717173 | 0.019738 ยฑ 0.000130 | 2.865925 | 0.332578 | -0.090 ยฑ 0.009 % | 2.290 ยฑ 0.037 % |
| 7.30 | 99.28% | 1.084746 ยฑ 0.002430 | 45.658594 ยฑ 1.686292 | 0.019120 ยฑ 0.000148 | 4.343514 | 0.295245 | -0.092 ยฑ 0.009 % | 2.291 ยฑ 0.048 % |
| 7.50 | 99.29% | 1.085544 ยฑ 0.002411 | 46.088344 ยฑ 1.685529 | 0.017551 ยฑ 0.000142 | 4.905227 | 0.298486 | -0.078 ยฑ 0.008 % | 2.215 ยฑ 0.050 % |
| 7.75 | 99.32% | 1.091225 ยฑ 0.002385 | 49.149079 ยฑ 1.714694 | 0.014173 ยฑ 0.000128 | 4.346512 | 0.237379 | -0.101 ยฑ 0.008 % | 2.013 ยฑ 0.052 % |
| 7.80 | 99.29% | 1.083209 ยฑ 0.002410 | 44.830380 ยฑ 1.669455 | 0.016992 ยฑ 0.000133 | 3.100509 | 0.285571 | -0.077 ยฑ 0.008 % | 2.162 ยฑ 0.041 % |
| 8.00 | 99.31% | 1.080260 ยฑ 0.002373 | 43.241571 ยฑ 1.636024 | 0.015367 ยฑ 0.000130 | 4.035001 | 0.269038 | -0.060 ยฑ 0.008 % | 2.071 ยฑ 0.049 % |
| 8.25 | 99.33% | 1.088309 ยฑ 0.002352 | 47.578112 ยฑ 1.683232 | 0.012024 ยฑ 0.000105 | 4.370751 | 0.192501 | -0.089 ยฑ 0.007 % | 1.860 ยฑ 0.044 % |
| 8.30 | 99.34% | 1.081929 ยฑ 0.002328 | 44.140973 ยฑ 1.628892 | 0.011498 ยฑ 0.000084 | 1.861967 | 0.182563 | -0.075 ยฑ 0.007 % | 1.784 ยฑ 0.037 % |
| 8.50 | 99.36% | 1.078487 ยฑ 0.002286 | 42.286422 ยฑ 1.587902 | 0.009095 ยฑ 0.000081 | 2.475720 | 0.147575 | -0.064 ยฑ 0.006 % | 1.635 ยฑ 0.048 % |
| 8.75 | 99.37% | 1.078905 ยฑ 0.002274 | 42.511791 ยฑ 1.587059 | 0.007852 ยฑ 0.000073 | 2.163608 | 0.118868 | -0.054 ยฑ 0.006 % | 1.535 ยฑ 0.049 % |
| 8.80 | 99.37% | 1.078538 ยฑ 0.002273 | 42.314029 ยฑ 1.583733 | 0.007742 ยฑ 0.000079 | 2.609957 | 0.127498 | -0.053 ยฑ 0.006 % | 1.533 ยฑ 0.053 % |
| 9.00 | 99.37% | 1.077841 ยฑ 0.002265 | 41.938514 ยฑ 1.575722 | 0.007380 ยฑ 0.000073 | 2.547039 | 0.125235 | -0.050 ยฑ 0.006 % | 1.499 ยฑ 0.053 % |
| 9.25 | 99.37% | 1.075351 ยฑ 0.002256 | 40.596804 ยฑ 1.555539 | 0.006905 ยฑ 0.000066 | 2.041534 | 0.103068 | -0.042 ยฑ 0.006 % | 1.448 ยฑ 0.044 % |
| 9.30 | 99.37% | 1.071370 ยฑ 0.002244 | 38.451639 ยฑ 1.524417 | 0.006834 ยฑ 0.000062 | 1.959199 | 0.105753 | -0.031 ยฑ 0.006 % | 1.444 ยฑ 0.040 % |
| 9.50 | 99.38% | 1.073834 ยฑ 0.002244 | 39.779313 ยฑ 1.540846 | 0.006470 ยฑ 0.000062 | 1.991340 | 0.098428 | -0.041 ยฑ 0.005 % | 1.400 ยฑ 0.041 % |
| 9.75 | 99.38% | 1.074517 ยฑ 0.002240 | 40.147460 ยฑ 1.543907 | 0.006206 ยฑ 0.000068 | 2.638701 | 0.101805 | -0.038 ยฑ 0.005 % | 1.388 ยฑ 0.053 % |
| 9.80 | 99.38% | 1.076663 ยฑ 0.002244 | 41.303655 ยฑ 1.559774 | 0.006146 ยฑ 0.000069 | 2.405736 | 0.094807 | -0.044 ยฑ 0.005 % | 1.401 ยฑ 0.057 % |
| 10.00 | 99.38% | 1.076016 ยฑ 0.002240 | 40.955259 ยฑ 1.553864 | 0.005796 ยฑ 0.000055 | 2.059185 | 0.095681 | -0.049 ยฑ 0.005 % | 1.313 ยฑ 0.039 % |
| 10.25 | 99.39% | 1.074654 ยฑ 0.002230 | 40.220957 ยฑ 1.540922 | 0.005526 ยฑ 0.000061 | 2.114943 | 0.099634 | -0.043 ยฑ 0.005 % | 1.315 ยฑ 0.050 % |
| 10.30 | 99.39% | 1.072069 ยฑ 0.002222 | 38.828737 ยฑ 1.519978 | 0.005418 ยฑ 0.000059 | 2.062696 | 0.094137 | -0.038 ยฑ 0.005 % | 1.288 ยฑ 0.047 % |
| 10.50 | 99.39% | 1.070241 ยฑ 0.002212 | 37.843815 ยฑ 1.503987 | 0.005140 ยฑ 0.000048 | 1.621503 | 0.087822 | -0.032 ยฑ 0.005 % | 1.218 ยฑ 0.028 % |
| 10.75 | 99.39% | 1.066075 ยฑ 0.002195 | 35.599290 ยฑ 1.470596 | 0.004415 ยฑ 0.000060 | 2.666409 | 0.074945 | -0.011 ยฑ 0.004 % | 1.152 ยฑ 0.049 % |
| 10.80 | 99.40% | 1.066699 ยฑ 0.002195 | 35.935359 ยฑ 1.473675 | 0.004566 ยฑ 0.000051 | 2.669465 | 0.074244 | -0.025 ยฑ 0.004 % | 1.109 ยฑ 0.017 % |
| 11.00 | 99.40% | 1.071004 ยฑ 0.002207 | 38.254894 ยฑ 1.509298 | 0.004029 ยฑ 0.000035 | 1.280535 | 0.060520 | -0.019 ยฑ 0.004 % | 1.112 ยฑ 0.045 % |
| 11.25 | 99.40% | 1.071390 ยฑ 0.002201 | 38.462580 ยฑ 1.508764 | 0.003687 ยฑ 0.000024 | 0.614354 | 0.054238 | -0.025 ยฑ 0.004 % | 0.995 ยฑ 0.011 % |
| 11.30 | 99.40% | 1.071772 ยฑ 0.002202 | 38.668624 ยฑ 1.511744 | 0.003665 ยฑ 0.000024 | 0.613789 | 0.055209 | -0.023 ยฑ 0.004 % | 0.984 ยฑ 0.011 % |
| 11.50 | 99.40% | 1.072891 ยฑ 0.002200 | 39.271504 ยฑ 1.518324 | 0.003498 ยฑ 0.000022 | 0.426572 | 0.052573 | -0.031 ยฑ 0.004 % | 0.977 ยฑ 0.013 % |
| 11.75 | 99.41% | 1.067707 ยฑ 0.002184 | 36.478332 ยฑ 1.478998 | 0.002967 ยฑ 0.000023 | 0.803354 | 0.045490 | -0.001 ยฑ 0.003 % | 0.895 ยฑ 0.013 % |
| 11.80 | 99.41% | 1.065799 ยฑ 0.002180 | 35.450545 ยฑ 1.465811 | 0.002931 ยฑ 0.000021 | 0.697018 | 0.045669 | 0.006 ยฑ 0.003 % | 0.899 ยฑ 0.018 % |
We're not able to determine the quantization variants.