Experimental global target bitsโ€‘perโ€‘weight quantization of unsloth/Qwen3-Embedding-0.6B

  • Using non-standard (forked) LLaMA C++ branch for quantization.
  • Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
  • Using dataset sources: tools, text_en, text_ru.
  • Using dataset chunks: 250.
  • Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
  • Small set of patches added.

Many thanks to Ed Addario for an impressive job.

Quantization comparison

BPW PPL correlation PPL mean ratio ฮ”PPL Mean KLD Maximum KLD 99.9% KLD Mean ฮ”p RMS ฮ”p
5.00 97.51% 1.229589 ยฑ 0.004913 123.695152 ยฑ 3.455547 0.254131 ยฑ 0.001247 13.281865 3.295894 -0.945 ยฑ 0.030 % 7.861 ยฑ 0.073 %
5.25 97.93% 1.198584 ยฑ 0.004387 106.990729 ยฑ 3.089464 0.201051 ยฑ 0.001020 16.191372 2.680833 -0.726 ยฑ 0.027 % 7.070 ยฑ 0.070 %
5.30 98.05% 1.199072 ยฑ 0.004269 107.253986 ยฑ 3.067627 0.181907 ยฑ 0.000926 11.642469 2.371547 -0.680 ยฑ 0.026 % 6.811 ยฑ 0.067 %
5.50 98.42% 1.143577 ยฑ 0.003665 77.354913 ยฑ 2.488164 0.132384 ยฑ 0.000693 9.056372 1.854777 -0.483 ยฑ 0.022 % 5.860 ยฑ 0.061 %
5.75 98.67% 1.107662 ยฑ 0.003264 58.005137 ยฑ 2.117964 0.097616 ยฑ 0.000599 11.806184 1.765460 -0.368 ยฑ 0.019 % 5.023 ยฑ 0.062 %
5.80 98.72% 1.117510 ยฑ 0.003238 63.310457 ยฑ 2.168267 0.092513 ยฑ 0.000578 10.540298 1.608747 -0.386 ยฑ 0.019 % 4.906 ยฑ 0.059 %
6.00 98.71% 1.125959 ยฑ 0.003273 67.862792 ยฑ 2.234984 0.092148 ยฑ 0.000585 12.128985 1.559407 -0.435 ยฑ 0.019 % 4.931 ยฑ 0.061 %
6.25 98.89% 1.082024 ยฑ 0.002912 44.191950 ยฑ 1.831029 0.067634 ยฑ 0.000431 7.849868 1.006397 -0.264 ยฑ 0.016 % 4.210 ยฑ 0.055 %
6.30 98.94% 1.089430 ยฑ 0.002886 48.181944 ยฑ 1.865568 0.062879 ยฑ 0.000377 7.467944 0.950639 -0.262 ยฑ 0.016 % 4.112 ยฑ 0.055 %
6.50 99.07% 1.110195 ยฑ 0.002773 59.369600 ยฑ 1.962020 0.046909 ยฑ 0.000273 5.759321 0.702734 -0.293 ยฑ 0.013 % 3.510 ยฑ 0.044 %
6.75 99.22% 1.071155 ยฑ 0.002479 38.336145 ยฑ 1.614146 0.027021 ยฑ 0.000217 6.292867 0.423712 -0.069 ยฑ 0.011 % 2.756 ยฑ 0.051 %
6.80 99.23% 1.079566 ยฑ 0.002483 42.867759 ยฑ 1.667153 0.026098 ยฑ 0.000205 6.737493 0.397282 -0.098 ยฑ 0.010 % 2.714 ยฑ 0.051 %
7.00 99.24% 1.083987 ยฑ 0.002490 45.249386 ยฑ 1.706124 0.023685 ยฑ 0.000190 6.608492 0.377540 -0.084 ยฑ 0.010 % 2.559 ยฑ 0.049 %
7.25 99.27% 1.088304 ยฑ 0.002447 47.575202 ยฑ 1.717173 0.019738 ยฑ 0.000130 2.865925 0.332578 -0.090 ยฑ 0.009 % 2.290 ยฑ 0.037 %
7.30 99.28% 1.084746 ยฑ 0.002430 45.658594 ยฑ 1.686292 0.019120 ยฑ 0.000148 4.343514 0.295245 -0.092 ยฑ 0.009 % 2.291 ยฑ 0.048 %
7.50 99.29% 1.085544 ยฑ 0.002411 46.088344 ยฑ 1.685529 0.017551 ยฑ 0.000142 4.905227 0.298486 -0.078 ยฑ 0.008 % 2.215 ยฑ 0.050 %
7.75 99.32% 1.091225 ยฑ 0.002385 49.149079 ยฑ 1.714694 0.014173 ยฑ 0.000128 4.346512 0.237379 -0.101 ยฑ 0.008 % 2.013 ยฑ 0.052 %
7.80 99.29% 1.083209 ยฑ 0.002410 44.830380 ยฑ 1.669455 0.016992 ยฑ 0.000133 3.100509 0.285571 -0.077 ยฑ 0.008 % 2.162 ยฑ 0.041 %
8.00 99.31% 1.080260 ยฑ 0.002373 43.241571 ยฑ 1.636024 0.015367 ยฑ 0.000130 4.035001 0.269038 -0.060 ยฑ 0.008 % 2.071 ยฑ 0.049 %
8.25 99.33% 1.088309 ยฑ 0.002352 47.578112 ยฑ 1.683232 0.012024 ยฑ 0.000105 4.370751 0.192501 -0.089 ยฑ 0.007 % 1.860 ยฑ 0.044 %
8.30 99.34% 1.081929 ยฑ 0.002328 44.140973 ยฑ 1.628892 0.011498 ยฑ 0.000084 1.861967 0.182563 -0.075 ยฑ 0.007 % 1.784 ยฑ 0.037 %
8.50 99.36% 1.078487 ยฑ 0.002286 42.286422 ยฑ 1.587902 0.009095 ยฑ 0.000081 2.475720 0.147575 -0.064 ยฑ 0.006 % 1.635 ยฑ 0.048 %
8.75 99.37% 1.078905 ยฑ 0.002274 42.511791 ยฑ 1.587059 0.007852 ยฑ 0.000073 2.163608 0.118868 -0.054 ยฑ 0.006 % 1.535 ยฑ 0.049 %
8.80 99.37% 1.078538 ยฑ 0.002273 42.314029 ยฑ 1.583733 0.007742 ยฑ 0.000079 2.609957 0.127498 -0.053 ยฑ 0.006 % 1.533 ยฑ 0.053 %
9.00 99.37% 1.077841 ยฑ 0.002265 41.938514 ยฑ 1.575722 0.007380 ยฑ 0.000073 2.547039 0.125235 -0.050 ยฑ 0.006 % 1.499 ยฑ 0.053 %
9.25 99.37% 1.075351 ยฑ 0.002256 40.596804 ยฑ 1.555539 0.006905 ยฑ 0.000066 2.041534 0.103068 -0.042 ยฑ 0.006 % 1.448 ยฑ 0.044 %
9.30 99.37% 1.071370 ยฑ 0.002244 38.451639 ยฑ 1.524417 0.006834 ยฑ 0.000062 1.959199 0.105753 -0.031 ยฑ 0.006 % 1.444 ยฑ 0.040 %
9.50 99.38% 1.073834 ยฑ 0.002244 39.779313 ยฑ 1.540846 0.006470 ยฑ 0.000062 1.991340 0.098428 -0.041 ยฑ 0.005 % 1.400 ยฑ 0.041 %
9.75 99.38% 1.074517 ยฑ 0.002240 40.147460 ยฑ 1.543907 0.006206 ยฑ 0.000068 2.638701 0.101805 -0.038 ยฑ 0.005 % 1.388 ยฑ 0.053 %
9.80 99.38% 1.076663 ยฑ 0.002244 41.303655 ยฑ 1.559774 0.006146 ยฑ 0.000069 2.405736 0.094807 -0.044 ยฑ 0.005 % 1.401 ยฑ 0.057 %
10.00 99.38% 1.076016 ยฑ 0.002240 40.955259 ยฑ 1.553864 0.005796 ยฑ 0.000055 2.059185 0.095681 -0.049 ยฑ 0.005 % 1.313 ยฑ 0.039 %
10.25 99.39% 1.074654 ยฑ 0.002230 40.220957 ยฑ 1.540922 0.005526 ยฑ 0.000061 2.114943 0.099634 -0.043 ยฑ 0.005 % 1.315 ยฑ 0.050 %
10.30 99.39% 1.072069 ยฑ 0.002222 38.828737 ยฑ 1.519978 0.005418 ยฑ 0.000059 2.062696 0.094137 -0.038 ยฑ 0.005 % 1.288 ยฑ 0.047 %
10.50 99.39% 1.070241 ยฑ 0.002212 37.843815 ยฑ 1.503987 0.005140 ยฑ 0.000048 1.621503 0.087822 -0.032 ยฑ 0.005 % 1.218 ยฑ 0.028 %
10.75 99.39% 1.066075 ยฑ 0.002195 35.599290 ยฑ 1.470596 0.004415 ยฑ 0.000060 2.666409 0.074945 -0.011 ยฑ 0.004 % 1.152 ยฑ 0.049 %
10.80 99.40% 1.066699 ยฑ 0.002195 35.935359 ยฑ 1.473675 0.004566 ยฑ 0.000051 2.669465 0.074244 -0.025 ยฑ 0.004 % 1.109 ยฑ 0.017 %
11.00 99.40% 1.071004 ยฑ 0.002207 38.254894 ยฑ 1.509298 0.004029 ยฑ 0.000035 1.280535 0.060520 -0.019 ยฑ 0.004 % 1.112 ยฑ 0.045 %
11.25 99.40% 1.071390 ยฑ 0.002201 38.462580 ยฑ 1.508764 0.003687 ยฑ 0.000024 0.614354 0.054238 -0.025 ยฑ 0.004 % 0.995 ยฑ 0.011 %
11.30 99.40% 1.071772 ยฑ 0.002202 38.668624 ยฑ 1.511744 0.003665 ยฑ 0.000024 0.613789 0.055209 -0.023 ยฑ 0.004 % 0.984 ยฑ 0.011 %
11.50 99.40% 1.072891 ยฑ 0.002200 39.271504 ยฑ 1.518324 0.003498 ยฑ 0.000022 0.426572 0.052573 -0.031 ยฑ 0.004 % 0.977 ยฑ 0.013 %
11.75 99.41% 1.067707 ยฑ 0.002184 36.478332 ยฑ 1.478998 0.002967 ยฑ 0.000023 0.803354 0.045490 -0.001 ยฑ 0.003 % 0.895 ยฑ 0.013 %
11.80 99.41% 1.065799 ยฑ 0.002180 35.450545 ยฑ 1.465811 0.002931 ยฑ 0.000021 0.697018 0.045669 0.006 ยฑ 0.003 % 0.899 ยฑ 0.018 %
Downloads last month
150
GGUF
Model size
0.6B params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF

Quantized
(3)
this model

Dataset used to train ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF