Using this on vllm but excessive tbinking

#13
by crystech - opened

Like to check if anyone else facing output limits exceeded issues ? It seem like this model overthinks and exceeded output limits . Even when I set to 132k limit for outputs still same

I can confirm this as well.

I can confirm this too,I use DGX Spark.

same here, this model is overthinking pro max...overthinking wasting lots of tokens and time...it is frustrating.

Thinking effort is configurable and defaults to 'xhigh'. You can change the default in VLLM with: --default-chat-template-kwargs '{"reasoning_effort": "low"}'

It is configurable. however, it definitely over thinks on xhigh, to the point that it runs into practical limits of its context window. I have a number of cases where it overthinks to the point of exhausting its context window in only a few turns.

What I will say is this doesn't appear to be an unsloth quant issues: I've replicated the same issues using BF16 and the FP8 from Qwen's official release.

Unsloth org

It is configurable. however, it definitely over thinks on xhigh, to the point that it runs into practical limits of its context window. I have a number of cases where it overthinks to the point of exhausting its context window in only a few turns.

What I will say is this doesn't appear to be an unsloth quant issues: I've replicated the same issues using BF16 and the FP8 from Qwen's official release.

Thinking effort is configurable and defaults to 'xhigh'. You can change the default in VLLM with: --default-chat-template-kwargs '{"reasoning_effort": "low"}'

That makes sense from our tests as well. Looks like the reasoning effort definitely does help. You just need to test more prompts and see the reasoning effort go to effect

It will never not be crazy to me how people expect a 27B to outperform WAAAY larger models without it thinking by alot, like it's crazy. That's how these small models close the gaps and you wanna eliminate it. That's craycray

Unsloth org

It will never not be crazy to me how people expect a 27B to outperform WAAAY larger models without it thinking by alot, like it's crazy. That's how these small models close the gaps and you wanna eliminate it. That's craycray

It's also the illusion, because you're on cloud , it may seem like its faster but its not because you cant see the thinking trace. GPT 5.6 and Claude 5 for example think for 8+mins a lot of times on extra high. Qwen3.8-27B is about the same.

Sign up or log in to comment