3 ms·
!! Be careful when testing the model. A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinkin
by benjiro29 3mo ago
!! Be careful when testing the model.
A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinking has a issue with the default configuration.
Important - make sure that THINKING is enabled. By default it wasn't although I was passing the flag --default-chat-template-kwargs '{"enable_thinking": true}' in vllm recipe. The generation_config.json file that is included has by default max_new_tokens as 32k which seems to be cutting off thinking altogether so increase it.
At first I was very disappointed with the output I was seeing, but once thinking is enabled, the code quality seems to be MUCH better. More real world testing to be done.
https://www.reddit.com/r/LocalLLaMA/comments/1v2pg99/laguna_s_21_released_cheaper_than_deepseek_v4/oyy69u8/ https://www.reddit.com/r/LocalLLaMA/comments/1v2pg99/laguna_...
- voxgen 3mo agoEven the official provider on OpenRouter seems to have this issue. Hope it's an easy fix for them.
- nshotton 3mo agoThanks for posting this, it made a huge difference tweaking the recipe.
- d2p 3mo agoLooks like the default chat template was updated on HF to enable this by default shortly after you posted this :-)
- deleted 3mo ago[deleted]