4 ms·
That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and th
by velcrovan 1mo ago
That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.
- NamlchakKhandro 1mo ago[flagged]
- w-ll 1mo agoits all still somewhat of a dice roll
- cellularmitosis 1mo agoWe don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison.
- spacebacon 1mo ago[dead]
- s1gsegv 1mo agoThey made a kind of strange decision with Qwen3.8 27B, the template defaults the reasoning_effort to xhigh. I found if you set it to medium it doesn’t just sit there churning forever.
- ekianjo 1mo agoxhigh gives better results
- dofm 1mo agoNot necessarily. I have seen xhigh go down several rabbit holes, dwell on edge cases and write worse code as a result; it literally distracted itself into writing a complex chain of functions ignoring my prompt, when on “low” reasoning it gets it right on a prompt that requires a few lines of code in the right places. Simon Willison’s blog has another example (SVG of a circle). It’s a bit like how giving LLMs access to web search tools can cause them to go down a blind alley based on their first “reasoning” output that then leaves them unable to solve a puzzle correctly that they can fully solve on their own.
- freehorse 1mo agoWith qwen 27b, setting the right reasoning effort for the specific task is important. With xhigh it has a chance at hard problems that bigger models may even fail. But for many everyday tasks, I have found that no reasoning and a system prompt instructing it to be brief is good enough. Note that even with thinking disabled, it may still get into long "chain of thought" reasoning state (out of thinking blocks) if the task is hard and you do not give further instructions, esp with access to tools etc.
- chmod775 1mo agoNot if it fills up its entire context with "But wait..."
- dofm 1mo agoxhigh tends not to do that. Uses caveman-ish language. But the reasoning trace does tend to obsess about stuff that it should just ask you about.
- apothegm 1mo agoIs there an easy way for a n00b with LMStudio to switch it to medium? Asking for a friend… XD
- velcrovan 1mo agoI had heard of this and actually did set the reasoning to medium ahead of time…
- deleted 1mo ago[deleted]
- noman-land 1mo agoI've recently learned and then observed that oMLX serves local models much, much faster than LM Studio.
- jonplackett 1mo agoSet its thinking lower. This is a known issue. It still thinks A LOT with lower reasoning levels
- phront 1mo agoMaybe giving pi more output by setting higher value to maxTokens will resolve his issue
- jonplackett 1mo agoI’ve been using the mlx version with orb studio an opencode
- drdaeman 1mo agoCheck out this: https://news.ycombinator.com/item?id=49402232 https://news.ycombinator.com/item?id=49402232 both article and comments. There are a lot of knobs to tweak, and some are pretty impactful.