8 ms·
Ok I find it funny that people compare models and are like, opus 4.7 is SOTA and is much better etc, but I have used glm 5.1 (I assume this comes form them trai
by alex7o 6mo ago
Ok I find it funny that people compare models and are like, opus 4.7 is SOTA and is much better etc, but I have used glm 5.1 (I assume this comes form them training on both opus and codex) for things opus couldn't do and have seen it make better code, haven't tried the qwen max series but I have seen the local 122b model do smarter more correct things based on docs than opus so yes benchmarks are one thing but reality is what the modes actually do and you should learn and have the knowledge of the real strengths that models posses. It is a tool in the end you shouldn't be saying a hammer is better then a wrench even tho both would be able to drive a nail in a piece of wood.
- solomatov 6mo ago>but I have seen the local 122b model do smarter more correct things based on docs than opus Could you please share more about this
- alex7o 6mo agoMaybe a bit misleading. I have used in in two places. One Is for local opencode coding and config of stuff the other is for agent-browser use and for both it did better (opus 4.6) for the thing I was testing atm. The problem with opus at the moment I tired it was overthinking and moving itself sometimes I the wrong direction (not that qwen does overthink sometimes). However sometimes less is more - maybe turning thinking down on opus would have helped me. Some people said that it is better to turn it of entirely when you start to impmenent code as it already knows what it needs to do it doesn't need more distraction. Another example is my ghostty config I learned from queen that is has theme support - opus would always just make the theme in the main file
- OtomotO 6mo agoMany people averted religion (which I can get behind with), but have never removed the dogmatic thinking that lay at its root. As so many things these days: It's a cult. I've used Claude for many months now. Since February I see a stark decline in the work I do with it. I've also tried to use it for GPU programming where it absolutely sucks at, with Sonnet, Opus 4.5 and 4.6 But if you share that sentiment, it's always a "You're just holding it wrong" or "The next model will surely solve this" For me it's just a tool, so I shrug.
- taurath 6mo agoI agree - the problem is it’s hard to see how people who say they’re using it effectively actually are using it, what they’re outputting, and making any sort of comparison on quality or maintainability or coherence. In the same way, it’s hard to see how people who say they’re struggling are actually using it. There’s truth somewhere in between “it’s the answer to everything” and “skill issue”. We know it’s overhyped. We know that it’s still useful to some extent, in many domains.
- balls187 6mo agoWell summarized. We're also seeing that the people up top are using this to cull the herd.
- psychoslave 6mo agoWhat is it that is dogma free? If one goes hardcore pyrrhonism, doubting that there is anything currently doubting as this statement is processed somehow, that is perfectly sound. At some point the is a need to have faith in some stable enough ground to be able to walk onto.
- Wolfbeta 6mo agoWho controls that need for you?
- ecshafer 6mo agoAll people think dogmatically. The only difference is what the ontological commitments and methaphysical foundations are. Take out God and people will fit politics, sports teams, tools, whatever in there. Its inescapable.
- bensyverson 6mo agoAllow me to introduce you to Buddhism
- 6mo ago
- cornedor 6mo agoI tried GLM and Qwen last week for a day. And some issues it could solve, while some, on surface relatively easy, task it just could not solve after a few tries, that Opus oneshotted this morning with the same prompt. It’s a single example ofcourse, but I really wanted to give it a fair try. All it had to do was create a sortable list in Magento admin. But on the other hand, GLM did oneshot a phpstorm plugin
- dev_l1x_be 6mo agoDo you use Opus through the API or with subscription? Did you use OpenCode or Code?
- cornedor 6mo agoOpus trough Claude Code, the Chinese models trough OpenCode Go, which seems like a great package to test them out.
- ternaryoperator 6mo agoThe models test roughly equal on benchmarks, with generally small differences in their scores. So, it’s reasonable to choose the model based on other criteria. In my case, I’d switch to any vendor that had a decent plugin for JetBrains.
- FlyingSnake 6mo agoI tried GLM5.1 last week after reading about it here. It was slow as molasses for routine tasks and I had to switch back to Claude. It also ran out of 5H credit limit faster than Claude.
- bensyverson 6mo agoIf you view the "thinking" traces you can see why; it will go back and forth on potential solutions, writing full implementations in the thinking block then debating them, constantly circling back to points it raised earlier, and starting every other paragraph with "Actually…" or "But wait!"
- nothinkjustai 6mo agoI see this with Opus too.
- girvo 6mo agoIndeed. And that’s with Anthropic hiding reading traces unlike these other comparisons.
- FlyingSnake 6mo ago> "Actually…" or "But wait!" You’re absolutely right! Jokes apart, I did notice GLM doing these back and forth loops.
- tonyarkles 6mo agoI was watching Qwen3.6-35B-A3B (locally) doing the same dance yesterday. It eventually finished and had a reasonable answer, but it sure went back and forth on a bunch of things I had explicitly said not to do before coming to a conclusion. At least said conclusion was not any of the things I'd said not to do.
- Lerc 6mo ago
- Moosdijk 6mo agoI wonder why glm is viewed so positively. Every time I try to build something with it, the output is worse than other models I use (Gemini, Claude), it takes longer to reach an answer and plenty of times it gets stuck in a loop.
- Akira1364 6mo agoIDK about GLM but GPT 5.4 Extra High has been great when I've used it in the VS Code Copilot extension, I see no actual reason Opus should consume 3x more quota than it the way it does
- Mashimo 6mo agoI have used GLM 4.7, 5 and 5.1 now for about 3 month via OpenCode harness and I don't remember it every being stuck in a loop. You have to keep it below ~100 000 token, else it gets funny in the head. I only use it for hobby projects though. Paid 3 EUR per month, that is not longer available though :( Not sure what I will choose end of month. Maybe OpenCode Go.
- gck1 6mo agoThat's unfortunate. 70-80k tokens is roughly the point where I start wrapping up with giving agent required context even on the small to medium sized requests. That would leave almost no tokens for actual work
- Mashimo 6mo agoEDIT: Ok, now I tried GLM for the first time in the morning CET, and it was .. bad. The reasoning took 5 mintues for a very very small .html file going around in circles. Evening CET experience for me is super smooth.
- pkulak 6mo agoI've been running Opus and GLM side-by side for a couple weeks now, and I've been impressed with GLM. I will absolutely agree that it's slow, but if you let it cook, it can be really impressive and absolutely on the level of Opus. Keep in mind, I don't really use AI to build entire services, I'm mostly using it to make small changes or help me find bugs, so the slowness doesn't bother me. Maybe if I set it to make a whole web app and it took 2 days, that would be different. The big kicker for GLM for me is I can use it in Pi, or whatever harness I like. Even if it was _slightly_ below Opus, and even though it's slower, I prefer it. Maybe Mythos will change everything, but who knows.
- dev_l1x_be 6mo agoBenchmarking is grossly misleading. Claude’s subscription with Code would not score this high on the benchmarks because how they lobotomized agentic coding.
- jxmesth 6mo agoThe only reason I'm stuck with Claude and Chatgpt is because of their tool calling. They do have some pretty useful features like skills etc. I've tried using qwen and deepseek but they can't even output documents. How are you guys handling documents and excels with these tools? I'd love to switch tbh.
- ecocentrik 6mo agoWhen was the last time you used Qwen models? Their 3.5 and 3.6 models are excellent with tool calling.
- jxmesth 6mo agoI gave it a try a few weeks ago tbh, I'll give it another shot tho. I mainly use their Web chats since that's easier to use and previously, qwen, deepseek, kimi, all were unable to output proper docx files or use skills.
- ecocentrik 6mo agoTry loading the models up in a coding harness like Claude Code. There's a few docx skills listed on Vercel's skill index. https://skills.sh/tfriedel/claude-office-skills/docx https://skills.sh/tfriedel/claude-office-skills/docx
- ycui1986 6mo agooutputting docx files does not have much to do with model capability. it is about whether tool calling has be configured .
- jwitthuhn 6mo agoI've been using qwen-code (the software, not to be confused with Qwen Code the service or Qwen Coder the model) which is a fork of gemini-cli and the tool use with Qwen models at least has been great.
- embedding-shape 6mo ago> I've tried using qwen and deepseek but they can't even output documents What agent harness did you use? Usually, "write_file", "shell_exec" or similar is two of the first tools you add to an agent harness, after read_file/list_files. If it doesn't have those tools, unsure if you could even call it a agent harness in the first place.
- ezekiel68 6mo agoQwen3-Coder produced much better rust code (that utilized rust's x86-64 vectorized extensions) a few months ago than Claude Opus or Google Gemini could. I was calling it from harnesses such as the Zed editor and trae CLI. I was very impressed.
- justincormack 6mo agoCodex is pretty good at Rust with x86 and arm intrinsics too, it replaced a bunch of hand written C/assembly code I was using. I will try Qwen and Kimi on this kind of task too.
- gck1 6mo agoI think claude in general, writes very lazy, poor quality code, but it writes code that works in fewer iterations. This could be one of the reasons behind it's popularity - it pushes towards the end faster at all costs. Every time codex reviews claude written rust, I can't explain it, but it almost feels like codex wants to scream at whoever wrote it.
- lambda 6mo agoTheir latest, Qwen3.6 35B-A3B is quite capable, and fast and small enough I don't really feel constrained running it locally. Some of the others that I've run that seem reasonably good, like Gemma 4 31B and Qwen3.5 122B-A10B just feel a bit too slow, or OOM my system too often, or run up on cache limits so spend a lot of time re-processing history. But the latest Qwen3.6 is both quite strong, and lightweight enough that it feels usable on consumer hardware.
- sirnicolaz 6mo agoConsider that SWE benchmarking is mainly done with python code. It tells something
- odie5533 6mo agoIf you showed me code from GLM 5.1, Opus 4.6, and Kimi K2.6, my ranking for best model would be highly random.
- mikenew 6mo agoGLM 5.1 was the model that made me feel like the Chinese models had truly caught up. I cancelled my Claude Max subscription and genuinely have not missed it at all. Some people seem to agree and some don't, but I think that indicates we're just down to your specific domain and usage patterns rather than the SOTA models being objectively better like they clearly used to be.
- operatingthetan 6mo agoIt seems like people can't even agree which SOTA model is best at any given moment anymore, so yeah I think it's just subjective at this point.
- fwipsy 6mo agoPerhaps not even necessarily subjective, just performance is highly task-dependent and even variable within tasks. People get objectively different experiences, and assume one or another is better, but it's basically random.
- operatingthetan 6mo ago>just performance is highly task-dependent and even variable within tasks. People get objectively different experiences, and assume one or another is better, but it's basically random. You are right that this is not exactly subjectivity, but I think for most people it feels like it. We don't have good benchmarks (imo), we read a lot about other people's experiences, and we have our own. I think certain models are going to be objectively better at certain tasks, it's just our ability to know which currently is impaired.
- easygenes 6mo agoUnless you're looking at something like a pass@100 benchmark, the benchmarks are confounded heavily by a likelihood of a "golden path" retrieval within their capabilities. This is on top of uncertainties like how well your task within a domain maps to the relevant test sets, as well as factors like context fullness and context complexity (heavy list of relevant complex instructions can weigh on capabilities in different ways than e.g. having a history where there's prior unrelated tasks still in context). The best tests are your own custom personal-task-relevant standardized tests (which the best models can't saturate, so aiming for less than 70% pass rate in the best case). All this is to say that most people are not doing the latter and their vibes are heavily confounded to the point of being mostly meaningless.
- bink-lynch 6mo agoI have been using GLM-5.1 with pi.dev through Ollama Cloud for my personal projects and I am very happy with this setup. I use pi.dev with Claude Sonnet/Opus 4.6 at work. Claude Code is great but the latest update has me compacting so much more frequently I could not stand it. I don't miss MCP tool calling when I am using pi.dev; it uses APIs just fine. I actually think GML-5.1 builds better websites than Claude Opus. For my personal projects I am building a full stack development platform and GLM-5.1 is doing a fantastic job.
- jadbox 6mo agoWhy use ollama cloud versus like Openrouter?
- zackify 6mo agoRecently it had great limits but this month I'm trying open router directly.
- bink-lynch 6mo agoThe limits seem higher on Ollama Cloud to me than paying for API access. I don't have solid stats on that though. I have an OpenRouter account and the service I am creating is going to need to use that. I will have better measuring stick then.
- zackify 6mo agoI'm using pi the same as you. However, I have an MCP I need to use and the popular extension for that support works fine for me. Really liking pi and glm 5.1!
- blurbleblurble 6mo agoOpus 4.6 was incredible but Opus 4.7 is genuinely frustrating to me so far. It's really sharp but can be so lazy. It's constantly telling me that we should save this for tomorrow, that it's time for bed (in the middle of the day), and very often quite sloppy and bold in its action. These adjustments are getting old. The next crop of open models seems ready to practically replace the big ones as sharp orchestrator agents.
- chillfox 6mo agoI have never seen a model be “lazy” before (I have seen them go for minimal change). I have been using the models through the api with various agents and no custom system prompt. So I am curious, how do people get these lazy outputs? Is it by having one of those custom system prompts that basically tells the model to be disrespectful? Or is it free tier? Cheap plans?
- enraged_camel 6mo agoI have seen some people complain about a new tendency where it can suggest wrapping up the current task even though it isn't done yet. I haven't seen it myself though.
- solenoid0937 6mo agoUsually this gets worse if you have a phrase like "wrap it up" earlier in the output, or if you're at a few hundred thousand tokens without compacting. In both cases the fix is really simple, just compact.
- szundi 6mo ago[dead]
- make3 6mo agoI had to write multiple times in my prompt that it's not the model's role to change the subject or end the conversation at all. I think that they do that to dodge conversations about controversial subjects without full-on refusing to answer. They'll give you an ok answer then tell you to go to get the walk you were talking about. I also feel like maybe they think people are still ready to pay a lot if they feel like they're getting a lot of "high value stuff" even if the low value stuff the model refuses to do, so they basically try to stop you from doing low value stuff on Opus. I suspect that Sonnet or Haiku never tells you to go take a hike.
- mkhalil 6mo agoNot to mention, that Opus cost orders of magnitude more money. These are VERY impressive and usage. FAANGS love to give away money to get people addicted to their platforms, and even they, the richest companies in the world, are throttling or reducing Opus usage for paying members, because even the money we pay them doesn't cover it. Meanwhile, these are usable on local deployments! (and that's with the limited allowance our AI overlords afford us when it comes to choices for graphics cards too!)
- vidarh 6mo agoI don't find GLM 5.1 beating Opus personally, but I do think it is good enough to consider it part of the SOTA pack at this point. It feels like it needs more time and tokens to achieve things, but that's okay - it's so much cheaper per token. If Qwen3.6-Max is up there as well, it will be very interesting.
- flyingsquirrel_ 6mo agoGLM 5.1 is sometimes better than Opus 4.7. So it made me buy GLM coding plan.