10 ms·

GLM-5.2 is the new leading open weights model on Artificial Analysis

• blt 4mo ago
  There's only one GLM in my heart: the one that includes vec3.hpp
• Tiberium 4mo ago
  It seems to really be a nice step-up and is getting quite close to the frontier. I wish they'd start focusing on the reasoning efficiency now, though. I have a simple (relatively) test task to evaluate LLMs: writing a simple math evaluator library in Nim (it's about 400-600 lines total max), and GLM 5.2 (xhigh which maps to max effort) spent over 15 minutes (!) reasoning, spending about 45k tokens, before it finally wrote the first file. I know it's hard to improve on that, but now that their models are good enough at raw intelligence, I think this should become a higher priority task. Currently on https://artificialanalysis.ai/#output-tokens https://artificialanalysis.ai/#output-tokens GPT 5.5 xhigh spends 16k tokens total on average, GPT 5.5 high is 10k, Fable 5 33k, Opus 4.8 41k, GLM 5.2 is 42k. GPT 5.5 is extremely reasoning efficient. Of course if you convert those values to actual request cost, GLM 5.2 will probably beat GPT 5.5/Opus 4.8, but speed matters for a lot of people, I think.
  • bertili 4mo ago
    This is GLM 5.2 Max. GLM 5.2 High which use less than half[1] the tokens. [1] https://z.ai/blog/glm-5.2 https://z.ai/blog/glm-5.2
    • Tiberium 4mo ago
      Yes, but the Artificial Analysis result is also from GLM 5.2 (max), not high.
      • andai 4mo ago
        They have this with a lot of models, measuring only the max setting, while the one you'd actually want to use for most tasks is much lower.
        • epolanski 4mo ago
          For the brief period with had Fable, I never had to use it above medium. Low nailed the overwhelming majority of mundane tasks on it's own, medium was good for more complex stuff.
  • vorticalbox 4mo ago
    This is a problem I find with opus is will spend so long thinking then going “but wait what if” To point where I stop it and simple tell it to “start writing code you can work it out as you go along” Seems writers block also effects LLM
    • epolanski 4mo ago
      Fable was 20 times worse on that. It's clear it was the vibe coding model, as like no other model before, fully turned you into his assistant instead of the other way around.
      • deleted 4mo ago
        [deleted]
      • RyanHamilton 4mo ago
        Could it be possible, these firms are optimizing for two things: a) Better performance. b) Gathering data from you to further improve performance later. I've also found the huge amount of planning rather than iteration frustrating. I've felt like I'm teaching a junior!
        • epolanski 4mo ago
          I think they simply optimize around E2E benchmarks, none of those benchmarks is designed as multi turn assistance to the user, but going from a prompt straight to the final solution.
          • celrod 4mo ago
            Exactly. How can "we" develop and encourage benchmarks for multi-turn user assistance? That is what I want. I feel like the models and harnesses push much too hard against this workflow -- that they push you towards letting go and vibe coding, with only your discipline (and desire for a quality and maintainable product) holding it back.
        • happyPersonR 4mo ago
  • deleted 4mo ago
    [deleted]
  • benjiro29 4mo ago
    GLM 5.2 Max = Opus 4.8 Max in thinking behavior. The thinking chain is so similar, and so is the amount of token usage on the output. If you want reasonable token usage, you need to run it GLM 5.2 at High. There is little drop in quality from Max to High (for most tasks). And it cuts token usage by 2 a 2.5x. GLM 5.2, Max is really something you only need for complex tasks. In essence, GLM 5.2 is Opus 4.8 its little brother, at a way, WAY cheaper price. There has been really no training on Opus models going on, really, none i tell you! /sarcasm
    • vitalyan123 4mo ago
      distillation of thinking models is not particularly effective - both "Open"AI and Misanthropic don't show you the real chain of thought, only its severely downscaled version. both do everything in their power to combat such outrageous copyright infringement, so the bulk of unethically scrapped data the Chinese have is from several generations ago.
      • duskdozer 4mo ago
        >such outrageous copyright infringement Sarcasm, considering the source of their own training data?
        • orphea 4mo ago
          Narrator: it was sarcasm, indeed.
        • baron3dl 4mo ago
          IP for me, not thee.
        • margalabargala 4mo ago
          Considering they called the company "Misanthropic", sarcasm is a safe bet.
          • duskdozer 4mo ago
  • cmrdporcupine 4mo ago
    > Of course if you convert those values to actual request cost, GLM 5.2 will probably beat GPT 5.5/Opus 4.8, but speed matters for a lot of people, I think. GLM5.2 ends up being far more expensive than I thought it would be when I tried it on openrouter. I ground through $5 USD worth of tokens quite quickly. And this was high, not max.
    • guelo 4mo ago
      Using these open models really makes you realize how subsidized Anthropic and OpenAi's subscription plans are.
      • nijave 4mo ago
        Absolutely. You can also run codeburn or ccusage and they'll scan the session files and tell you how much you burnt in API token pricing equivalent.
  • robmccoll 4mo ago
    That's interesting. I gave nearly the same task to Gemma4 31b as a test yesterday. Write a symbolic math engine in Typescript that can perform evaluation and simple expression reductions over +-/*(). It performed the task correctly with minimal reasoning - much fewer reasoning tokens than output tokens.
    • gbingles 4mo ago
      Tbh, so what? I googled "symbolic math engine in Typescript that can perform evaluation and simple expression reductions over +-/*()" and got what looks to be viable answers without using any AI model at all. Reciting well established things from memory isn't terribly interesting. Show it a novel codebase and have it implement something within it.
      • drob518 4mo ago
        So, a natural question would be why a model would ever get it wrong?
      • SubiculumCode 4mo ago
        TBH, while your point is a fair one, your attitude is off-putting and needlessly condescending.
  • h14h 4mo ago
    Hopefully the recent work Moonshot did with Kimi K2.7 Code trickles in to the other open-model labs. Per AA, while K2.7 Code is roughly on par w/ K2.6 in terms of intelligence, it uses half the output tokens to get there.
    • h14h 4mo ago
      I've been doing some testing with GLM 5.2 on Fireworks and it looks like the "High" reasoning level uses fewer tokens than even K2.7 Code by a considerable margin (roughly half). Don't have any evals indicating how it compares on upper-bound quality, but for a well-defined task it seems like GLM 5.2 on "High" is remarkably token efficient. Looking forward to seeing where it lands on the AA index.
  • rdsubhas 4mo ago
    As per stats in other comments, it is frontier, not close to frontier.
  • esafak 4mo ago
    I agree. I've noticed that it is quite smart but it has a tendency to doubt itself and overthink. I monitor its internal dialogue and prod it when it does this. They need to optimize the chain of thought early stopping.
  • deleted 4mo ago
    [deleted]
  • alexjplant 4mo ago
    > It seems to really be a nice step-up and is getting quite close to the frontier. IMHO it's already surpassed them. I vastly prefer my personal GLM and OpenCode setup to the Claude Code and Opus one that I have to use at work. The former makes way fewer StackOverflow brogrammer-tier mistakes and is considerably better at following instructions. The harness UX is also vastly superior as it doesn't ignore, randomly change, or incorrectly report settings. Maybe it's the harness and I'd have even greater success with OpenCode and Anthropic, but I think it safe to say that Anthropic's moat is evaporating.
    • carter2099 4mo ago
      You would be surprised at how much of an impact the harness has. I switched to Pi and chinese open source models, and models that _I know_ are less capable than sonnet outperform my sonnet + claude code stack at work.
  • nurumaik 4mo ago
    You asked for maximum effort, you got maximum effort
  • xyzsparetimexyz 4mo ago
    Reminiscent of https://en.wikipedia.org/wiki/Portia_(spider) https://en.wikipedia.org/wiki/Portia_(spider)
  • HWR_14 4mo ago
    I thought you could not compare tokens across models because their cost and speed was so different between models.
  • gateonai 4mo ago
    [flagged]
  • abgruszecki 4mo ago
    Agreed that models should get better at working with rare programming languages like Nim! Using them tends to confuse agents a lot in general. We're working on a paper right now where we compare how token-efficient models are when trying to implement the exact same program in different programming languages, and that's one of the trends we're seeing.
• CuriouslyC 4mo ago
  I've been playing with this model a fair amount over the last 24 hours, and I can confirm it's quite capable, while being a little bit verbose (I've seen it reconsider things 3-4 times in thinking traces before deciding on a path forward), and not being quite as good as GPT5.5 at working through complex abstract requirements. Honestly it's good enough that I feel comfortable recommending a Z.AI sub + a $20/mo OpenAI sub for all but the most AI pilled multi-orchestrators, or the die hard Claude fans. GLM writing + GPT reviewing/debugging feels pretty unlimited and minimally worse than just doing everything in GPT with the $200/mo plan.
  • igravious 4mo ago
    After having got a taste of Fable 5 for me Opus 4.8 doesn't cut it any more -- and I don't know how to put this, I don't know if it's just me, but it's rhetorical flourishes are starting to really grate on me, never mind that it is at times deliberately weasel-wordy and economical with the truth until pressed. Opus 4.8 is definitely a stronger coding agent than DeepSeek 4.0 or Kimi 2.7 succeeding where they flounder and fail but its way of expressing itself conversationally is making me reconsider my subscription …
    • elwebmaster 4mo ago
      You are not alone. How about GPT 5.5? Does it come close to Fable 5?
      • fragmede 4mo ago
        5.5 is pretty good. It's no Fable though. It is definitely better than opus tho.
      • theplumber 4mo ago
        GPT 5.5 xhigh is smarter than Fable but Fable like Opus 4.8 as well is faster and seems more “agentic”. It’s easy to test this. Build a fairly complex software with Claude(opus or Fable). Review the commits with both Claude and GPT 5.5 Xhigh. You can see that Fable is still sloppy(er) compared to GPT. You can test it the other way around as well(drive the dev with GPT and review with GPT and Claude). You get the same result Claude has an edge though and that’s on building more beautiful user interfaces.
• Havoc 4mo ago
  It’s pretty good. More talkative than 5.1. Reminds me of deepseek 4 Their servers are melting though - getting more timeouts etc
• unrvl22 4mo ago
  Why aren't more people talking about this? It's literally Opus 4.7 quality stupid prices. I know providers who are offering this at unlimited tokens for $50 a month. Some are even offering API rates at 3x lower than the official ZAI api rates which are already like 10x cheaper than Opus. (Crof and Umans btw) This is a huge blow to Anthropic/OpenAI/Google and a massive win for the rest of the world. The official API prices and speeds mean nothing for open source models.
  • cedws 4mo ago
    In my org everyone is extremely Claude-pilled to the point you’d think it’s the only LLM that exists, purely because it caters to non-engineers within enterprises.
  • unrvl22 4mo ago
    I cancelled my claude sub after realizing I can burn 300m tokens a day of this quality, for $50 a month.
    • spelk 4mo ago
      Which coding plan are you using? How are you finding it?
  • Hamuko 4mo ago
    I’m not that interested in models that I can’t run on my desktop for ~0€, which is my AI budget.
    • igravious 4mo ago
      Cool beans. You're not the target audience then.
      • Hamuko 4mo ago
        Did I claim I was? I just said why I and people like me are not talking about it.
• nh43215rgb 4mo ago
  > GLM-5.2 sits off the most attractive quadrant on the Intelligence vs Output Tokens chart. That is unfortunate...
• kingstnap 4mo ago
  According to many benchmarks this model is straight up frontier level and Zai seriously cooked. Some of these numbers are incredible. Excited to see if this turns out to be a Open Weight Opus 4.5 or better.
  • andai 4mo ago
    The only benchmarks that matters is your actual task. I've had models that benched poorly but performed great. And I constantly see models at near the top of AA, which are terrible. There doesn't necessarily seem to be a lot of overlap between benchmarks and real world usage. (Let alone common sense!) As far as they go, though, these harder benchmarks match my experience more closely: https://deepswe.datacurve.ai/ https://deepswe.datacurve.ai/ and https://cognition.ai/blog/frontier-code https://cognition.ai/blog/frontier-code Where we see "top" models drop way down in score when given longer tasks. That being said, I've had a reasonably pleasant time with GLM-5.2 so far. (And have had an OK time with DeepSeek as well.) By the time I'm done testing all the Chinese models, they'll be obsolete :)
  • adastra22 4mo ago
    According to reports in this thread it is somewhere between Opus 4.7 and 4.8. This is effectively frontier.
• deleted 4mo ago
  [deleted]
• davidwritesbugs 4mo ago
  I like their models, super cheap - I'm a Lite plan subscriber, and subjective performance seems to be same as lower Anthropic models, useful for lots of grunt work. The problem is that Ziphu really __really__ struggle with capacity - everyone is complaining of timeouts or very slow speeds. I can't get direct access to the model though I see it is in OpenRouter so I may play. But the capacity issues means DeepSeek is my main provider these days
• mohsen1 4mo ago
  [flagged]
  • benjiro29 4mo ago
    A yes, the stealth advertisement post ...
• tensegrist 4mo ago
  > On the Intelligence vs. Cost per Task Pareto Frontier: GLM-5.2 is on the Pareto frontier of the Intelligence vs Cost per Task chart, with the lowest cost per task among models at its intelligence level. GLM-5.2 costs ~$0.46 per task, compared to GLM-5.1 ($0.25), Kimi K2.6 ($0.31), MiniMax-M3 ($0.18) and DeepSeek V4 Pro (max, $0.05) am i missing something?
  • xiaoyu2006 4mo ago
    Some models are heavily subsidized. Total params & active params are better measurement of inference cost.
    • simianwords 4mo ago
      No models are subsidised -- there are lots of third party hosting services that will still run at breakeven/profit. (except Deepseek after discount)
      • stymaar 4mo ago
        > No models are subsidised We have no proof in either direction, it's not like we had access to their financial numbers in details. And the pricing itself muddies the water, as input tokens that are already in the KV cache are practically free for the provider, whereas other tokens are expensive. So they could still make money overall thanks to people having multi-turn conversation (and as such, paying multiple times for the same token), but lose money on actual compute done. > there are lots of third party hosting services that will still run at breakeven/profit. How can you be sure that they are making profit directly from token price, and are not billing at marginal cost (i.e. electricity price, without counting the cost of the GPUs) and aiming to make a profit later on from the valuable training data that they are collecting in the process?
        • simianwords 4mo ago
          > How can you be sure You are free to believe that they are doing all this. Or you can simply believe the intuition that models are getting cheaper by the day. I can run Gemma 4 31B from my laptop today.
• rahidz 4mo ago
  Correct me if I'm wrong, but neither DeepSeek nor GLM have image input modality. This makes them less useful when looking at UIs, photos, screenshots, etc. doesn't it? Or do they have alternate ways of doing so?
  • mordae 4mo ago
    They do not and it sucks for certain tasks. It also means that if they actually trained with vision, they'd be on par with Anthropic models as vision seems to improve model performance across the board even for non-vision tasks.
    • osti 4mo ago
      Many other open source models have vision but they don't compare to GLM in terms of coding quality. So I don't think it's because of vision that the frontier models are better, it's more that they are probably just much bigger models.
    • freigeist79 4mo ago
      it helps giving them a cli vision tool (curl to openrouter vision model for example)
  • adrian_b 4mo ago
    That's right, but there are other recent open weights and relatively big LLMs that are multimodal, e.g. MiniMax-M3. With open weights LLMs, it is affordable to use many different models, each for whatever it is better. Moreover, for analyzing "UIs, photos, screenshots, etc." there are small models that can be run locally on smartphones or laptops, e.g. IBM granite-vision-4.1-4B, certain Google Gemma 4 variants and certain Qwen variants, whose output you can use as input for a big LLM, in order to accomplish some more complex task.
  • dryarzeg 4mo ago
    Yes, you are right (as far as I'm aware). For things where you need the LLM to look at screenshots, photos or other images you can use Kimi-K2.6/K2.7 - comparable pricing, somewhat comparable performance and quality. You can even probably combine two models (e.g Kimi and GLM) in one agent, using Kimi for multimodal inputs and GLM for everything else, although 1) I'm not sure if this will not cause some kind of context poisoning with low-quality patterns for better performing model (e.g. in some cases Kimi may be worse than GLM, but GLM, when following up, may adopt the same reasoning patterns as Kimi, undermining it's own performance), and 2) I'm not quite sure if it's possible with the tools currently available (I'm not really into agentic or chatbots stuff to be honest).
• creamyhorror 4mo ago
  It's a real step forward, getting closer to SOTA. It seems to be very epistemically cautious in its reasoning. I hope Deepseek and the other open-weights labs stay in the game and catch up too.
• xiaoyu2006 4mo ago
  This open source model is quite near SOTA with only 700B/40B MoE. Truly efficient.
• lousken 4mo ago
  Cerebras really needs to have this on their API list (if they even still exist).
  • Marciplan 4mo ago
    they went public a few weeks ago
    • lousken 4mo ago
      That's cool and all, but they are still on GLM 4.7
      • 0xbadcafebee 4mo ago
        Which is fine for their target market. Their latest model is Kimi K2.6, available to enterprise customers. But older models become more powerful when you have time to do more reasoning. Also many applications don't need advanced models. Cerebras is making bank from all the other use cases that SOTA providers left on the table by focusing on 0-shot intelligence over speed
• ramon156 4mo ago
  I've made a comment before that 5.1 will sometimes get stuck looping over a simple decision or statement. It will basically contradict and then not realize that one option is the definite option. Sometimes it's two statements that aren't even exclusive. Nonetheless, a lot of tokens that get wasted from this. I haven't extensively used 5.2 yet, but it seems a lot better.
• _pdp_ 4mo ago
  I am helpful. DeepSeek V4 has been quite amazing in our workloads and it operates at a fraction of the cost. I have not tried GLM 5.2 but it seems that it hits a sweet spot.
  • LUmBULtERA 4mo ago
    Your system prompt is showing.
    • kreddor 4mo ago
      Maybe he meant "hopeful"...
      • _pdp_ 4mo ago
        This is exactly what I meant - sorry honest typo.
        • LUmBULtERA 4mo ago
          haha fair enough. Sorry!
• XCSme 4mo ago
  In my tests[0] GLM-5.2 is not much better than GLM-5, and overall DeepSeek V4 Flash seems to be the better/more cost-effective choice: [0]: https://aibenchy.com/compare/deepseek-deepseek-v4-flash-high/z-ai-glm-5-2-medium/ https://aibenchy.com/compare/deepseek-deepseek-v4-flash-high...
  • XCSme 4mo ago
    I think the problem is, as can also be seen on other benchmarks, is that most models nowadays are focused more and more purely on tool calling and coding. This means, that models are losing more and more general and domain-specific knowledge. Look at those graphs on ARtificialAnalysis, GLM-5.1 still performs similarly or better: AA-Omnisicence Accuracy: https://i.snipboard.io/5DYmpx.jpg https://i.snipboard.io/5DYmpx.jpg IFBench: https://i.snipboard.io/74kg0R.jpg https://i.snipboard.io/74kg0R.jpg I still feel like models are not getting any smarter for a few months already, they just changed their training to be focused more on some areas than others, so shifting the intelligence from one place to another, not necessarily increasing the overall intelligence or "AGI" score.
    • HDBaseT 4mo ago
      Well, in that example it still seems the big players are increasing overall "intelligence" as Fable tops the list. OpenAI has big incentives to improve general interligence as a large percentage of users use ChatGPT for support, finances, questions, etc. Not just coding.
  • sourcecodeplz 4mo ago
    man, i love dsv4-flash but i found its weaknesses in complex projects with multiple moving parts. tried kimi 2.6 and it understood and could work on the task. bigger is better..
• Pragmata 4mo ago
  So this basically means we will have a near opus level model able to be run locally in the next couple of months right? QWEN 3.6 27b is already pretty good, but it should be possible to get a better option now that runs in the same hardware, right?
  • XCSme 4mo ago
    Which Opus? GLM-5.2 is already close to Opus-4.7 level: https://aibenchy.com/compare/anthropic-claude-opus-4-7-medium/z-ai-glm-5-2-medium/ https://aibenchy.com/compare/anthropic-claude-opus-4-7-mediu...
    • XCSme 4mo ago
      Oh, or you meant a smaller model than GLM-5.2 with similar capabilities?
      • Pragmata 4mo ago
        Yep! I'm running things locally on a RTX5080 + RTX1060 + 64GB DDR5 ram, and would love to get a more capable model if possible! QWEN3.6 27b is pretty good, but i can still notice some spots where it's not as good as the frontier models.
      • segmondy 4mo ago
        Probably not. Qwen3.(5|6)-27B seems like an "accidental freak". I'm not even sure they know what they did to create that. A decent amount of the team members left after that, so unfortunately, we might not be seeing another small model that packs such a punch for a while. Hopefully the team is studying their entire training recipe for that and is able to replicate. If they are, then a 50-70B dense model might give us such capabilities...
        • SwellJoe 4mo ago
          Gemma 4 is competitive with Qwen 3.6. I had vague feelings that Qwen was better at coding tasks, based on anecdotes and public benchmarks, but I've been doing some benchmarking lately, and Gemma 4 31b is consistently beating Qwen 3.6 at the really hard stuff (finding hard security bugs, vision tasks for fixing UI layout or categorizing assets, in particular..and for vision, nothing self-hostable beats Gemma 4 12b, including 31b). I'm still hoping for a bigger Gemma 4 version, but I think they may be worried about competing with their own hosted models, since Gemma 4 is already better than a lot of Google's proprietary models that are still available in AI Studio. But, it is a shame that Qwen probably won't be doing more open models going forward. It is really strong for its size.
• CubsFan1060 4mo ago
  Knowing very little about how to run these, how close are we to medium or larger businesses starting to buy hardware to run models like this to keep the models local? It’s expensive, and not as capable as the frontier models, but would have some pretty big benefits around privacy and agency.
  • re-thc 4mo ago
    > how close are we to medium or larger businesses starting to buy hardware to run models like this to keep the models local? Years. Even Microsoft said they don't have enough for Github and need to call Amazon. Getting a few even at decent prices is hard. Unless the shortages goes down...
  • petesergeant 4mo ago
    Unless you have genuine national security concerns, you’d be better off just negotiating a commercial agreement with privacy protections with a couple of existing vendors.
    • CubsFan1060 4mo ago
      I think that's true until it isn't, which may end up being the problem. Fable/Mythos doesn't fall under the ZDR agreements with Anthropic. And I'm curious if others will follow suit.
    • tancop 4mo ago
      if you can afford the investment you get stable low costs for years with better security (at least if your cyber team is good). its even better in regulated industries where some vendors might add a premium for hipaa/soc/pci dss compliance to the point its a lot cheaper to self host. for a smaller business its not worth it and you should just use a hosted open model.
      • petesergeant 4mo ago
        > to the point its a lot cheaper to self host I'm pretty skeptical, especially given typical utilization patterns. Do you have numbers, or this is just vibes?
• mrngld 4mo ago
  Artificial Analysis coding benchmark shows GLM5.1 on high pretty close to GPT5.5 xhigh in cost to run, with GPT5.5 on medium significantly less expensive. Compared to GPT5.5 medium GLM5.1xhigh is twice the cost and half the intelligence. They don't have GLM5.2 on there yet, but that'd a big gap to bridge. https://artificialanalysis.ai/agents/coding-agents?coding-agents-performance-chart=deep-swe&coding-agents-harness-comparison-chart=harness-deep-swe#coding-agents-performance-chart-tabs https://artificialanalysis.ai/agents/coding-agents?coding-ag... I thought I was "holding it wrong" until DeepSWE came along -- personally it seems to match my own experiences pretty well. Really makes me wonder how legitimate some of the internet noise is about open models. There's surely some use cases for them, not everything needs the absolute frontier (GPT5.5 on low is awesome), but if you want to be near the frontier everyone needs to be honest about the fact that we're only talking about Opus, Fable, GPT5.5.
  • cmrdporcupine 4mo ago
    I gave GLM 5.2 a spin on openrouter yesterday and it was mostly fine but it racked up $5 in token use in 30 minutes of (relatively slow) work. It's easily 4x the cost of DeepSeek V4 but I didn't actually feel the results were that much better. I had GPT 5.5 in Codex review it after it was done and there was plenty of slop to go around. Having better luck with MiniMax M3, from a cost/benefit ratio.
    • zooming 4mo ago
      Try MiMo-2.5, I'm having astonishing success with it in opencode for cents per day. Not even the pro model.
      • spelk 4mo ago
        I've found MiMo-2.5 is fun for front-end design since you can use its multimodal capabilities to drop in whatever it produced and correct it for you.
    • pjerem 4mo ago
      I really like DeepSeek V4 Pro. It's pretty smart and I get so much usage out of it on a $20 Ollama cloud plan. With a good harness, that's my favorite model for any personal project. I use Opus 4.8 at work because i don't have to pay for it and of course I love it, but DeepSeek is like 80% there for one tenth of the price.
• kissgyorgy 4mo ago
  I tried it today through Openrouter and the API is atrocious. I got multiple rate limit and random errors every turn. Somebody wrote [1]; "I am never touching Minimax or GLM again. Their APIs had constant outages and I had to restart my runs multiple times — after burning money on the runs that failed midway." and I 100% agree. The model might be good, but if the API is so bad, it's effectively useless. [1]: https://kasra.blog/blog/i-spent-1500-seeing-if-llms-could-hack-my-app/ https://kasra.blog/blog/i-spent-1500-seeing-if-llms-could-ha...
  • Havoc 4mo ago
    That’s what happens when you offer something decent at a fraction of the price of opus - more demand than you can serve
  • osti 4mo ago
    I indeed got a few timeouts yesterday using the official API, I imagine for the coding plan users it'll be even worse.
  • segmondy 4mo ago
    The entire point of this post is that it's open weights, you can run it yourself and don't have to deal with the API issues. You really do have that choice.
    • ac29 4mo ago
      You could subscribe to Anthropic/OpenAI for the rest of your life for the cost it would take to host GLM5.2 locally - you need 1.5TB of VRAM just for the weights
      • zozbot234 4mo ago
        You don't need that much VRAM unless you're targeting a high-performance deployment that's intended to scale far beyond local use. For a lower-throughput case, you can keep the model weights on SSD at very low cost and stream them in for inference. This could actually scale reasonably well if you have something as simple as a previous-gen HEDT with a decent amount of PCIe lanes to host fast storage from.
• dsrtslnd23 4mo ago
  looks like I need a GB300 workstation
• wongarsu 4mo ago
  It's also third best overall on "AA-Omniscience Non-Hallucination Rate", far higher than DeepSeek, GPT 5.5 or Fable. That's the one benchmark that allows LLMs to answer "I don't know" and punishes them for trying to bullshit their way through the questions
• hit8run 4mo ago
  Ok, it is nice to see another great open source model. Not sure what to think of all these benchmarks but GLM was already quite strong before so an update is very welcome.
• eckelhesten 4mo ago
  Sure, but whatever you do, don't buy their (Z.ai) lite plan. I feel like i threw 15 dollars in the sea. I'm getting rate limited after 3-4 prompts. You get way less value than just paying 25 dollars for Claude or OpenAI models.
  • granra 4mo ago
    How are you using it? I have the lite plan and I've only ever maxed my weekly usage a few hours before reset. I will concede that I'm not a super heavy LLM user but it's been really good for me. My workflow is usually: - read file. I want to achieve X, how do? Do not implement anything. - I would do a, b and c - sketch a brief implementation of your suggestion - <code> (not writing files yet) - instead of your approach x, wouldn't it make sense to instead do z? What would that look like? - <code> - nice, implement this - starts writing files, run tests, etc.
    • eckelhesten 4mo ago
      Try pointing it to a small codebase, or even ask it to conjure information found online. You'll see that it quickly gives up. Thing is, they seem to count cached hits as if they were the non-cached tokens. I wont be subscribing again thats for sure. I am not paying iPhone money for a Xiaomi.
      • granra 4mo ago
        That's what I've been doing. I use crush normally. While the codebase are by no means huge, they're not tiny either.
        • eckelhesten 4mo ago
          Are you using it in an agentic workflow? Just reading the codebase will consume a lot of cached tokens, but seemingly, z.ai counts these as normal input tokens the way they're rate limiting.
          • 4mo ago
• Imustaskforhelp 4mo ago
  I have been trying out GLM 5.2 and I am really impressed by it for the most part. To all people on Hackernews, I am curious as to what agent harness are you using it with. Previously I was using opencode and then I switched to using Opencode + obra/superpowers and creating custom skill.md themselves for it. I found things to take more time and intervene more but the result of it has been that I have found it to work better. Now I have also started using oh-my-pi as well and I found it to be faster compared to Opencode. I am unsure how much of there is a difference to it and how much of things are placebo but what is your opinion regarding the best Agent harness for GLM 5.2?
  • Alifatisk 4mo ago
    I just used CC with GLM, I was satisfied.
• sourcecodeplz 4mo ago
  1m context btw.
  • Alifatisk 4mo ago
    And apparently, actual support for 1M context window, not just theoretical.
• kristopolous 4mo ago
  I have a script that ranks these based on codingindex from Artificial Analysis. All it does is pull a json from their main table page and parses it with the fields I care about (coding). There used to be a mailing list associated with it but eh ... there wasn't much interest. I use the script every day though. Current partial output score age size name 47.1 58 large Kimi K2.6 47.5 54 large DeepSeek V4 Pro (Reasoning, Max Effort) 47.5 70 - Muse Spark 47.6 132 - Claude Opus 4.6 (Non-reasoning, High Effort) 47.8 205 - Claude Opus 4.5 (Reasoning) 48.1 132 - Claude Opus 4.6 (Adaptive Reasoning, Max Effort) 48.6 55 - GPT-5.5 (Non-reasoning) 48.7 188 - GPT-5.2 (xhigh) 50.1 29 - Qwen3.7 Max 50.7 1 large GLM-5.2 (max) 50.9 120 - Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) 51.5 92 - GPT-5.4 mini (xhigh) 52.1 55 - GPT-5.5 (low) 52.5 62 - Claude Opus 4.7 (Adaptive Reasoning, Max Effort) 53.1 132 - GPT-5.3 Codex (xhigh) 53.1 62 - Claude Opus 4.7 (Non-reasoning, High Effort) 55.5 118 - Gemini 3.1 Pro Preview 56.2 55 - GPT-5.5 (medium) 56.7 20 - Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 57.2 104 - GPT-5.4 (xhigh) 58.5 55 - GPT-5.5 (high) 59.1 55 - GPT-5.5 (xhigh) 62 8 - Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) To see everything, run it like so $ curl day50.dev/art-analysis.sh | bash The repo: https://github.com/day50-dev/aa-eval-email https://github.com/day50-dev/aa-eval-email some key takeaways: * open models are on about a 4-7 month lag right now depending on how you want to measure it * if this keeps up, you might see an open-weights model doing claude fable 5 level work before the new year. if people sign up for the free mailing list (that just does this) I'll go and put it back on ... emails when new model evals drop - it was pretty useful.
  • alecco 4mo ago
    Consider using decrementing score order (best on top)
    • kristopolous 4mo ago
      then I'd have to scroll up over 500 lines after running it every time to see what I care about. But if that's your thing, here you go: https://github.com/day50-dev/aa-eval-email/commit/1853be64610393c5b2a069c610c70d7879a9ebe1 https://github.com/day50-dev/aa-eval-email/commit/1853be6461... add an argument (any argument) and it will be sorted as your specified. It just works as a toggle flipping the order ... so literally any string will do. The original link has been updated accordingly with the new code.
• simonw 4mo ago
  I was surprised that GLM 5.1/5.2 are not vision models - they are text input only. That's actually pretty uncommon these days. All of the OpenAI/Anthropic/Gemini models accept images, and so do the other leading open weight families - Gemma 4, Qwen 3.6, Kimi 2.x. In GLM's case image input would be useful because it's a model that scores very highly for tasks like web design, but without image input it can't take a screenshot and output HTML+CSS. Don't get me wrong, GLM is a phenomenal model, but the image thing is a bit of a gap.
  • _pdp_ 4mo ago
    I don't see this being such a big gap. There are some use-cases for sure but apart from UX/UI work it is not really needed. Besides, none of the frontier models can replicate actual images - the can approximate at least in my own experience.
    • simonw 4mo ago
      One of my tests for a new model is dumping in a screenshot of a web page and seeing if it can recreate it from scratch in HTML and CSS. Even the local models I run on my Mac are getting surprisingly good at that now.
      • kamranjon 4mo ago
        a pretty fun and quick tests i do with vision models is to screenshot the hackernews homepage and ask the model to return a json representation of the screenshot - qwen 3.5 0.8b did surprisingly well at this.
    • deleted 4mo ago
      [deleted]
    • tiahura 4mo ago
      Using llms to generate docx. Being able to rasterize and review is an important part of the process.
  • ashenke 4mo ago
• JustSkyfall 4mo ago
  The problem with these benchmarks is that the Chinese models tend to be incredible on paper, and absolutely terrible in practice :/
  • Mashimo 4mo ago
    I have used GLM since version 4.8 I think and do enjoy using them. More then other models like Kimi or Deepseek. Though only tested them on smaller private projects.
    • Alifatisk 4mo ago
      > I have used GLM since version 4.8 I think You probably refer to GLM-4.7
  • jingpostmedia 4mo ago
    [flagged]
  • segmondy 4mo ago
    You are obviously lying because it shows you have no experience with. GLM since 4.5 have been crushing it. all their models since then haven't skipped a beat. 4.5/4.5-air, 4.6, 4.7, 4.8, 5, 5.1. That aside, MiMoV2.5, MiniMax from 2.0, DeepSeek from V3, Kimi since V2, Qwen since 3, Hy3 have all been amazing models. All from China, we need to get over it. China is not losing yet as far as the AI race is concerned.
    • Alifatisk 4mo ago
      Is there a GLM-4.8 model?
  • bel8 4mo ago
    I beg to differ. I replaced a $40/mo GitHub Copilot subscription where I used Opus 4.6 and GPT 5.5 with a $10/mo opencode Go plan where I use mostly DeepSeek V4 Flash and testing MiMo 2.5. I work on mid-sized projects currently (200k to 1kk lines of code).
• dizhn 4mo ago
  FYI.. This is coming with 3mil GLM 5.2 tokens right now. (Needs login. Google SSO fine) https://zcode.z.ai/en https://zcode.z.ai/en
  • Alifatisk 4mo ago
    Where can I read more about the coming 3mil GLM 5.2?
    • dizhn 4mo ago
      I meant the credits are included in the application you download from there. Install, log in (via google) and You'll get glm 5.2 + turbo. Mine actually either got replenished or they are not checking by login because they are full after installing at a second desktop.
      • Alifatisk 4mo ago
        Oh, cool
• zftnb666 4mo ago
  Open-weight models are winning. The gap with closed models is now measured in months, not years.
• m-dot-reviews 4mo ago
  For anyone who's interested, I've put together a simple site for sharing ratings/opinions on models at a task-specific granularity. https://model.reviews/ https://model.reviews/ The idea is that benchmark score comparisons are useful for a large cross-product comparison across models + their settings, but less useful if you're looking for the best model for <your-specific-task>. So I thought having a place to review and comment could be beneficial to people. I'm not sure how best to get the corpus bootstrapped (i.e. people will likely only visit/post on the site if there's already activity), so posting it here for anyone who'd like to contribute.
  • swingboy 4mo ago
    I get a 500 when clicking “Explore the Models”
    • m-dot-reviews 4mo ago
      Oops, thanks for telling me that. I think the issue should be fixed now.
• KaoruAoiShiho 4mo ago
  This is really held back by one bench (omniscience accuracy) where it's really very far behind otherwise i think it's got at least a couple of points higher.
• ponyous 4mo ago
  Just ran and scored 63 3d model generations (via code) across high and no reasoning. 3D Modeling benchmark quickly shows spatial, logic and code performance of the model so I think it's a very good indicator of the quality. Here are the results compared to Gemini 3.5 Flash: Model + config CodeErr/gen Cost/gen Median time Quality gemini-3.5-flash, low 0.71 $0.18 68s baseline GLM 5.2, reasoning high 0.61 $0.18 289s -6.0% GLM 5.2, reasoning off 1.52 $0.10 126s -13.6% Although it is cheaper, it is significantly slower, and results are worse overall. Surprisingly - high reasoning produces less code errors than gemini 3.5 flash, but when I actually look at the models they are worse. Edit: I recently ran evals with Kimi 2.7 and MiniMax-M3 and this is clearly open source SOTA model, by far.
  • NiloCK 4mo ago
    Very interested in this! Can you share more about the modelling method (eg, three js?), the task list, and outputs here? I think there's probably some good juice to squeeze in terms of spacial awareness by doing a benchmark something like - give 3d modelling task - render and snapshot from a variety of angles - feed to third-party vision model for a "what is this" type query - grade on end-to-end accuracy Bonus points for asking the vision model something like "how beautiful is this 1-10".
    • ponyous 4mo ago
      I don't have the eval results live yet, so I cannot share them yet. I was benchmarking using a soon to be released new version of my AI CAD modeling software[0]. It's basically an agent that has access to tools that can execute build123d scripts, get sculpted models, blender to combine sculpts + parametric models, tools to inspect the model (visually and with code), search datasheets, ... I tried what you recommend a while ago (asking an AI to evaluate using different angles) and the AI evaluations were extremely bad - barely any correlation to what I scored. Things have gotten better, but I don't trust it enough yet. Here is how I score adherence (and how AI did as well, but I tried methods where it would just give back a boolean "pass" or not): <0.2 → Poor – Misses core intent; largely irrelevant or incorrect. <0.4 → Weak – Partially relevant; significant omissions or errors. <0.6 → Fair – Covers main points but lacks completeness or precision. <0.8 → Good – Mostly accurate; minor gaps or deviations. <=1.0 → Excellent – Fully aligned; precise, comprehensive, and faithful to intent. Here is the scenario list (prompts are much more detailed): dragon-bottle-stopper editing-param-mid-conv editing-parametric-enclosure editing-swap-material-param editing-text-edit-cube multi-turn-bird-house multi-turn-dice-tower multi-turn-modular-planter multi-turn-phone-stand multi-turn-shelf one-shot-bookend one-shot-cable-clip one-shot-chess-queen one-shot-coaster one-shot-coffee-cup one-shot-dog-tag one-shot-dragon-figurine one-shot-hex-bracket one-shot-keychain-fob one-shot-low-poly-tree one-shot-pegboard-hook one-shot-pi4-case one-shot-threaded-jar [0]: https://grandpacad.com https://grandpacad.com
• Computer0 4mo ago
  Regrettably I haven’t tried 5.2 yet but 5.1 I did not see as anything special. In practice I found it to be ~70% as good as Claude sonnet.
• piterrro 4mo ago
  DeepSeek v4 pro is still 10x cheaper than GLM-5.2 and the quality is still enough for 95% of coding tasks.
  • enraged_camel 4mo ago
    People always say stuff like this, but it is misleading. The reason it's misleading is because that remaining 5% makes a huge difference, and is where most of the value of using AI agents lies. I'm not interested in using AI to write code that would have taken me 5-10 minutes to write myself. I use AI to debug complex bugs and develop large features that span multiple domains - stuff that normally takes hours, if not days/weeks. A model that is "enough for 95%" does not cut it for that, because the failures compound during long-horizon tasks and the thing becomes a mess.
    • sinuhe69 4mo ago
      I get what you mean. But for many people, AI coding is not about solving complex problems. No, they do it mostly themselves. AI coding for many is a productivity tool, where it helps you with mundane, but laborious tasks. In my setup, I use a daily workhorse for such things. They should be fast, cheap and reasonably working well. I don’t expect it to be smart, but need it to follow instructions perfectly and handle tool calling well. For architectural work or debugging help, I use the top models instead. That works reasonably well for me with a low cost.
  • 0xbadcafebee 4mo ago
    ....so use DeepSeek v4 Pro for 95% of your coding tasks, and GLM 5.2 for the other 5%? You don't need to stick to one model.
• alansaber 4mo ago
  These open source models need better multi-turn capabilities. They are always lacklustre in "agent mode". Whether it's just less RL, whatever, it's a worse "product". Whereas it feels like the frontier labs have been all-in on "agentic" multi-turn reasoning for a long time now.
• Asfand3099 4mo ago
  [flagged]
• robertwt7 4mo ago
  what is that moodboard and chart of hypertension in the middle of the article that isn't explained? This is a great step up in open models however the pricing to support z.ai is not far cheaper than Claude / OpenAI subscription
• deleted 4mo ago
  [deleted]
• RDTvlokip 4mo ago
  I have a question, as it happens: Do you think the benchmarks and models were trained on benchmark datasets to skew the results, even though in real-world applications we realize they're not that great?
  • sinuhe69 4mo ago
    Recent incident with the Rio 3.5 model clearly shows that many coding models are specifically trained/fine tuned for the benchmarks.
    • RDTvlokip 4mo ago
      That's what I thought
• jayess 4mo ago
  I asked z.ai what z.ai is, and it said "It seems you might be referring to xAI, as "z.ai" isn't a widely known or major AI company or platform at this time."
• hyqzz8 4mo ago
  It is a very useful model
• leemoore 4mo ago
  GLM 5.2 feels like Opus 4.6 level. I actually think 4.6 and GLM work better in practice than opus 4.7 or 4.8 as I find both of those more erratic and seem to randomly have a super dumb turn. That random bad turn I see doesn't seem to be hitting the benchmark scores but they make 4.7 and 4.8 very hard to use for me. GLM is more stable like opus 4.6
• osti 4mo ago
  Fun fact: Zhipu aka Z.ai, Knowledge Atlas etc., the company that made GLM, is listed on Hong Kong stock exchange, is up over 10x since the IPO at the beginning of this year.
• hereme888 4mo ago
  Hmmm... GLM insists it's Gemini. https://github.com/zai-org/GLM-5/issues/79 https://github.com/zai-org/GLM-5/issues/79
  • adastra22 4mo ago
    Then why does it score better than any Gemini model?
    • hereme888 4mo ago
      As I understand, some people tend to "distill" LLM models. Google hasn't released a new Pro version in a while. I'm not an expert in LLMs.
  • bityard 4mo ago
    It's a surprisingly common misconception that models contain any metadata at all about themselves in their weights. If you ask them, "What model are you?" they either retrieve the answer from the system prompt, or they hallucinate an answer. Same goes for questions about knowledge cut-off, how many parameters they have, the source of their training data, etc.
    • hereme888 4mo ago
      Huh. That kinda makes sense. So you think it's hallucinating it's model name?
      • bityard 4mo ago
        Yes, it definitely is.
  • coder543 4mo ago
    Claude Sonnet 4.6 identified itself as DeepSeek repeatedly: https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_sonnet_46_says_its_deepseek_when_system/ https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_so... I tested this myself a few months ago, and confirmed that it was really happening. LLMs don't know who they are unless the system prompt tells them, and as all of them are trained on model responses that exist on the web that end up being scraped, the weights may predict a certain incorrect response. LLMs have no ability to introspect, and do not know anything about themselves, so they will hallucinate in response to that question unless they are carefully trained on that exact, pointless question.
• deleted 4mo ago
  [deleted]
• maxothex 4mo ago
  [flagged]
• adithyaharish 4mo ago
  why do not all open source LLM's have open weights like this model?
  • Retro_Dev 4mo ago
    "open source" means that the code itself (for LLMs - this is training code) is available to the general public. "open weights" means that the weights (trained over time) are available publicly, rather than locked behind a paywalled chat. I do not know of an open source LLM that is not also open weights (unless they never bothered training it). Models like Claude and Gemini are neither open source, nor are they open weights.
    • adithyaharish 4mo ago
      Got it, thanks for the thought
  • bigyabai 4mo ago
    https://en.wikipedia.org/wiki/Artificial_scarcity https://en.wikipedia.org/wiki/Artificial_scarcity
• gertlabs 4mo ago
  GLM 5.2 is the first model we've tested that is unambiguously on par with, or better than Opus 4.6 (although as usual, we have GLM 5.2 and most other Chinese models a bit below most other benchmarks with more vulnerable test methodologies). Data at https://gertlabs.com/rankings https://gertlabs.com/rankings
  • minraws 4mo ago
    [dead]
  • nsoonhui 4mo ago
    I really have to take your score with a grain of salt because Opus 4.5 does better than Opus 4.6
    • gertlabs 4mo ago
      They're within confidence intervals of each other, but remember how much discussion there was that Opus 4.6 had been nerfed in March. We averaged samples over the entire lifetime of Opus 4.6, which likely served many different underlying checkpoints. Even the best version of Opus 4.6 was hardly an upgrade. We find a lot of interesting anomalies with our benchmark that hold up under large sample sizes.
• guybedo 4mo ago
  It's probably a good model but they used GLM 5.1 to code their infra. I signed up to their max plan yesterday, did some light coding work, and i'm at 180M tokens used and 40% weekly quota gone. Even when tokenmaxxing on the Claude Max or GPT $200 plan, i couldn't get more than 20% quota gone per day.
  • bigyabai 4mo ago
    Are you using it for long context windows? I burn through my 5hr quota with GLM almost instantly on 200k+ contexts, but if I reset every ~100k or so it's much more manageable.
• daniban 4mo ago
  I'm curious what harness everyone is using for these? I want to start to test some of these open models but don't know what tools people use to get these working "agenticaly"
  • zackify 4mo ago
    pi.dev and ask ai to add features you miss from claude or codex. i configure keyboard shortcuts and swap models easily
  • gorbypark 4mo ago
    I am using OpenCode with the DeepSeek API with some pretty good results.
• SwellJoe 4mo ago
  I added it to my benchmark based on Mythos-reported bugs, and it's better than GLM 5.1, but still behind several other models, maybe most directly comparable to Qwen 3.7 Max. But, several other open models, including small self-hostable ones (Gemma 4 and Qwen 3.6), found the same number of bugs, 3 of 9. Though it also gets partial credit for reporting one bug in the right spot, but kinda misunderstanding the bug. I also added Kimi K2.7-code in the same run, and it did poorly, consistent with 2.6 performance. Anyway, there are better, cheaper, models on this particular benchmark. https://swelljoe.com/post/will-it-mythos/ https://swelljoe.com/post/will-it-mythos/ (This small benchmark doesn't prove anything. It's a limited data set and each model only gets one shot at each file in the corpus. But, I find it useful for quickly sussing out if a model can reason about pretty complicated problems in code.)
  • be7a 4mo ago
    [dead]
• jauntywundrkind 4mo ago
  Also so wild that it's relatively compact. 753B-40A is so reasonable, shows incredible scaling in what the model can do, without just throwing heaps of new parameters in. This is silly but I dig how 753 is very close to 745, which is the watts in a HP. 1bHP parameter model. Silly, but I enjoy it.
• mesmertech 4mo ago
  Seems really good at frontend work, and as a result on remotion programmatic videos. Not the best yet, thats still Gemini 3.1 pro(trained on actual videos) or Fable, but often better than what Opus can come up with https://mesmer.tools/benchmarks/ai-video-generation https://mesmer.tools/benchmarks/ai-video-generation
• redbell 4mo ago
  Launch announcement from four days ago: https://news.ycombinator.com/item?id=48518684 https://news.ycombinator.com/item?id=48518684 The requirements to run this model locally: https://www.reddit.com/r/LocalLLaMA/comments/1u8ai2a/glm52_is_a_win_for_local_ai/ https://www.reddit.com/r/LocalLLaMA/comments/1u8ai2a/glm52_i...
• PetrBrzyBrzek 4mo ago
  I'm a bit shocked that GLM 5.2 is not multimodal. Like, how should I use it? I use images all the time.
• hottrends 4mo ago
  [flagged]
• aunty_helen 4mo ago
  Before you go and sign up to the max plan like I did, they are obviously struggling for capacity. I'm getting API rate limited and 429'd on a simple "hello"
• tomerbd 4mo ago
  I code daily with AI - real programming tasks, professional, real work, read customers, I use below 3: - codex 5.5 medium - best results less hand holding medium speed - opus 4.8 max - mediocre with hand holding medium speed - glm 5.2 max - mediocre with hand holding and super slow - composer 2.5 - mediocre with hand holding and super fast I use all, since i run mulitple coding in parallel. disclosure - I use rexide which we created for all these agents to run in parallel with good visibility and feedback.
• gauravvij137 4mo ago
  They've come along pretty far now. I remember when there was hype around GLM 5 reaching great heights on benchmarks but eventually failing on practical coding and reasoning tasks. I guess this time the hype is real.
• bizer 4mo ago
  Z-ai/GLM’s KV caching technology is truly impressive; the implicit cache hit rate of its official API exceeds 95%, far surpassing other APIs that support implicit caching, such as Gemini and Qwen. I’ve been pondering the architectural design behind this, though I haven't yet formed a fully coherent theory.
• catigula 4mo ago
  Which American model did they distill this one from?
• casey2 4mo ago
  Mark my words, by the end of 2027, there will be an open weights model that is better than anything OpenAI and Anthropic are capable of making. They will lose at inference scaling too.