5 ms·

I've been playing around with GLM-4.5 as a coding model for a while now and it's really, really good. In the coding agent I've been working on, Octofriend [1],

I've been playing around with GLM-4.5 as a coding model for a while now and it's really, really good. In the coding agent I've been working on, Octofriend [1], I've sometimes had it on and confused it for Claude 4. Subjectively, my experience has been: 1. Claude is somewhat better at whole-codebase tasks, where you need to reason over a bunch of context and consider system interactions. 2. GLM-4.5 is somewhat better at being "honest" — i.e. I rarely see it doing the things Claude does like making broken tests pass by changing the test instead of fixing the bug. Both are quite good though, and GLM-4.5 has found bugs that both Claude 4 Sonnet and 4.1 Opus have failed to catch. In general I think Claude wins a little more frequently on debugging tasks than GLM-4.5, but it's close. Compared to GPT-5, both Claude and GLM feel like they're more consistent, although GPT-5 sometimes has long brilliant runs where it nails everything with subjectively higher code quality than either of the latter. However, once GPT-5 goes off the rails, it's hard to get it back on track, so it can be a bit frustrating to work with in comparison. 1: https://github.com/synthetic-lab/octofriend https://github.com/synthetic-lab/octofriend

nico 1y ago
How are you using glm-4.5? Are you consuming the api or running something like glm-4.5 air locally?
- reissbaker 1y ago
  I run a privacy-focused inference company, Synthetic [1], and I use our API of course :P I actually like GLM-4.5 enough that it's currently our default recommended model for new users. But yes, otherwise I'd use the official zai API most likely, or Fireworks. GLM-4.5-Air is quite good for a local model but GLM-4.5 is better; up to you if the tradeoff is worth it — there's definitely value in the data not ever leaving your machine, but it's not going to be as strong of a model. 1: https://synthetic.new https://synthetic.new
  - azinman2 1y ago
    I’m curious for your service, if it’s centered around privacy, why is the data stored for 14 days at all? My understanding with fireworks is that it’s 0 logging — nothing to store. To me that’s private.
    - reissbaker 1y ago
      Great question! We actually don't store prompts or completions for the API at all — but legally we say 14 days so that if someone accidentally commits a log statement, we're not in breach as long as we catch it quickly and revert.
  - mrtesthah 1y ago
    Amazing! So I’m assuming that because it’s privacy focused, you accept payment in cryptocurrencies like Monero and Zcash?
    - reissbaker 1y ago
      We accept USDC and USDP crypto payments via Stripe. We don't currently support Monero or Zcash — right now all our payments are via Stripe since it simplifies security + compliance for us. It would be a pretty neat feature to build though.
  - throwdbaaway 1y ago
    You support logprobs, that's wonderful! Fireworks, Synthetic, (ik_)llama.cpp, now I have a quorum.
  - fariszr 1y ago
    What makes your service especially privacy friendly? I think if you are striving for full privacy, you should implement the secure enclave idea presented by ollama, it makes the entire pipeline fully encrypted, I'm waiting for an actual provider to finally implement this. https://ollama.com/blog/secureminions https://ollama.com/blog/secureminions
    - reissbaker 1y ago
      We don't store prompts or completions for the API (our privacy policy says "for longer than 14 days," as mentioned elsewhere in this thread — we don't actually store them at all, but the 14 day legal guarantee is to make sure that if someone accidentally commits a log statement, we have a little bit of time to catch it and revert without being in breach of policy). And, we don't train on your data, even for messages in the UI: we only store UI messages in order to let you view your message history, not for training. Compared to using — for example — DeepSeek from deepseek.com, I think we're much more private. Even compared to using OpenAI and opting-out of your data being used for training, we're still more private, since OpenAI makes no guarantees for individuals that they don't store the data — notably, any data ever sent to them is apparently now being shared with New York courts (and the New York Times!) due to their ongoing legal battle with the New York Times [1]. And compared to using OpenRouter with "data_collection: deny", we uh, actually work :P Surprisingly sad how many broken model implementations there are if you're just round-robin-ing between inference companies... Especially reasoning models, and especially with tool-calling. (And if something's broken, you can email us and we'll generally fix it; OpenRouter doesn't actually host any models themselves, so there's not much they can do if one isn't working well other than just de-list.) 1: https://arstechnica.com/tech-policy/2025/07/nyt-to-start-searching-deleted-chatgpt-logs-after-beating-openai-in-court/ https://arstechnica.com/tech-policy/2025/07/nyt-to-start-sea...
      - traceroute66 1y ago
        Do you let people explicitly choose EU servers ? Also, your Privacy Policy is not currently EU GDPR compliant. ;-)
- sagarpatil 1y ago
  Not OP. Chutes.ai charges $0.20 per 1M tokens. I don’t think it uses caching though because I ended up burning $30 in an hour or two. I had to move back to Claude Code.
  - esafak 1y ago
    Caching makes price comparisons hard. Does anyone have tips?
UncleOxidant 1y ago
I just read your comment and decided to give GLM-4.5 a try in Kilocode. I'd been using Gemini CLI all day to try to resolve a tricky bug in some compiler code (a compiler for a subset of C that generates microcode for... a weird architecture, I'll leave it at that). So GLM-4.5 zoomed in on the problem right away. A problem that's eluded Gemini CLI all day. Gemini was leading me on a wild goose chase implicating a function that turns out wasn't the problem (and trying to make all kinds of lame changes to the function saying that would fix the problem - and it never did because the problem wasn't that function).
- p4coder 1y ago
  Sometimes getting a second pair of eyes to look at the problem helps and is usually not a judgement of smartness of the first pair of eyes. Seems like it also applies to coding agents.
  - diggan 1y ago
    Indeed, I've also found that various models are good at various tasks, but I have yet been able to categorize "Model X is good at Y-class of bugs", so I end up using N models for a first pass "Find the root-cause of this issue", then once it's found, pass it along to same N models for them to attempt to solve it. So far, which model can find/solve what is really scattered all over the place.
    - energy123 1y ago
      This is so true. Another thing, a model might be better at something in general, but worse if the context is too long. Looking at how GLM-4.5 is trained, on lots of short context, this may be the case for it. GPT-5: Exceptional at abstract reasoning, planning and following the intention behind instructions. Concise and intentional. Not great at manipulating text or generating python code. Gemini 2.5 Pro: Exceptional at manipulating text and python, not great at abstract reasoning. Verbose. Doesn't follow instructions well. Another thing I've learned is that models work better when they work on code that they themselves generated. It's "in distribution" and more comprehensible to them.
faangguyindia 1y ago
I've been using architect mode in aider Deepseek R1 (does high level planning) combined with Qwen3 480B (does low level coding) or whatever is available from qwen code apis. It's working great. It solves 99.99% problem on tis own. The seperation isn't very good in aider so i later plan to make my own tool to achieve better workflow.
- manmal 1y ago
  What’s your monthly bill (OpenRouter?) if I may ask? I have Claude Max and always on the lookout for alternatives, at least for the easier to solve problems.
nmfisher 1y ago
I've had similarly good experiences with GLM-4.5 for smaller projects/requests. Unfortunately that did degrade with larger contexts, so I'm still treating it as a good fallback for Sonnet 4, rather than a full-blown replacement.
mrklol 1y ago
About your first point, I also feel like Claude is better if there’s more in the context where 4.5 is getting "worse".
- faangguyindia 1y ago
  claude used to be better not anymore or atleast the difference is not that much deepseek r1+ qwen3 is close enough along with gemini2.5 pro so i don't see any point of claude anymore
deleted 1y ago
[deleted]

nico 1y ago

How are you using glm-4.5? Are you consuming the api or running something like glm-4.5 air locally?

reissbaker 1y ago
I run a privacy-focused inference company, Synthetic [1], and I use our API of course :P I actually like GLM-4.5 enough that it's currently our default recommended model for new users. But yes, otherwise I'd use the official zai API most likely, or Fireworks. GLM-4.5-Air is quite good for a local model but GLM-4.5 is better; up to you if the tradeoff is worth it — there's definitely value in the data not ever leaving your machine, but it's not going to be as strong of a model. 1: https://synthetic.new https://synthetic.new
- azinman2 1y ago
  I’m curious for your service, if it’s centered around privacy, why is the data stored for 14 days at all? My understanding with fireworks is that it’s 0 logging — nothing to store. To me that’s private.
  - reissbaker 1y ago
    Great question! We actually don't store prompts or completions for the API at all — but legally we say 14 days so that if someone accidentally commits a log statement, we're not in breach as long as we catch it quickly and revert.
- mrtesthah 1y ago
  Amazing! So I’m assuming that because it’s privacy focused, you accept payment in cryptocurrencies like Monero and Zcash?
  - reissbaker 1y ago
    We accept USDC and USDP crypto payments via Stripe. We don't currently support Monero or Zcash — right now all our payments are via Stripe since it simplifies security + compliance for us. It would be a pretty neat feature to build though.
- throwdbaaway 1y ago
  You support logprobs, that's wonderful! Fireworks, Synthetic, (ik_)llama.cpp, now I have a quorum.
- fariszr 1y ago
  What makes your service especially privacy friendly? I think if you are striving for full privacy, you should implement the secure enclave idea presented by ollama, it makes the entire pipeline fully encrypted, I'm waiting for an actual provider to finally implement this. https://ollama.com/blog/secureminions https://ollama.com/blog/secureminions
  - reissbaker 1y ago
    We don't store prompts or completions for the API (our privacy policy says "for longer than 14 days," as mentioned elsewhere in this thread — we don't actually store them at all, but the 14 day legal guarantee is to make sure that if someone accidentally commits a log statement, we have a little bit of time to catch it and revert without being in breach of policy). And, we don't train on your data, even for messages in the UI: we only store UI messages in order to let you view your message history, not for training. Compared to using — for example — DeepSeek from deepseek.com, I think we're much more private. Even compared to using OpenAI and opting-out of your data being used for training, we're still more private, since OpenAI makes no guarantees for individuals that they don't store the data — notably, any data ever sent to them is apparently now being shared with New York courts (and the New York Times!) due to their ongoing legal battle with the New York Times [1]. And compared to using OpenRouter with "data_collection: deny", we uh, actually work :P Surprisingly sad how many broken model implementations there are if you're just round-robin-ing between inference companies... Especially reasoning models, and especially with tool-calling. (And if something's broken, you can email us and we'll generally fix it; OpenRouter doesn't actually host any models themselves, so there's not much they can do if one isn't working well other than just de-list.) 1: https://arstechnica.com/tech-policy/2025/07/nyt-to-start-searching-deleted-chatgpt-logs-after-beating-openai-in-court/ https://arstechnica.com/tech-policy/2025/07/nyt-to-start-sea...
    - traceroute66 1y ago
      Do you let people explicitly choose EU servers ? Also, your Privacy Policy is not currently EU GDPR compliant. ;-)
sagarpatil 1y ago
Not OP. Chutes.ai charges $0.20 per 1M tokens. I don’t think it uses caching though because I ended up burning $30 in an hour or two. I had to move back to Claude Code.
- esafak 1y ago
  Caching makes price comparisons hard. Does anyone have tips?

UncleOxidant 1y ago

I just read your comment and decided to give GLM-4.5 a try in Kilocode. I'd been using Gemini CLI all day to try to resolve a tricky bug in some compiler code (a compiler for a subset of C that generates microcode for... a weird architecture, I'll leave it at that). So GLM-4.5 zoomed in on the problem right away. A problem that's eluded Gemini CLI all day. Gemini was leading me on a wild goose chase implicating a function that turns out wasn't the problem (and trying to make all kinds of lame changes to the function saying that would fix the problem - and it never did because the problem wasn't that function).

p4coder 1y ago
Sometimes getting a second pair of eyes to look at the problem helps and is usually not a judgement of smartness of the first pair of eyes. Seems like it also applies to coding agents.
- diggan 1y ago
  Indeed, I've also found that various models are good at various tasks, but I have yet been able to categorize "Model X is good at Y-class of bugs", so I end up using N models for a first pass "Find the root-cause of this issue", then once it's found, pass it along to same N models for them to attempt to solve it. So far, which model can find/solve what is really scattered all over the place.
  - energy123 1y ago
    This is so true. Another thing, a model might be better at something in general, but worse if the context is too long. Looking at how GLM-4.5 is trained, on lots of short context, this may be the case for it. GPT-5: Exceptional at abstract reasoning, planning and following the intention behind instructions. Concise and intentional. Not great at manipulating text or generating python code. Gemini 2.5 Pro: Exceptional at manipulating text and python, not great at abstract reasoning. Verbose. Doesn't follow instructions well. Another thing I've learned is that models work better when they work on code that they themselves generated. It's "in distribution" and more comprehensible to them.

faangguyindia 1y ago

I've been using architect mode in aider Deepseek R1 (does high level planning) combined with Qwen3 480B (does low level coding) or whatever is available from qwen code apis. It's working great. It solves 99.99% problem on tis own. The seperation isn't very good in aider so i later plan to make my own tool to achieve better workflow.

manmal 1y ago
What’s your monthly bill (OpenRouter?) if I may ask? I have Claude Max and always on the lookout for alternatives, at least for the easier to solve problems.

nmfisher 1y ago

I've had similarly good experiences with GLM-4.5 for smaller projects/requests. Unfortunately that did degrade with larger contexts, so I'm still treating it as a good fallback for Sonnet 4, rather than a full-blown replacement.

mrklol 1y ago

About your first point, I also feel like Claude is better if there’s more in the context where 4.5 is getting "worse".

faangguyindia 1y ago
claude used to be better not anymore or atleast the difference is not that much deepseek r1+ qwen3 is close enough along with gemini2.5 pro so i don't see any point of claude anymore

deleted 1y ago

[deleted]