4 ms·
People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowada
by ramon156 2mo ago
People will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick.
I care a lot about speed and efficiency right now. For my setup I would like to have 2-3 different model families. I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting, Deepseek V4 Pro 0813 for developing, and Gemini flash lite (any recent cheap model) for repo scouting. I'll add another one in the mix for reviewing (in this case Gemini 3.7) and that's all I need.
I've tried most models except Grok.
Qwen is too expensive IMO (Alibaba Cloud subscriptions are hard to come by and I'm not spending 50 euros a month for a tool, so 18 euros it is). If it ever becomes efficient enough to run locally I will definitely look back.
Claude is slow and expensive (the cache hit prices are absurd).
OAI is pretty good, I might add it to my arsenal seeing how cheap it is.
These opinions change every day. Last week I would've never picked Deepseek until I read about the pricing. even post aug 16 it's worth it (although it's getting close to gemini pricing).
Right now my costs are 12 euros a month (z.ai) + whatever deepseek consumes. This typically isn't more than 8 euros a week. 44 euros a month and I have a setup that is doing pretty well.
- hypfer 2mo ago> I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting Dude, GLM-5.3 released _today_. The phrasing "I've settled on" is incorrect for this context.
- ramon156 2mo agohence the "former deepseek v4 pro". I tried it out this morning and have had no complaints. I already liked glm 5.2
- hypfer 2mo agoThe sentence still doesn't make sense, because "settled on" implies a long testing phase with a verdict eventually emerging out of that. What you're currently doing is "testing out"
- Topfi 2mo agoHonest question, how do you assess models this quickly? What metrics are you using? Would love to get my suite from multiple days and hundreds of prompts down to minutes. Got a few first pass tasks I run upon release for an initial experience, but those only work because even Fable and Sol fail despite objectively correct solutions existing, so it works because most models fail, but then, those are consciously not enough for coding, tool use, adherence or task specific inference and assessment…
- satvikpendem 2mo agoWhat are you working on? That can dictate which models are best.
- Topfi 2mo agoAnything from ML pipelines for language specific pruning over a Rust/JS/CSS mix codebase to assistance in motorcycle maintenance and different canvas coatings. Most of my evals build on those requirements and especially past failures, whether in pure information, task execution and coding or tool calling beyond the overfitted mainstream. All stuff derived from actual failures encountered, some still only few models come even close to passing. With such a mix, it just takes a while to get any serious opinion on a model. Doubt anyone can do that in such short time, unless their tasks are so simple that most modern models not only succeed but could themselves accurately rate output. If even Fable still confuses PU or wax coated cotton canvas with a nylon shell, or tells me with a straight face to adjust the valves on a bike that has hydraulic lifters that needs experience for human assessment and the time that comes with it. Anyone with less knowledge either wouldn’t see the mistakes staring them in the face and just go by vibes, any model rating these equally can not tell what is accurate and will just go by the output sounding accurate over being. Gives sometimes very interesting results far different to public benchmarks. Inkling, e.g. is more accurate in not telling you to adjust valves that are simply not adjustable then Fable or Sol, which just tell you to adjust every 5000km. Sometimes even when their reasoning and search includes sections about the fact this is not necessary or possible. The beauty of overfitting and unbalanced training data…
- kristjansson 2mo ago> Deepseek v4 pro 0813 Which itself released yesterday? You're writing, reading, and evaluating enough software in a ~36 hour period to form, reject, and form another opinion about which model makes better _architectural_ choices?
- throw310822 2mo agoIt's the singularity, baby.
- mlsu 2mo agoHe's 100x, bro. 36 hours is basically like a whole 2 week sprint for you.
- stusmall 2mo agoI'm generally extremely skeptical about a lot of the model hype that show up in comments. Except when there is an extreme mismatch the performance, quirks and quality of these things are difficult to nail down. You wouldn't know that from the comment section of every single release. I think some of these are excited, eager users always ready to hype up the new thing. The same crowd that previously would constantly push for a rewrite from angular->react->svelt->god knows what. Instead now it is on a 6 week cycle and about models/harnesses. I think most of it is bot driven spam by the various labs. It's hard not to notice 3 month old accounts with very strong opinions about various frontier labs and little else. Ultimately, I think some of it is legitimate shifts in who's in lead and what is the best. You gotta dig through a lot of crap to get to that, and I don't really know how to to. Ultimately I'm saying is that I always applied a fair amount of skepticism about what I see in comment sections but these day it is extreme amounts.
- EB66 2mo ago> I think most of it is bot driven spam by the various labs. It's hard not to notice 3 month old accounts with very strong opinions about various frontier labs and little else. I've assumed the same as well. I also assume that many of the companies developing these models engage in benchmaxxing. At my company we've developed our own internal benchmarks for evaluating LLM models as they become available. The benchmarks are tailored to our particular use cases but the utility and knowledge our benchmarks assess is still fairly universally applicable. I see wide differences between what our internal benchmarks report and what the major benchmarks do. There was a whole lot of fanfare about how amazing GLM 5.2 was when it was released, but it was pure rubbish on our internal benchmark -- far behind OpenAI, Anthropic, Gemini, DeepSeek, etc. I don't know how to reconcile the fact that GLM 5.2 performed very well on some of the major public benchmarks, but consistently performs so poorly on ours. OpenAI models tend to dominate our internal benchmarks.
- ebhc 2mo agoThis resonates. You need to tailor evaluation to your own use cases in order to know if you should adopt the next thing or not.
- simplyluke 2mo agoI'm convinced a lot of the anti-open-weight model comments at this point are inorganic traffic - there's trillions in investor money riding on a world where these models aren't cheap commodities. Having actually used things like the recent GLM, Kimi, and Qwen I think any edge the labs have is marginal at most and actually prefer the open weight models in most day to day usage. Anthropic's recent releases are wordy to the point of exhaustion. Every time I use opus recently I find myself wanting to yell "GET TO THE POINT" at a terminal, which is exacerbated by it being slow.
- mococa 2mo agoOpus is pretty dumb and lazy nowadays, cancelled my subscription.
- orangedog 2mo agoWhy are you convinced of that? Pretty much every time I see statements like that online, I can find plenty of organic traffic supporting it, not everything is a bot. I just wouldn't bias myself that way, most people haven't really used local models. This stuff is pretty much all subjective evaluation, there's plenty of reasons for people to favor certain models or disfavor others.
- anana_ 2mo agoWhen stuff like this: https://doublespeed.ai/ https://doublespeed.ai/ exists I don't find that hard to believe at all, although it cuts both ways
- tosh 2mo agoi think you will like luna if you haven't tried it yet
- johnnyApplePRNG 2mo agoLuna is twice the price of Deepseek V4 Flash 0731, and less capable :/
- copperx 2mo agoDidn't DeepSeek go up 2x?
- alecsm 2mo agoUp to 4x but in 2 days.
- johnnyApplePRNG 2mo agoThe price on DeepInfra and most providers on OpenRouter has not changed... if anything, it's gotten cheaper. All of this DeepSeek price increase stuff is related to their specific API, not the entire market.
- farlight 2mo agoThey were significantly more expensive for coding compared to DeepSeek's offering because of much higher cost of cache reads and lower cache hit rate. Now, after the price jump, the effective cost of deepseek.com API vs 3-rd party providers is about the same.
- johnnyApplePRNG 2mo agoDeepSeek's API was never "cheap". They were stealing your code for that extremely cheap cached token price. That was never the real price. Every Chinese LLM API provider is the same to the best of my knowledge. "CHEAP API" (but pssst we train on every single input aka your code and ideas are ours forever)
- satvikpendem 2mo agoYou should check out Grok, it's quite a good deal from the Cursor subscription side but it's cheap even by API prices.
- ai_fry_ur_brain 2mo ago[dead]
- altruios 2mo ago[flagged]
- satvikpendem 2mo agoYeah I'm sure everyone on r/cursor or in previous HN threads about Grok 4.5 or 4.6 are all unserious and insane. No one actually cares about the politics as long as the model codes well. Edit, quite interesting to see the reception to this comment compared to essentially the same type of comment I made on a Grok 4.6 benchmark HN post: https://news.ycombinator.com/item?id=49275385#49275571 https://news.ycombinator.com/item?id=49275385#49275571 It's true that Cursor gives a lot of usage with Grok, most users of Cursor don't care about Musk.
- kennywinker 2mo agoThis is the “Mussolini made the trains run on time” of ai hot takes. (Btw, mussolini didn’t make the trains run on time)
- d2kx 2mo ago> Alibaba Cloud subscriptions are hard to come by no they aren't. they discontinued their always-sold-out coding plan and launched QwenCloud (basically a friendly frontend with Alibaba Cloud as the hidden backend) and launched typical subscription plans for Qwen & co alongside it.
- lacoolj 2mo agoGrok 4.6 is a game-changer. I have yet to go back to other models after starting to use it. You just can't beat the price + output quality (even K3 is more expensive)
- pbhjpbhj 2mo agoSupporting a far-right megalomaniac, whilst helping them to train their ML, and giving them all your data ... what could go wrong.
- whatshisface 2mo agoI try to do everything with open-weight models, to support the companies who support me.
- focxle 2mo ago[flagged]