4 ms·
$1/M input tokens and $5/M output tokens is good compared to Claude Sonnet 4.5 but nowadays thanks to the pace of the industry developing smaller/faster LLMs fo
by minimaxir 1y ago
$1/M input tokens and $5/M output tokens is good compared to Claude Sonnet 4.5 but nowadays thanks to the pace of the industry developing smaller/faster LLMs for agentic coding, you can get comparable models priced for much lower which matters at the scale needed for agentic coding.
Given that Sonnet is still a popular model for coding despite the much higher cost, I expect Haiku will get traction if the quality is as good as this post claims.
- Bolwin 1y agoWith caching that's 10 cents per million in. Most of the cheap open source models (which this claims to beat, except glm 4.6) have limited and not as effective caching. This could be massive.
- logicchains 1y ago$1/M is hardly a big improvement over GPT5's $1.250/M (or Gemini Pro's $1.5/M), and given how much worse Haiku is than those at any kind of difficult problem (or problems with a large context size), I can't imagine it being a particularly competitive alternative for coding. Especially for anything math/logic related, I find GPT5 and Gemini Pro to be significantly better even than Opus (which reflects in their models having won Olympiad prizes while Anthropic's have not).
- HarHarVeryFunny 1y agoGPT-5 is $10/M for output tokens, twice the cost of Haiku 4.5 at $5/M, despite Haiku apparently being better at some tasks (SWE Bench). I suppose it depends on how you are using it, but for coding isn't output cost more relevant than input - requirements in, code out ?
- criemen 1y ago> I suppose it depends on how you are using it, but for coding isn't output cost more relevant than input - requirements in, code out ? Depends on what you're doing, but for modifying an existing project (rather than greenfield), input tokens >> output tokens in my experience.
- logicchains 1y agoUnless you're working on a small greenfield project, you'll usually have 10s-100s of thousands of relevant words (~tokens) of relevant code in context for every query, vs a few hundred words of changes being output per query. Because most changes to an existing project are relatively small in scope.
- Tiberium 1y agoThe funny thing is that even in this area Anthropic is behind other 3 labs (Google, OpenAI, xAI). It's the only one out of those 4 that requires you to manually set cache breakpoints, and the initial cache costs 25% more than usual context. The other 3 have fully free implicit caching. Although Google also offers paid, explicit caching. https://docs.claude.com/en/docs/build-with-claude/prompt-caching https://docs.claude.com/en/docs/build-with-claude/prompt-cac... https://ai.google.dev/gemini-api/docs/caching https://ai.google.dev/gemini-api/docs/caching https://platform.openai.com/docs/guides/prompt-caching https://platform.openai.com/docs/guides/prompt-caching https://docs.x.ai/docs/models#cached-prompt-tokens https://docs.x.ai/docs/models#cached-prompt-tokens
- tempusalaria 1y agoI vastly prefer the manual caching. There are several aspects of automatic caching that are suboptimal, with only moderately less developer burden. I don’t use Anthropic much but I wish the others had manual cache options
- simonw 1y agoWhat's sub-optimal about the OpenAI approach, where you get 90% discount on tokens that you've previously sent within X minutes?
- stavros 1y agoIs it wherever the tokens are, or is it the N first tokens they've seen before? Ie if my prompt is 99% the same, except for the first token, will it be cached?
- simonw 1y agoThe prefix has to be stable. If you are 99% the same but the first token is different it won't cache at all. You end up having to design your prompts to accommodate this.
- simonw 1y agoYeah, I'm a bit disappointed by the price. Claude 3.5 Haiku was $0.8/$4, 4.5 Haiku is $1/$5. I was hoping Anthropic would introduce something price-competitive with the cheaper models from OpenAI and Gemini, which get as low as $0.05/$0.40 (GPT-5-Nano) and $0.075/$0.30 (Gemini 2.0 Flash Lite).
- odie5533 1y agoThere's probably less margin on the low end, so they don't want to focus on capturing it.
- dr_dshiv 1y agoMargin? Hahahahaha
- odie5533 1y agoInference is profitable.
- reppap 1y agoIf you completely ignore inference revenue needing to offset training costs. Is inference still profitable if you account for the amortized training cost?
- yunwal 1y ago> If you completely ignore inference revenue needing to offset training costs. This is what people mean when they say margin. When you buy a pair of shoes, the margin is price/(materials+labor), and doesn’t include the price of the factory or the store they were bought in
- simonw 1y agoNot for the big labs, who are engaged in an astonishingly competitive buildout right now. There are a bunch of companies who offer inference against open weight models trained by other people. They get to skip the training costs.
- rudedogg 1y agoThis also means API usage through Claude Code got more expensive (but better if benchmarks are to be believed)
- justinbaker84 1y agoI am a professional developer so I don't care about the costs. I would be willing to pay more for 4.5 Haiku vs 4.5 Sonnet because the speed is so valuable. I spend way to much time waiting for the cutting edge models to return a response. 73% on SWE Bench is plenty good enough for me.
- jhancock 1y agoHow do you review code when the LLM can produce so much so fast?
- evan_ 1y agowith an LLM
- justinbaker84 1y agoJust read it when it is done writing it.