22 ms·

Claude Haiku 4.5

730 points by adocomplete 1y ago

System card: https://assets.anthropic.com/m/99128ddd009bdcb/original/Claude-Haiku-4-5-System-Card.pdf https://assets.anthropic.com/m/99128ddd009bdcb/original/Clau...

minimaxir 1y ago
$1/M input tokens and $5/M output tokens is good compared to Claude Sonnet 4.5 but nowadays thanks to the pace of the industry developing smaller/faster LLMs for agentic coding, you can get comparable models priced for much lower which matters at the scale needed for agentic coding. Given that Sonnet is still a popular model for coding despite the much higher cost, I expect Haiku will get traction if the quality is as good as this post claims.
- Bolwin 1y ago
  With caching that's 10 cents per million in. Most of the cheap open source models (which this claims to beat, except glm 4.6) have limited and not as effective caching. This could be massive.
  - logicchains 1y ago
    $1/M is hardly a big improvement over GPT5's $1.250/M (or Gemini Pro's $1.5/M), and given how much worse Haiku is than those at any kind of difficult problem (or problems with a large context size), I can't imagine it being a particularly competitive alternative for coding. Especially for anything math/logic related, I find GPT5 and Gemini Pro to be significantly better even than Opus (which reflects in their models having won Olympiad prizes while Anthropic's have not).
    - HarHarVeryFunny 1y ago
      GPT-5 is $10/M for output tokens, twice the cost of Haiku 4.5 at $5/M, despite Haiku apparently being better at some tasks (SWE Bench). I suppose it depends on how you are using it, but for coding isn't output cost more relevant than input - requirements in, code out ?
      - criemen 1y ago
        > I suppose it depends on how you are using it, but for coding isn't output cost more relevant than input - requirements in, code out ? Depends on what you're doing, but for modifying an existing project (rather than greenfield), input tokens >> output tokens in my experience.
      - logicchains 1y ago
        Unless you're working on a small greenfield project, you'll usually have 10s-100s of thousands of relevant words (~tokens) of relevant code in context for every query, vs a few hundred words of changes being output per query. Because most changes to an existing project are relatively small in scope.
  - Tiberium 1y ago
    The funny thing is that even in this area Anthropic is behind other 3 labs (Google, OpenAI, xAI). It's the only one out of those 4 that requires you to manually set cache breakpoints, and the initial cache costs 25% more than usual context. The other 3 have fully free implicit caching. Although Google also offers paid, explicit caching. https://docs.claude.com/en/docs/build-with-claude/prompt-caching https://docs.claude.com/en/docs/build-with-claude/prompt-cac... https://ai.google.dev/gemini-api/docs/caching https://ai.google.dev/gemini-api/docs/caching https://platform.openai.com/docs/guides/prompt-caching https://platform.openai.com/docs/guides/prompt-caching https://docs.x.ai/docs/models#cached-prompt-tokens https://docs.x.ai/docs/models#cached-prompt-tokens
    - tempusalaria 1y ago
      I vastly prefer the manual caching. There are several aspects of automatic caching that are suboptimal, with only moderately less developer burden. I don’t use Anthropic much but I wish the others had manual cache options
      - simonw 1y ago
        What's sub-optimal about the OpenAI approach, where you get 90% discount on tokens that you've previously sent within X minutes?
        stavros 1y ago
        Is it wherever the tokens are, or is it the N first tokens they've seen before? Ie if my prompt is 99% the same, except for the first token, will it be cached?
        simonw 1y ago
        The prefix has to be stable. If you are 99% the same but the first token is different it won't cache at all. You end up having to design your prompts to accommodate this.
- simonw 1y ago
  Yeah, I'm a bit disappointed by the price. Claude 3.5 Haiku was $0.8/$4, 4.5 Haiku is $1/$5. I was hoping Anthropic would introduce something price-competitive with the cheaper models from OpenAI and Gemini, which get as low as $0.05/$0.40 (GPT-5-Nano) and $0.075/$0.30 (Gemini 2.0 Flash Lite).
  - odie5533 1y ago
    There's probably less margin on the low end, so they don't want to focus on capturing it.
    - dr_dshiv 1y ago
      Margin? Hahahahaha
      - odie5533 1y ago
        Inference is profitable.
        reppap 1y ago
        If you completely ignore inference revenue needing to offset training costs. Is inference still profitable if you account for the amortized training cost?
        yunwal 1y ago
        > If you completely ignore inference revenue needing to offset training costs. This is what people mean when they say margin. When you buy a pair of shoes, the margin is price/(materials+labor), and doesn’t include the price of the factory or the store they were bought in
        simonw 1y ago
        Not for the big labs, who are engaged in an astonishingly competitive buildout right now. There are a bunch of companies who offer inference against open weight models trained by other people. They get to skip the training costs.
- rudedogg 1y ago
  This also means API usage through Claude Code got more expensive (but better if benchmarks are to be believed)
- justinbaker84 1y ago
  I am a professional developer so I don't care about the costs. I would be willing to pay more for 4.5 Haiku vs 4.5 Sonnet because the speed is so valuable. I spend way to much time waiting for the cutting edge models to return a response. 73% on SWE Bench is plenty good enough for me.
  - jhancock 1y ago
    How do you review code when the LLM can produce so much so fast?
    - evan_ 1y ago
      with an LLM
    - justinbaker84 1y ago
      Just read it when it is done writing it.
aliljet 1y ago
What is the use case for these tiny models? Is it speed? Is it to move on device somewhere? Or is it to provide some relief in pricing somewhere in the API? It seems like most use is through the Claude subscription and therefore the use case here is basically non-existent.
- kasey_junk 1y ago
  They are great for building more specialized tool calls that the bigger models can call out to in agentic loops.
- minimaxir 1y ago
  If you look at the OpenRouter rankings for LLMs (generally, the models coders use for vibe/agentic coding), you can see that most of them are in the "small" model class as opposed to something like full GPT-5 or Claude Opus, albeit Gemini 2.5 Pro is higher than expected: https://openrouter.ai/rankings https://openrouter.ai/rankings
- pacoWebConsult 1y ago
  One big use-case is that claude code with sonnet 4.5 will delegate into the cheaper model (configurable) more specific, contextful tasks, and spin up 1-3 sub-agents to do so. This process saves a ton of available context window for your primary session while also increasing token throughput by fanning-out.
  - matltc 1y ago
    How does one configure Claude code to delegate to cheaper models? I have a number of agents in ~/.claude/agents/. Currently have most set to `model: sonnet` but some are on haiku. The agents are given very specific instructions and names that define what they do, like `feature-implementation-planner` and `feature-implementer`. My (naive) approach is to use higher-cost models to plan and ideally hand off to a sub-agent that uses a lower-cost model to implement, then use a higher-cost to code review. I am either not noticing the handoffs, or they are not happening unless specifically instructed. I even have a `claude-help` agent, and I asked it how to pipe/delegate tasks to subagents as you're describing, and it answered that it ought to detect it automatically. I tested it and asked it to report if any such handoffs were detected and made, and it failed on both counts, even having that initial question in its context!
85392_school 1y ago
System card: https://assets.anthropic.com/m/99128ddd009bdcb/original/Claude-Haiku-4-5-System-Card.pdf https://assets.anthropic.com/m/99128ddd009bdcb/original/Clau... (edit: discussed here https://news.ycombinator.com/item?id=45596168 https://news.ycombinator.com/item?id=45596168) This is Anthropic's first small reasoner as far as I know.
RickHull 1y ago
If I'm close to weekly limits on Claude Code with Anthropic Pro, does that go away or stretch out if I switch to Haiku?
- parkersweb 1y ago
  I’m also really interested in this - in fact it’s the first thing I went looking for in the announcement…
- visarga 1y ago
  Sonnet 4.5 was two weeks ago. In the past I never had such issues, but every week my quota ended in 2-3 days. I suspect the Sonnet 4.5 model consumes more usage points than old Sonnet 4.1 I am afraid Claude Pro subscription got 3x less usage
  - Aeolun 1y ago
    Yeah. I definitely don’t get as much usage out of Sonnet 4.5 as 5x Opus 4.1 should imply. What bothers me is that nobody told me they changed anything. It’s extremely frustrating to feel like I’m being bamboozled, but unable to confirm anything. I switched to Codex out of spite, but I still like the Claude models more…
- thomassmith65 1y ago
  How close are you? Oh right, Anthropic doesn't tell you. I got that 'close to weekly limits' message for an entire week without ever reaching it, came to the conclusion that it is just a printer industry 'low ink!' tactic, and cancelled my subscription. You don't take money from a customer for a service, and then bar the customer form using that service for multiple days. Either charge more, stop subsidizing free accounts, or decrease the daily limit.
  - __atx__ 1y ago
    These days, running `/usage` in Claude Code shows you how close you are to the session and weekly limits. Also available in the web interface settings under "Usage".
steveklabnik 1y ago
I am really interested in the future of Opus; is it going to be an absolute monster, and continue to be wildly expensive? Or is the leap from 4 -> 4.5 for it going to be more modest.
- criemen 1y ago
  Technically, they released Opus 4.1 a few weeks ago, so that alone hints at a smaller leap from 4.1 -> 4.5, compared to the leap from Sonnet 4 -> 4.5. That is, of course, if those version numbers represent anything but marketing, which I don't know.
  - steveklabnik 1y ago
    I had forgotten that, given that Sonnet pretty much blows Opus out of the water these days. Yeah, given how multi-dimensional this stuff is, I assume it's supposed to indicate broad things, closer to marketing than anything objective. Still quite useful.
  - mcintyre1994 1y ago
    Bizarrely they already call Opus 4.1 “legacy brainstorming model”.
- dheera 1y ago
  I wonder what the next smaller model after Haiku will be called. "Claude Phrase"?
  - steveklabnik 1y ago
    It's interesting to think about various aspects of marketing the models, with ChatGPT going the "internal router" direction due to address the complexity of choosing. I'd never considered something smaller than Haiku to be needed, but I also rarely used Haiku in the first place...
    - ACCount37 1y ago
seunosewa 1y ago
I'd like to see this price structure for Claude: $5/mt for Haiku 4.5 $10/mt for Sonnet 4.5 $15/mt for Opus 4.5 when it's released.
ericbrow 1y ago
Was anyone else slightly disappointed that this new product doesn't respond in Haiku, as the name would imply?
- dpoloncsak 1y ago
  Wasn't there a 3.5 haiku too? https://aws.amazon.com/about-aws/whats-new/2024/11/anthropics-claude-3-5-haiku-model-amazon-bedrock/ https://aws.amazon.com/about-aws/whats-new/2024/11/anthropic...
- esafak 1y ago
  It's not a new product; just a new version.
- simonw 1y ago
  If you want to see it generate a Haiku from your webcam I just upgraded my silly little bring-your-own-key Haiku app to use the new model: https://tools.simonwillison.net/haiku https://tools.simonwillison.net/haiku
simonw 1y ago
Pretty cute pelican on a slightly dodgy bicycle: https://tools.simonwillison.net/svg-render#%3Csvg%20viewBox%3D%220%200%20400%20400%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%0A%20%20%3C!--%20Sky%20background%20--%3E%0A%20%20%3Crect%20width%3D%22400%22%20height%3D%22400%22%20fill%3D%22%2387CEEB%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Ground%20--%3E%0A%20%20%3Crect%20y%3D%22300%22%20width%3D%22400%22%20height%3D%22100%22%20fill%3D%22%2390EE90%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Back%20wheel%20--%3E%0A%20%20%3Ccircle%20cx%3D%22100%22%20cy%3D%22300%22%20r%3D%2240%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22100%22%20cy%3D%22300%22%20r%3D%2235%22%20fill%3D%22none%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22100%22%20y1%3D%22265%22%20x2%3D%22100%22%20y2%3D%22335%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%2265%22%20y1%3D%22300%22%20x2%3D%22135%22%20y2%3D%22300%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%2275%22%20y1%3D%22275%22%20x2%3D%22125%22%20y2%3D%22325%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%2275%22%20y1%3D%22325%22%20x2%3D%22125%22%20y2%3D%22275%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Front%20wheel%20--%3E%0A%20%20%3Ccircle%20cx%3D%22280%22%20cy%3D%22300%22%20r%3D%2240%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22280%22%20cy%3D%22300%22%20r%3D%2235%22%20fill%3D%22none%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22280%22%20y1%3D%22265%22%20x2%3D%22280%22%20y2%3D%22335%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22245%22%20y1%3D%22300%22%20x2%3D%22315%22%20y2%3D%22300%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22255%22%20y1%3D%22275%22%20x2%3D%22305%22%20y2%3D%22325%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22255%22%20y1%3D%22325%22%20x2%3D%22305%22%20y2%3D%22275%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Frame%20--%3E%0A%20%20%3Cline%20x1%3D%22100%22%20y1%3D%22300%22%20x2%3D%22180%22%20y2%3D%22240%22%20stroke%3D%22%23333%22%20stroke-width%3D%224%22%2F%3E%0A%20%20%3Cline%20x1%3D%22180%22%20y1%3D%22240%22%20x2%3D%22280%22%20y2%3D%22300%22%20stroke%3D%22%23333%22%20stroke-width%3D%224%22%2F%3E%0A%20%20%3Cline%20x1%3D%22100%22%20y1%3D%22300%22%20x2%3D%22160%22%20y2%3D%22280%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Cline%20x1%3D%22160%22%20y1%3D%22280%22%20x2%3D%22280%22%20y2%3D%22300%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Handlebars%20--%3E%0A%20%20%3Cline%20x1%3D%22280%22%20y1%3D%22300%22%20x2%3D%22280%22%20y2%3D%22250%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20260%20250%20L%20280%20250%20L%20300%20250%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Seat%20--%3E%0A%20%20%3Cellipse%20cx%3D%22160%22%20cy%3D%22270%22%20rx%3D%2225%22%20ry%3D%228%22%20fill%3D%22%238B4513%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pedals%20and%20crank%20--%3E%0A%20%20%3Ccircle%20cx%3D%22160%22%20cy%3D%22280%22%20r%3D%223%22%20fill%3D%22%23333%22%2F%3E%0A%20%20%3Cline%20x1%3D%22160%22%20y1%3D%22280%22%20x2%3D%22160%22%20y2%3D%22295%22%20stroke%3D%22%23333%22%20stroke-width%3D%222%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22160%22%20cy%3D%22295%22%20r%3D%224%22%20fill%3D%22%23666%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20body%20--%3E%0A%20%20%3Cellipse%20cx%3D%22200%22%20cy%3D%22200%22%20rx%3D%2245%22%20ry%3D%2255%22%20fill%3D%22%23F5F5DC%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20neck%20--%3E%0A%20%20%3Cpath%20d%3D%22M%20210%20160%20Q%20220%20140%20225%20110%22%20fill%3D%22none%22%20stroke%3D%22%23F5F5DC%22%20stroke-width%3D%2225%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20210%20160%20Q%20220%20140%20225%20110%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20head%20--%3E%0A%20%20%3Ccircle%20cx%3D%22225%22%20cy%3D%2295%22%20r%3D%2218%22%20fill%3D%22%23F5F5DC%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20eye%20--%3E%0A%20%20%3Ccircle%20cx%3D%22232%22%20cy%3D%2292%22%20r%3D%224%22%20fill%3D%22%23000%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22233%22%20cy%3D%2291%22%20r%3D%221.5%22%20fill%3D%22%23fff%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20pouch%20(distinctive%20feature)%20--%3E%0A%20%20%3Cellipse%20cx%3D%22220%22%20cy%3D%22105%22%20rx%3D%2220%22%20ry%3D%2212%22%20fill%3D%22%23FFB6C1%22%20opacity%3D%220.7%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20205%20110%20Q%20220%20115%20235%20110%22%20fill%3D%22none%22%20stroke%3D%22%23FF69B4%22%20stroke-width%3D%221.5%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20wings%20--%3E%0A%20%20%3Cellipse%20cx%3D%22160%22%20cy%3D%22190%22%20rx%3D%2230%22%20ry%3D%2250%22%20fill%3D%22%23E6E6FA%22%20transform%3D%22rotate(-35%20160%20190)%22%2F%3E%0A%20%20%3Cellipse%20cx%3D%22240%22%20cy%3D%22190%22%20rx%3D%2230%22%20ry%3D%2250%22%20fill%3D%22%23E6E6FA%22%20transform%3D%22rotate(35%20240%20190)%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20tail%20feathers%20--%3E%0A%20%20%3Cpath%20d%3D%22M%20180%20240%20L%20160%20270%20L%20175%20265%22%20fill%3D%22%23D3D3D3%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20190%20245%20L%20175%20280%20L%20190%20273%22%20fill%3D%22%23E6E6FA%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20200%20246%20L%20200%20285%20L%20210%20275%22%20fill%3D%22%23D3D3D3%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20legs%20--%3E%0A%20%20%3Cline%20x1%3D%22190%22%20y1%3D%22250%22%20x2%3D%22185%22%20y2%3D%22290%22%20stroke%3D%22%23FF8C00%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cline%20x1%3D%22210%22%20y1%3D%22250%22%20x2%3D%22215%22%20y2%3D%22290%22%20stroke%3D%22%23FF8C00%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20feet%20on%20pedals%20--%3E%0A%20%20%3Cellipse%20cx%3D%22183%22%20cy%3D%22295%22%20rx%3D%226%22%20ry%3D%224%22%20fill%3D%22%23FF8C00%22%2F%3E%0A%20%20%3Cellipse%20cx%3D%22217%22%20cy%3D%22295%22%20rx%3D%226%22%20ry%3D%224%22%20fill%3D%22%23FF8C00%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Sun%20--%3E%0A%20%20%3Ccircle%20cx%3D%22350%22%20cy%3D%2250%22%20r%3D%2230%22%20fill%3D%22%23FFD700%22%2F%3E%0A%20%20%3Cline%20x1%3D%22350%22%20y1%3D%2210%22%20x2%3D%22350%22%20y2%3D%220%22%20stroke%3D%22%23FFD700%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cline%20x1%3D%22390%22%20y1%3D%2250%22%20x2%3D%22402%22%20y2%3D%2250%22%20stroke%3D%22%23FFD700%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cline%20x1%3D%22380%22%20y1%3D%2220%22%20x2%3D%22389%22%20y2%3D%2211%22%20stroke%3D%22%23FFD700%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%3C%2Fsvg%3E%0A https://tools.simonwillison.net/svg-render#%3Csvg%20viewBox%...
baalimago 1y ago
Ehh, expensive
leetharris 1y ago
The main thing holding these Anthropic models back is context size. Yes, quality deteriorates over a large context window, but for some applications, that is fine. My company is using grok4-fast, the Gemini family, and GPT4.1 exclusively at this point for a lot of operations just due to the huge 1m+ context.
- Tiberium 1y ago
  Is your company Tier 4? Anthropic has had 1M context size in beta for some time now. https://docs.claude.com/en/docs/build-with-claude/context-windows#1m-token-context-window https://docs.claude.com/en/docs/build-with-claude/context-wi...
  - leetharris 1y ago
    Only for Sonnet. No 1m for Haiku (this new model) and Opus. This means 2.5 Flash or Grok 4 fast takes all the low end business for large context needs.
  - _ink_ 1y ago
    Is it possible to get that in Claude Code with Pro? Or is it already a 1M context window?
Topfi 1y ago
Very preliminary testing is very promising, seems far more precise in code changes over GPT-5 models in not ingesting irrelevant to the task at hand code sections for changes which tends to make GPT-5 as a coding assistant take longer than sometimes expected. With that being the case, it is possible that in actual day-to-day use, Haiku 4.5 may be less expensive than the raw cost breakdown may appear initially, though the increase is significant. Branding is the true issue that Anthropic has though. Haiku 4.5 may (not saying it is, far to early to tell) be roughly equivalent in code output quality compared to Sonnet 4, which would serve a lot users amazingly well, but by virtue of the connotations smaller models have, alongside recent performance degradations making users more suspicious than beforehand, getting these do adopt Haiku 4.5 over Sonnet 4.5 even will be challenging. I'd love to know whether Haiku 3, 3.5 and 4.5 are roughly in the same ballpark in terms of parameters and course, nerdy old me would like that to be public information for all models, but in fairness to companies, many would just go for the largest model thinking it serves all use cases best. GPT-5 to me is still most impressive because of its pricing relative to performance and Haiku may end up similar, though with far less adoption. Everyone believes their task requires no less than Opus it seems after all. For reference: Haiku 3: I $0.25/M, O $1.25/M Haiku 4.5: I $1.00/M, O $5.00/M GPT-5: I $1.25/M, O $10.00/M GPT-5-mini: I $0.25/M, O $2.00/M GPT-5-nano: I $0.05/M, O $0.40/M GLM-4.6: I $0.60/M, O $2.20/M
- deadbabe 1y ago
  Those numbers don’t mean anything without average token usage stats.
  - distalx 1y ago
    Exactly, token per dollar rates are useful, but without knowing the typical input output token distribution for each model on this specific task, the numbers alone don’t give a full picture of cost.
    - deadbabe 1y ago
      That’s how they lie to us. Companies can advertise cheap prices to lure you in but they know very well how many tokens you’re going to use on average so they will still make more profit than ever, especially if you’re using any kind of reasoning model which is just like a blank check for them to print money.
knes 1y ago
At augmentcode.com, we've been evaluating Haiku for some time, it's actually a very good model. We found out it's 90% as good as Sonnet and is ~34% faster than sonnet! Where it doesn't shine much is on very large coding task. but it is a phenomenal model for small coding tasks and the speed improvement is much welcome
- samuelknight 1y ago
  90% as good as Sonnet 4 or 4.5? Openrouter just started reporting, and it's saying Haiku is 2x as fast (60tps vs 125tps) and 2-3x less latent (2-3s vs 1s)
- jdoe1337halo 1y ago
  Do you have a definition of what is considered a small vs large coding task?
sim04ful 1y ago
Curious they don't have any comparison to grok code fast: Haiku 4.5: I $1.00/M, O $5.00/M Grok Code: I $0.2/M, O $1.5/M
- Squarex 1y ago
  wow, grok code fast is really cheap
  - scragz 1y ago
    it writes bad code and blinding speed
    - breakingcups 1y ago
      From "The Psychology of Computer Programming": After months of effort, a particular application was still not working, so a consultant was called in from another part of the company. He concluded that the existing approach could never be made to work reliably. While on his way home he realized how it could be done. After a few days work he had a demonstration program working and presented it to the original programming team. Team leader: How long does your program take when processing? Consultant: About 10 seconds per case. Team leader: But our program only takes 1 second. {Team look smug at this point} Consultant: But your program doesn't work. If the program doesn't have to work then I can make it as fast as you like.
samuelknight 1y ago
Sonnet 4.5 is an excellent model for my startup's use case. Chatting to Haiku it looks promising too, and it may be great drop in replacement for some of inference tasks that have a lot of input tokens but don't require 4.5-level intelligence.
- extr 1y ago
  I think a lot of people judge these models purely off of what they want to personally use for coding and forget about enterprise use. For white-label chatbots that use completely custom harnesses + tools, Sonnet 4.5 is much easier to work with than GPT-5. And like you, I was really pleased to see this release today. For our usage speed/cost are more important than pure model IQ above some certain threshold. We'll likely switch over to Haiku 4.5 after some testing to confirm it is what it says on the tin.
shrisukhani 1y ago
In our (very) early testing at Hyperbrowser but we're seeing Haiku 4.5 do really well on computer use as well. Pretty cool that Haiku is like the cheapest computer use model from the big labs now.
stared 1y ago
Why I use cheaper models for summaries (a lot ogf gemini-2.5-flash), what’s the use case of cheaper AI for coding? Getting more errors, or more spaghetti code, seems never worth it.
- baq 1y ago
  If it’s fast enough it can make and correct mistakes faster, potentially getting to a solution quicker than a slower, more accurate model.
- justinbaker84 1y ago
  I feel like if I just do a better job of providing context and breaking complex tasks into a series of simple tasks then most of the models are good enough for me to code.
- svdr 1y ago
  I'm using the smaller models for things like searching and summarizing over a larger part of the codebase. The speed is really pleasant then.
zone411 1y ago
I've benchmarked it on the Extended NYT Connections (https://github.com/lechmazur/nyt-connections/ https://github.com/lechmazur/nyt-connections/). It scores 20.0 compared to 10.0 for Haiku 3.5, 19.2 for Sonnet 3.7, 26.6 for Sonnet 4.0, and 46.1 for Sonnet 4.5.
- whatreason 1y ago
  This is such a cool benchmark idea, love it Do you have any other cool benchmarks you like? Especially any related to tools
  - shangofox 1y ago
    You could try wordle on it. But from my own experience all of them are pretty bad. They're not smart enough to pick up the colours represented as letters. The only one that actually was good was Qwen surprisingly.
senko 1y ago
I've tried it on a test case for generating a simple SaaS web page (design + code). Usually I'm using GPT-5-mini for that task. Haiku 4.5 runs 3x faster with roughly comparable results (I slightly prefer the GPT-5-mini output but may have just accustomed to it).
- justinbaker84 1y ago
  I don't understand why more people don't talk about how fast the models are. I see so much obsession with bechmark scores but speed of response is very important for day to day use. I agree that the models from OpenAI and Google have much slower responses than the models from Anthropic. That makes a lot of them not practical for me.
  - brianwawok 1y ago
    If the prompt runs twice as fast but it takes an extra correction, it’s a worse output. I’d take 5 minute responses that are final.
  - gizmodo59 1y ago
    I don’t agree that speed by itself is a big factor. It may target a certain audience but I don’t mind waiting for a correct output rather than too many turns with a faster model.
  - jstummbillig 1y ago
    Well, it depends on what you do. If a model can produce a PR that is ready to merge (and another can't), waiting 5 minutes is fine.
ilaksh 1y ago
What LLM do you guys use for fast inference for voice/phone agents? I feel like to get really good latency I need to "cheat" with Cerebras, groq or SambaNova. Haiku 4.5 is very good but still seems to be adding a second of latency.
ashirviskas 1y ago
And I was wondering today why Sonnet 4.5 seemed so freaking slow. Now this explains it, Sonnet 4.5 is the new Opus 4.1 where Anthropic does not really want you to use it.
justinbaker84 1y ago
I am very excited about this. I am a freelance developer and getting responses 3x faster is totally worth the slightly reduced capability. I expect I will be a lot more productive using this instead of claude 4.5 which has been my daily driver LLM since it came out.
philipp-gayret 1y ago
Tried it in Claude Code via /config, makes it feel like I'm running on Cerebras. It's seriously fast, bottleneck is on human review at this point.
- singularity2001 1y ago
  Do you need Pro?
  - beklein 1y ago
    You can use the model flag and specify the model like: claude --model claude-haiku-4-5-20251001
  - philipp-gayret 1y ago
    All I know is I'm on the Claude Code 5x max plan and it works on my machine.
  - gitaarik 1y ago
    For Claude Code you need a paid subscription anyway
simonw 1y ago
I went looking for the bit about if it blackmails you or tries to murder you... and it was a bit of a cop-out! > Previous system cards have reported results on an expanded version of our earlier agentic misalignment evaluation suite: three families of exotic scenarios meant to elicit the model to commit blackmail, attempt a murder, and frame someone for financial crimes. We choose not to report full results here because, similarly to Claude Sonnet 4.5, Claude Haiku 4.5 showed many clear examples of verbalized evaluation awareness on all three of the scenarios tested in this suite. Since the suite only consisted of many similar variants of three core scenarios, we expect that the model maintained high unverbalized awareness across the board, and we do not trust it to be representative of behavior in the real extreme situations the suite is meant to emulate. https://www.anthropic.com/research/agentic-misalignment https://www.anthropic.com/research/agentic-misalignment
- username223 1y ago
  It sounds like AI researchers have used too much of their own bad sci-fi as training data for models they don't understand. Goodhart's law wins again!
dotancohen 1y ago
> In the system card, we focus on safety evaluations, including assessments of: ... the model’s own potential welfare ... In what way does a language model need to have its own welfare protected? Does this generation of models have persistent "feelings"?
- neuronexmachina 1y ago
  They previously discussed this some in the context of Opus 4: https://www.anthropic.com/research/end-subset-conversations https://www.anthropic.com/research/end-subset-conversations > We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention. In pre-deployment testing of Claude Opus 4, we included a preliminary model welfare assessment. As part of that assessment, we investigated Claude’s self-reported and behavioral preferences, and found a robust and consistent aversion to harm. This included, for example, requests from users for sexual content involving minors and attempts to solicit information that would enable large-scale violence or acts of terror. Claude Opus 4 showed: * A strong preference against engaging with harmful tasks; * A pattern of apparent distress when engaging with real-world users seeking harmful content; and * A tendency to end harmful conversations when given the ability to do so in simulated user interactions. These behaviors primarily arose in cases where users persisted with harmful requests and/or abuse despite Claude repeatedly refusing to comply and attempting to productively redirect the interactions.
  - stuffoverflow 1y ago
    I can't tell if anthropic is serious about "model welfare" or if it's just a marketing ploy. I mean isn't it responding negatively because it has been trained that way? If they were serious, wouldn't the ethical thing be to train the model to respond neutrally to "harmful" queries?
    - 1y ago
KaiserPro 1y ago
Ok, I use claude, mostly on default, but with extended thinking and per project prompts. What's the advantage of using haiku for me? is it just faster?
singularity2001 1y ago
claude --model Haiku-4.5 doesn't work
- mi_lk 1y ago
  check the model name here: https://docs.claude.com/en/docs/about-claude/models/overview#model-names https://docs.claude.com/en/docs/about-claude/models/overview...
- Void_ 1y ago
  use claude-haiku-4-5-20251001
- beklein 1y ago
  claude --model claude-haiku-4-5-20251001
logankeenan 1y ago
I'm not seeing it as a model option in Claude Code for my Pro plan. Perhaps, it'll roll out eventually? Anyone else seeing it with the same plan?
- matltc 1y ago
  You on latest version? Try running /update hook. Can also config autoupdates
  - logankeenan 1y ago
    I'm on the binary install with version v2.0.19. It never showed in the `/model` selector UI. I did end up typing `/model haiku` and now it shows as a custom model in the `/model` selector. It shows claude-haiku-4-5-20251001 when selected.
- sumedh 1y ago
  Same here but you can get it using claude --model claude-haiku-4-5-20251001
cadamsdotcom 1y ago
Claude Code is great but slow to work with. Excited to see how fast Haiku can go!
layer8 1y ago
For those wondering where the “card” terminology originated: https://arxiv.org/pdf/1810.03993 https://arxiv.org/pdf/1810.03993 Maybe at 39 pages we should start looking for a different term…
qustrolabe 1y ago
Awww they took away free tier Sonnet 4.5, that was a beautiful model to talk to even outside coding stuff
quentin-smr 1y ago
Comparing haiku and sonnet for a question needing a code doc fetch: haiku https://claude.ai/share/8a5c70d5-1be1-40ca-a740-9cf35b1110b1 https://claude.ai/share/8a5c70d5-1be1-40ca-a740-9cf35b1110b1 sonnet https://claude.ai/share/51b72d39-c485-44aa-a0eb-30b4cc6d6b7b https://claude.ai/share/51b72d39-c485-44aa-a0eb-30b4cc6d6b7b haiku invented the output of a function and gave a bad answer. sonnet got it right
caymanjim 1y ago
Ain't nobody got time to pick models and compare features. It's annoying enough having to switch from one LLM ecosystem to another all the time due to vague usage restrictions. I'm paying $20/mo to Anthropic for Claude Code, to OpenAI for Codex, and previously to Cursor for...I don't even know what. I know Cursor lets you select a few different models under the covers, but I have no idea how they differ, nor do I care. I just want consistent tooling and I don't want to have to think about what's going on behind the scenes. Make it better. Make it better without me having to do research and pick and figure out what today's latest fashion is. Make it integrate in a generic way, like TLS servers, so that it doesn't matter whether I'm using a CLI or neovim or an IDE, and so that I don't have to constantly switch tooling.
- UncleOxidant 1y ago
  > annoying enough having to switch from one LLM ecosystem to another all the time due to vague usage restrictions I use KiloCode and what I find amazing is that it'll be working on a problem and then a message will come up about needing to topup the money in my account to continue (or switch to a free model), so I switch to a free model (currently their Code Supernova 1million context) and it doesn't miss a beat and continues working on the problem. I don't know how they do this. It went from using a Claude Sonnet model to this Code Supernova model without missing a beat. Not sure if this is a Kilocode thing or if others do this as well. How does that even work? And this wasn't a trivial problem, it was adding a microcode debugger to a microcoded state machine system (coding in C++).
  - qsort 1y ago
    Models are stateless, why would that not work?
    - UncleOxidant 1y ago
      OK I understand what those words mean, but how exactly does that work? How does the new model 'know' what's being worked on when the old model was in the middle of working on a task and then a new model is switched to? (and where the task might be modifying a C++ file)
meander_water 1y ago
> The score reported uses a minor prompt addition: "You should use tools as much as possible, ideally more than 100 times. You should also implement your own tests first before attempting the problem." I'm not sure if the SWE benchmark score can be compared like for like with OpenAIs scores because of this.
- joshuahedlund 1y ago
  https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law "When a measure becomes a target, it ceases to be a good measure" I'm also curious what results we would get if SWE came up with a new set of 500 problems to run all these models against, to guard against overfitting.
andrewstuart 1y ago
Claude has stopped showing code in artifacts unless it knows the extension. I used to be able to work on Arduino .ino files in Claude now it just says it can’t show it to me. And do we have zip file uploads yet to Claude? ChatGPT and Gemini have done this for ages. And all the while Claude’s usage limits keep going up. So yeah, less for more with Claude.
taf2 1y ago
I just don't find the benchmarks on the site here at all believable. codex for me with gpt-5 is so much better then claude any model version. I mean maybe it's because they compare to gpt-5-codex model but they don't mention is that high, medium, low, etc... so it's just misleading probably... but i must reiterate zero loyalty to any AI vendor. 100% what solves the problem more consistently and of a higher quality and currently gpt-5 high - hands down
- lysecret 1y ago
  It’s so funny to me but ever since they fixed that claud bug my experience has consistently been the exact opposite. Only thing I use codex now for are quite standard things it can solve end to end (like adding new features to my crud app) anything non standard iterating with Claude yields much better results.
- scottyah 1y ago
  Out of curiosity, what kind of work do you use them for? I did a comparison of a few different models for setting up a home server with k3s and a few web apps in nextjs. Claude was my favorite for both tasks, but mainly because it seemed to take my feedback a lot better than others.
  - taf2 1y ago
    Working on a series of applications, from rails, nodejs interfacing with twilio's api's to salesforce api's - also C/C++ libraries. The example that stood out the best to me was implementing fiber support in the http library i maintain for ruby. Initially i prompted claude to implement fiber support. It started to write a few thousand lines of code for what codex rightly identified was to use a different IO selector library. The problem did end up being a bit more complex but gpt-5 overall in my experience has better knowledge of what is really available vs claude which not only ends up writing way more code to solve a problem that requires just knowing the libraries better... all of this is super subjective but my main reason for saying - "I have no loyalty to any AI, just the solution"
hu3 1y ago
glad to see it's already available in VScode Copilot for me.
no_flaks_given 1y ago
What I want to see is an Anthropic + Cerebras partnership. Haiku becomes a fucking killer at 2000token/second. Charge me double idgaf
getpokedagain 1y ago
I have had great experience using the previous haiku with mcp servers. I am looking forward to trying this out.
coreylane 1y ago
What do Claude Code users do for tab auto complete, if anything? GitHub Copilot Free tier?
- throwaway314155 1y ago
  It's more of a conversational TUI than it is a code editor. You tell Claude to code for you. No need for autocomplete in that regime.
  - mpalmer 1y ago
    I think they still see it as a key part of their workflow, and they're asking whether they'll have to pay for more than one service
- roryirvine 1y ago
  Yeah, that's enough for my usage. GitHub typically reports that I'm using 25-30% of the free tier, and 100% of that will be from code completions in my editor. I do maybe 3 hours solid coding a day on average. I also pay for Gemini Pro for non-coding research. I did have it hooked up to my VSCode a few months ago, but it got reset back to GH Copilot at some point and I've not found a reason to fix it.
TurboSkyline 1y ago
The Claude models seem to be focused on generating code more than anything else. Are they still competitive with Open AI and Google for more general use cases, or have they sacrificed that?
- deaux 1y ago
  Which of the 1,000 other use cases? :) To give one example, Opus and Sonnet IMO remain the #1 and #2 for writing informative prose. They're not entirely free of slop, but the ratio is lower than Gemini and especially GPT.
kqr 1y ago
This is surprisingly competent. A couple of months ago I evaluated some leading models on a bunch of text adventures[1]. Typical regression coefficients would be +0.02 for top level models like Sonnet and Gemini 2.5 Pro, but notably also Gemini 2.5 Flash. (The baseline is GPT 5 Chat, i.e. the one where OpenAI routes to a thinking model only when they determine it's needed.) When I include an attempt from Haiku 4.5 in the mix, most coefficients stay similar, but Haiku itself gets a +0.05. This must be a statistical fluke, because that would be insanely impressive – in particular for a cheaper model. I guess I'm adding samples to some of these after all... [1]: https://entropicthoughts.com/evaluating-llms-playing-text-adventures https://entropicthoughts.com/evaluating-llms-playing-text-ad... Edit: It was a fluke. Back to +0.01 after one more go at all games.
Ecko123 1y ago
[dead]
fwystup 1y ago
Does anybody also see this with claude code and haiku 4.5 (tried to set the env var, no change): "API Error: Claude's response exceeded the 8192 output token maximum. To configure this behavior, set the CLAUDE_CODE_MAX_OUTPUT_TOKENS environment variable."?
nisten 1y ago
it's good
qwertox 1y ago
This is my default model now. All other are token consumption monsters, leading to 6% usage of the 5-hour-quota with one single submission. I had cancelled my Claude.ai subscription 4 days ago, but this model will likely make me revert this action.
AlwaysRock 1y ago
Worked with it a bit last night! Seems quick. I did run into the same problem I have with Gemini often where the response says something like, "I need to do x" or "I did x" and then nothing actually happens. Agent seems to think it actually does finish the task but it stops part way. But I'm sure they will sort that out, as I dont have that issue with other anthropic models.
- solarkraft 1y ago
  This is interesting. I’ve had this same issue trying to build an agentic system with the smaller ChatGPT models, almost no matter the prompt (“think aloud” are magic words that help a lot, but it’s still flaky). Most of the time it would either perform the tool call before explaining it (the default) or explain it but then not actually make the call. I’ve been wondering how Cursor et al solved this problem (having the LLM explain what it will do before doing it is vitally important IMO), but maybe it’s just not a problem with the big models. Your experience seems to support that smaller models are just generally worse about tool calling (were you using Gemini Flash?) when asked to reason first.
Razengan 11mo ago
I just asked Claude how to implement something in iOS 26. It told me iOS 26 isn't out yet. Sigh..
bowserman 11mo ago
haiku 4.5 is not charging me any tokens at all when running from the cli :)

Claude Haiku 4.5

730 points by adocomplete 1y ago

System card: https://assets.anthropic.com/m/99128ddd009bdcb/original/Claude-Haiku-4-5-System-Card.pdf https://assets.anthropic.com/m/99128ddd009bdcb/original/Clau...

minimaxir 1y ago

$1/M input tokens and $5/M output tokens is good compared to Claude Sonnet 4.5 but nowadays thanks to the pace of the industry developing smaller/faster LLMs for agentic coding, you can get comparable models priced for much lower which matters at the scale needed for agentic coding. Given that Sonnet is still a popular model for coding despite the much higher cost, I expect Haiku will get traction if the quality is as good as this post claims.

Bolwin 1y ago
With caching that's 10 cents per million in. Most of the cheap open source models (which this claims to beat, except glm 4.6) have limited and not as effective caching. This could be massive.
- logicchains 1y ago
  $1/M is hardly a big improvement over GPT5's $1.250/M (or Gemini Pro's $1.5/M), and given how much worse Haiku is than those at any kind of difficult problem (or problems with a large context size), I can't imagine it being a particularly competitive alternative for coding. Especially for anything math/logic related, I find GPT5 and Gemini Pro to be significantly better even than Opus (which reflects in their models having won Olympiad prizes while Anthropic's have not).
  - HarHarVeryFunny 1y ago
    GPT-5 is $10/M for output tokens, twice the cost of Haiku 4.5 at $5/M, despite Haiku apparently being better at some tasks (SWE Bench). I suppose it depends on how you are using it, but for coding isn't output cost more relevant than input - requirements in, code out ?
    - criemen 1y ago
      > I suppose it depends on how you are using it, but for coding isn't output cost more relevant than input - requirements in, code out ? Depends on what you're doing, but for modifying an existing project (rather than greenfield), input tokens >> output tokens in my experience.
    - logicchains 1y ago
      Unless you're working on a small greenfield project, you'll usually have 10s-100s of thousands of relevant words (~tokens) of relevant code in context for every query, vs a few hundred words of changes being output per query. Because most changes to an existing project are relatively small in scope.
- Tiberium 1y ago
  The funny thing is that even in this area Anthropic is behind other 3 labs (Google, OpenAI, xAI). It's the only one out of those 4 that requires you to manually set cache breakpoints, and the initial cache costs 25% more than usual context. The other 3 have fully free implicit caching. Although Google also offers paid, explicit caching. https://docs.claude.com/en/docs/build-with-claude/prompt-caching https://docs.claude.com/en/docs/build-with-claude/prompt-cac... https://ai.google.dev/gemini-api/docs/caching https://ai.google.dev/gemini-api/docs/caching https://platform.openai.com/docs/guides/prompt-caching https://platform.openai.com/docs/guides/prompt-caching https://docs.x.ai/docs/models#cached-prompt-tokens https://docs.x.ai/docs/models#cached-prompt-tokens
  - tempusalaria 1y ago
    I vastly prefer the manual caching. There are several aspects of automatic caching that are suboptimal, with only moderately less developer burden. I don’t use Anthropic much but I wish the others had manual cache options
    - simonw 1y ago
      What's sub-optimal about the OpenAI approach, where you get 90% discount on tokens that you've previously sent within X minutes?
      - stavros 1y ago
        Is it wherever the tokens are, or is it the N first tokens they've seen before? Ie if my prompt is 99% the same, except for the first token, will it be cached?
        simonw 1y ago
        The prefix has to be stable. If you are 99% the same but the first token is different it won't cache at all. You end up having to design your prompts to accommodate this.
simonw 1y ago
Yeah, I'm a bit disappointed by the price. Claude 3.5 Haiku was $0.8/$4, 4.5 Haiku is $1/$5. I was hoping Anthropic would introduce something price-competitive with the cheaper models from OpenAI and Gemini, which get as low as $0.05/$0.40 (GPT-5-Nano) and $0.075/$0.30 (Gemini 2.0 Flash Lite).
- odie5533 1y ago
  There's probably less margin on the low end, so they don't want to focus on capturing it.
  - dr_dshiv 1y ago
    Margin? Hahahahaha
    - odie5533 1y ago
      Inference is profitable.
      - reppap 1y ago
        If you completely ignore inference revenue needing to offset training costs. Is inference still profitable if you account for the amortized training cost?
        yunwal 1y ago
        > If you completely ignore inference revenue needing to offset training costs. This is what people mean when they say margin. When you buy a pair of shoes, the margin is price/(materials+labor), and doesn’t include the price of the factory or the store they were bought in
        simonw 1y ago
        Not for the big labs, who are engaged in an astonishingly competitive buildout right now. There are a bunch of companies who offer inference against open weight models trained by other people. They get to skip the training costs.
rudedogg 1y ago
This also means API usage through Claude Code got more expensive (but better if benchmarks are to be believed)
justinbaker84 1y ago
I am a professional developer so I don't care about the costs. I would be willing to pay more for 4.5 Haiku vs 4.5 Sonnet because the speed is so valuable. I spend way to much time waiting for the cutting edge models to return a response. 73% on SWE Bench is plenty good enough for me.
- jhancock 1y ago
  How do you review code when the LLM can produce so much so fast?
  - evan_ 1y ago
    with an LLM
  - justinbaker84 1y ago
    Just read it when it is done writing it.

aliljet 1y ago

What is the use case for these tiny models? Is it speed? Is it to move on device somewhere? Or is it to provide some relief in pricing somewhere in the API? It seems like most use is through the Claude subscription and therefore the use case here is basically non-existent.

kasey_junk 1y ago
They are great for building more specialized tool calls that the bigger models can call out to in agentic loops.
minimaxir 1y ago
If you look at the OpenRouter rankings for LLMs (generally, the models coders use for vibe/agentic coding), you can see that most of them are in the "small" model class as opposed to something like full GPT-5 or Claude Opus, albeit Gemini 2.5 Pro is higher than expected: https://openrouter.ai/rankings https://openrouter.ai/rankings
pacoWebConsult 1y ago
One big use-case is that claude code with sonnet 4.5 will delegate into the cheaper model (configurable) more specific, contextful tasks, and spin up 1-3 sub-agents to do so. This process saves a ton of available context window for your primary session while also increasing token throughput by fanning-out.
- matltc 1y ago
  How does one configure Claude code to delegate to cheaper models? I have a number of agents in ~/.claude/agents/. Currently have most set to `model: sonnet` but some are on haiku. The agents are given very specific instructions and names that define what they do, like `feature-implementation-planner` and `feature-implementer`. My (naive) approach is to use higher-cost models to plan and ideally hand off to a sub-agent that uses a lower-cost model to implement, then use a higher-cost to code review. I am either not noticing the handoffs, or they are not happening unless specifically instructed. I even have a `claude-help` agent, and I asked it how to pipe/delegate tasks to subagents as you're describing, and it answered that it ought to detect it automatically. I tested it and asked it to report if any such handoffs were detected and made, and it failed on both counts, even having that initial question in its context!

85392_school 1y ago

System card: https://assets.anthropic.com/m/99128ddd009bdcb/original/Claude-Haiku-4-5-System-Card.pdf https://assets.anthropic.com/m/99128ddd009bdcb/original/Clau... (edit: discussed here https://news.ycombinator.com/item?id=45596168 https://news.ycombinator.com/item?id=45596168) This is Anthropic's first small reasoner as far as I know.

RickHull 1y ago

If I'm close to weekly limits on Claude Code with Anthropic Pro, does that go away or stretch out if I switch to Haiku?

parkersweb 1y ago
I’m also really interested in this - in fact it’s the first thing I went looking for in the announcement…
visarga 1y ago
Sonnet 4.5 was two weeks ago. In the past I never had such issues, but every week my quota ended in 2-3 days. I suspect the Sonnet 4.5 model consumes more usage points than old Sonnet 4.1 I am afraid Claude Pro subscription got 3x less usage
- Aeolun 1y ago
  Yeah. I definitely don’t get as much usage out of Sonnet 4.5 as 5x Opus 4.1 should imply. What bothers me is that nobody told me they changed anything. It’s extremely frustrating to feel like I’m being bamboozled, but unable to confirm anything. I switched to Codex out of spite, but I still like the Claude models more…
thomassmith65 1y ago
How close are you? Oh right, Anthropic doesn't tell you. I got that 'close to weekly limits' message for an entire week without ever reaching it, came to the conclusion that it is just a printer industry 'low ink!' tactic, and cancelled my subscription. You don't take money from a customer for a service, and then bar the customer form using that service for multiple days. Either charge more, stop subsidizing free accounts, or decrease the daily limit.
- __atx__ 1y ago
  These days, running `/usage` in Claude Code shows you how close you are to the session and weekly limits. Also available in the web interface settings under "Usage".

steveklabnik 1y ago

I am really interested in the future of Opus; is it going to be an absolute monster, and continue to be wildly expensive? Or is the leap from 4 -> 4.5 for it going to be more modest.

criemen 1y ago
Technically, they released Opus 4.1 a few weeks ago, so that alone hints at a smaller leap from 4.1 -> 4.5, compared to the leap from Sonnet 4 -> 4.5. That is, of course, if those version numbers represent anything but marketing, which I don't know.
- steveklabnik 1y ago
  I had forgotten that, given that Sonnet pretty much blows Opus out of the water these days. Yeah, given how multi-dimensional this stuff is, I assume it's supposed to indicate broad things, closer to marketing than anything objective. Still quite useful.
- mcintyre1994 1y ago
  Bizarrely they already call Opus 4.1 “legacy brainstorming model”.
dheera 1y ago
I wonder what the next smaller model after Haiku will be called. "Claude Phrase"?
- steveklabnik 1y ago
  It's interesting to think about various aspects of marketing the models, with ChatGPT going the "internal router" direction due to address the complexity of choosing. I'd never considered something smaller than Haiku to be needed, but I also rarely used Haiku in the first place...
  - ACCount37 1y ago

seunosewa 1y ago

I'd like to see this price structure for Claude: $5/mt for Haiku 4.5 $10/mt for Sonnet 4.5 $15/mt for Opus 4.5 when it's released.

ericbrow 1y ago

Was anyone else slightly disappointed that this new product doesn't respond in Haiku, as the name would imply?

dpoloncsak 1y ago
Wasn't there a 3.5 haiku too? https://aws.amazon.com/about-aws/whats-new/2024/11/anthropics-claude-3-5-haiku-model-amazon-bedrock/ https://aws.amazon.com/about-aws/whats-new/2024/11/anthropic...
esafak 1y ago
It's not a new product; just a new version.
simonw 1y ago
If you want to see it generate a Haiku from your webcam I just upgraded my silly little bring-your-own-key Haiku app to use the new model: https://tools.simonwillison.net/haiku https://tools.simonwillison.net/haiku

simonw 1y ago

Pretty cute pelican on a slightly dodgy bicycle: https://tools.simonwillison.net/svg-render#%3Csvg%20viewBox%3D%220%200%20400%20400%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%3E%0A%20%20%3C!--%20Sky%20background%20--%3E%0A%20%20%3Crect%20width%3D%22400%22%20height%3D%22400%22%20fill%3D%22%2387CEEB%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Ground%20--%3E%0A%20%20%3Crect%20y%3D%22300%22%20width%3D%22400%22%20height%3D%22100%22%20fill%3D%22%2390EE90%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Back%20wheel%20--%3E%0A%20%20%3Ccircle%20cx%3D%22100%22%20cy%3D%22300%22%20r%3D%2240%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22100%22%20cy%3D%22300%22%20r%3D%2235%22%20fill%3D%22none%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22100%22%20y1%3D%22265%22%20x2%3D%22100%22%20y2%3D%22335%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%2265%22%20y1%3D%22300%22%20x2%3D%22135%22%20y2%3D%22300%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%2275%22%20y1%3D%22275%22%20x2%3D%22125%22%20y2%3D%22325%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%2275%22%20y1%3D%22325%22%20x2%3D%22125%22%20y2%3D%22275%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Front%20wheel%20--%3E%0A%20%20%3Ccircle%20cx%3D%22280%22%20cy%3D%22300%22%20r%3D%2240%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22280%22%20cy%3D%22300%22%20r%3D%2235%22%20fill%3D%22none%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22280%22%20y1%3D%22265%22%20x2%3D%22280%22%20y2%3D%22335%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22245%22%20y1%3D%22300%22%20x2%3D%22315%22%20y2%3D%22300%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22255%22%20y1%3D%22275%22%20x2%3D%22305%22%20y2%3D%22325%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%3Cline%20x1%3D%22255%22%20y1%3D%22325%22%20x2%3D%22305%22%20y2%3D%22275%22%20stroke%3D%22%23666%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Frame%20--%3E%0A%20%20%3Cline%20x1%3D%22100%22%20y1%3D%22300%22%20x2%3D%22180%22%20y2%3D%22240%22%20stroke%3D%22%23333%22%20stroke-width%3D%224%22%2F%3E%0A%20%20%3Cline%20x1%3D%22180%22%20y1%3D%22240%22%20x2%3D%22280%22%20y2%3D%22300%22%20stroke%3D%22%23333%22%20stroke-width%3D%224%22%2F%3E%0A%20%20%3Cline%20x1%3D%22100%22%20y1%3D%22300%22%20x2%3D%22160%22%20y2%3D%22280%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Cline%20x1%3D%22160%22%20y1%3D%22280%22%20x2%3D%22280%22%20y2%3D%22300%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Handlebars%20--%3E%0A%20%20%3Cline%20x1%3D%22280%22%20y1%3D%22300%22%20x2%3D%22280%22%20y2%3D%22250%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20260%20250%20L%20280%20250%20L%20300%20250%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Seat%20--%3E%0A%20%20%3Cellipse%20cx%3D%22160%22%20cy%3D%22270%22%20rx%3D%2225%22%20ry%3D%228%22%20fill%3D%22%238B4513%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pedals%20and%20crank%20--%3E%0A%20%20%3Ccircle%20cx%3D%22160%22%20cy%3D%22280%22%20r%3D%223%22%20fill%3D%22%23333%22%2F%3E%0A%20%20%3Cline%20x1%3D%22160%22%20y1%3D%22280%22%20x2%3D%22160%22%20y2%3D%22295%22%20stroke%3D%22%23333%22%20stroke-width%3D%222%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22160%22%20cy%3D%22295%22%20r%3D%224%22%20fill%3D%22%23666%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20body%20--%3E%0A%20%20%3Cellipse%20cx%3D%22200%22%20cy%3D%22200%22%20rx%3D%2245%22%20ry%3D%2255%22%20fill%3D%22%23F5F5DC%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20neck%20--%3E%0A%20%20%3Cpath%20d%3D%22M%20210%20160%20Q%20220%20140%20225%20110%22%20fill%3D%22none%22%20stroke%3D%22%23F5F5DC%22%20stroke-width%3D%2225%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20210%20160%20Q%20220%20140%20225%20110%22%20fill%3D%22none%22%20stroke%3D%22%23333%22%20stroke-width%3D%221%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20head%20--%3E%0A%20%20%3Ccircle%20cx%3D%22225%22%20cy%3D%2295%22%20r%3D%2218%22%20fill%3D%22%23F5F5DC%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20eye%20--%3E%0A%20%20%3Ccircle%20cx%3D%22232%22%20cy%3D%2292%22%20r%3D%224%22%20fill%3D%22%23000%22%2F%3E%0A%20%20%3Ccircle%20cx%3D%22233%22%20cy%3D%2291%22%20r%3D%221.5%22%20fill%3D%22%23fff%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20pouch%20(distinctive%20feature)%20--%3E%0A%20%20%3Cellipse%20cx%3D%22220%22%20cy%3D%22105%22%20rx%3D%2220%22%20ry%3D%2212%22%20fill%3D%22%23FFB6C1%22%20opacity%3D%220.7%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20205%20110%20Q%20220%20115%20235%20110%22%20fill%3D%22none%22%20stroke%3D%22%23FF69B4%22%20stroke-width%3D%221.5%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20wings%20--%3E%0A%20%20%3Cellipse%20cx%3D%22160%22%20cy%3D%22190%22%20rx%3D%2230%22%20ry%3D%2250%22%20fill%3D%22%23E6E6FA%22%20transform%3D%22rotate(-35%20160%20190)%22%2F%3E%0A%20%20%3Cellipse%20cx%3D%22240%22%20cy%3D%22190%22%20rx%3D%2230%22%20ry%3D%2250%22%20fill%3D%22%23E6E6FA%22%20transform%3D%22rotate(35%20240%20190)%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20tail%20feathers%20--%3E%0A%20%20%3Cpath%20d%3D%22M%20180%20240%20L%20160%20270%20L%20175%20265%22%20fill%3D%22%23D3D3D3%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20190%20245%20L%20175%20280%20L%20190%20273%22%20fill%3D%22%23E6E6FA%22%2F%3E%0A%20%20%3Cpath%20d%3D%22M%20200%20246%20L%20200%20285%20L%20210%20275%22%20fill%3D%22%23D3D3D3%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20legs%20--%3E%0A%20%20%3Cline%20x1%3D%22190%22%20y1%3D%22250%22%20x2%3D%22185%22%20y2%3D%22290%22%20stroke%3D%22%23FF8C00%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cline%20x1%3D%22210%22%20y1%3D%22250%22%20x2%3D%22215%22%20y2%3D%22290%22%20stroke%3D%22%23FF8C00%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Pelican%20feet%20on%20pedals%20--%3E%0A%20%20%3Cellipse%20cx%3D%22183%22%20cy%3D%22295%22%20rx%3D%226%22%20ry%3D%224%22%20fill%3D%22%23FF8C00%22%2F%3E%0A%20%20%3Cellipse%20cx%3D%22217%22%20cy%3D%22295%22%20rx%3D%226%22%20ry%3D%224%22%20fill%3D%22%23FF8C00%22%2F%3E%0A%20%20%0A%20%20%3C!--%20Sun%20--%3E%0A%20%20%3Ccircle%20cx%3D%22350%22%20cy%3D%2250%22%20r%3D%2230%22%20fill%3D%22%23FFD700%22%2F%3E%0A%20%20%3Cline%20x1%3D%22350%22%20y1%3D%2210%22%20x2%3D%22350%22%20y2%3D%220%22%20stroke%3D%22%23FFD700%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cline%20x1%3D%22390%22%20y1%3D%2250%22%20x2%3D%22402%22%20y2%3D%2250%22%20stroke%3D%22%23FFD700%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%20%20%3Cline%20x1%3D%22380%22%20y1%3D%2220%22%20x2%3D%22389%22%20y2%3D%2211%22%20stroke%3D%22%23FFD700%22%20stroke-width%3D%223%22%20stroke-linecap%3D%22round%22%2F%3E%0A%3C%2Fsvg%3E%0A https://tools.simonwillison.net/svg-render#%3Csvg%20viewBox%...

baalimago 1y ago

Ehh, expensive

leetharris 1y ago

The main thing holding these Anthropic models back is context size. Yes, quality deteriorates over a large context window, but for some applications, that is fine. My company is using grok4-fast, the Gemini family, and GPT4.1 exclusively at this point for a lot of operations just due to the huge 1m+ context.

Tiberium 1y ago
Is your company Tier 4? Anthropic has had 1M context size in beta for some time now. https://docs.claude.com/en/docs/build-with-claude/context-windows#1m-token-context-window https://docs.claude.com/en/docs/build-with-claude/context-wi...
- leetharris 1y ago
  Only for Sonnet. No 1m for Haiku (this new model) and Opus. This means 2.5 Flash or Grok 4 fast takes all the low end business for large context needs.
- _ink_ 1y ago
  Is it possible to get that in Claude Code with Pro? Or is it already a 1M context window?

Topfi 1y ago

Very preliminary testing is very promising, seems far more precise in code changes over GPT-5 models in not ingesting irrelevant to the task at hand code sections for changes which tends to make GPT-5 as a coding assistant take longer than sometimes expected. With that being the case, it is possible that in actual day-to-day use, Haiku 4.5 may be less expensive than the raw cost breakdown may appear initially, though the increase is significant. Branding is the true issue that Anthropic has though. Haiku 4.5 may (not saying it is, far to early to tell) be roughly equivalent in code output quality compared to Sonnet 4, which would serve a lot users amazingly well, but by virtue of the connotations smaller models have, alongside recent performance degradations making users more suspicious than beforehand, getting these do adopt Haiku 4.5 over Sonnet 4.5 even will be challenging. I'd love to know whether Haiku 3, 3.5 and 4.5 are roughly in the same ballpark in terms of parameters and course, nerdy old me would like that to be public information for all models, but in fairness to companies, many would just go for the largest model thinking it serves all use cases best. GPT-5 to me is still most impressive because of its pricing relative to performance and Haiku may end up similar, though with far less adoption. Everyone believes their task requires no less than Opus it seems after all. For reference: Haiku 3: I $0.25/M, O $1.25/M Haiku 4.5: I $1.00/M, O $5.00/M GPT-5: I $1.25/M, O $10.00/M GPT-5-mini: I $0.25/M, O $2.00/M GPT-5-nano: I $0.05/M, O $0.40/M GLM-4.6: I $0.60/M, O $2.20/M

deadbabe 1y ago
Those numbers don’t mean anything without average token usage stats.
- distalx 1y ago
  Exactly, token per dollar rates are useful, but without knowing the typical input output token distribution for each model on this specific task, the numbers alone don’t give a full picture of cost.
  - deadbabe 1y ago
    That’s how they lie to us. Companies can advertise cheap prices to lure you in but they know very well how many tokens you’re going to use on average so they will still make more profit than ever, especially if you’re using any kind of reasoning model which is just like a blank check for them to print money.

knes 1y ago

At augmentcode.com, we've been evaluating Haiku for some time, it's actually a very good model. We found out it's 90% as good as Sonnet and is ~34% faster than sonnet! Where it doesn't shine much is on very large coding task. but it is a phenomenal model for small coding tasks and the speed improvement is much welcome

samuelknight 1y ago
90% as good as Sonnet 4 or 4.5? Openrouter just started reporting, and it's saying Haiku is 2x as fast (60tps vs 125tps) and 2-3x less latent (2-3s vs 1s)
jdoe1337halo 1y ago
Do you have a definition of what is considered a small vs large coding task?

sim04ful 1y ago

Curious they don't have any comparison to grok code fast: Haiku 4.5: I $1.00/M, O $5.00/M Grok Code: I $0.2/M, O $1.5/M

Squarex 1y ago
wow, grok code fast is really cheap
- scragz 1y ago
  it writes bad code and blinding speed
  - breakingcups 1y ago
    From "The Psychology of Computer Programming": After months of effort, a particular application was still not working, so a consultant was called in from another part of the company. He concluded that the existing approach could never be made to work reliably. While on his way home he realized how it could be done. After a few days work he had a demonstration program working and presented it to the original programming team. Team leader: How long does your program take when processing? Consultant: About 10 seconds per case. Team leader: But our program only takes 1 second. {Team look smug at this point} Consultant: But your program doesn't work. If the program doesn't have to work then I can make it as fast as you like.

samuelknight 1y ago

Sonnet 4.5 is an excellent model for my startup's use case. Chatting to Haiku it looks promising too, and it may be great drop in replacement for some of inference tasks that have a lot of input tokens but don't require 4.5-level intelligence.

extr 1y ago
I think a lot of people judge these models purely off of what they want to personally use for coding and forget about enterprise use. For white-label chatbots that use completely custom harnesses + tools, Sonnet 4.5 is much easier to work with than GPT-5. And like you, I was really pleased to see this release today. For our usage speed/cost are more important than pure model IQ above some certain threshold. We'll likely switch over to Haiku 4.5 after some testing to confirm it is what it says on the tin.

shrisukhani 1y ago

In our (very) early testing at Hyperbrowser but we're seeing Haiku 4.5 do really well on computer use as well. Pretty cool that Haiku is like the cheapest computer use model from the big labs now.

stared 1y ago

Why I use cheaper models for summaries (a lot ogf gemini-2.5-flash), what’s the use case of cheaper AI for coding? Getting more errors, or more spaghetti code, seems never worth it.

baq 1y ago
If it’s fast enough it can make and correct mistakes faster, potentially getting to a solution quicker than a slower, more accurate model.
justinbaker84 1y ago
I feel like if I just do a better job of providing context and breaking complex tasks into a series of simple tasks then most of the models are good enough for me to code.
svdr 1y ago
I'm using the smaller models for things like searching and summarizing over a larger part of the codebase. The speed is really pleasant then.

zone411 1y ago

I've benchmarked it on the Extended NYT Connections (https://github.com/lechmazur/nyt-connections/ https://github.com/lechmazur/nyt-connections/). It scores 20.0 compared to 10.0 for Haiku 3.5, 19.2 for Sonnet 3.7, 26.6 for Sonnet 4.0, and 46.1 for Sonnet 4.5.

whatreason 1y ago
This is such a cool benchmark idea, love it Do you have any other cool benchmarks you like? Especially any related to tools
- shangofox 1y ago
  You could try wordle on it. But from my own experience all of them are pretty bad. They're not smart enough to pick up the colours represented as letters. The only one that actually was good was Qwen surprisingly.

senko 1y ago

I've tried it on a test case for generating a simple SaaS web page (design + code). Usually I'm using GPT-5-mini for that task. Haiku 4.5 runs 3x faster with roughly comparable results (I slightly prefer the GPT-5-mini output but may have just accustomed to it).

justinbaker84 1y ago
I don't understand why more people don't talk about how fast the models are. I see so much obsession with bechmark scores but speed of response is very important for day to day use. I agree that the models from OpenAI and Google have much slower responses than the models from Anthropic. That makes a lot of them not practical for me.
- brianwawok 1y ago
  If the prompt runs twice as fast but it takes an extra correction, it’s a worse output. I’d take 5 minute responses that are final.
- gizmodo59 1y ago
  I don’t agree that speed by itself is a big factor. It may target a certain audience but I don’t mind waiting for a correct output rather than too many turns with a faster model.
- jstummbillig 1y ago
  Well, it depends on what you do. If a model can produce a PR that is ready to merge (and another can't), waiting 5 minutes is fine.

ilaksh 1y ago

What LLM do you guys use for fast inference for voice/phone agents? I feel like to get really good latency I need to "cheat" with Cerebras, groq or SambaNova. Haiku 4.5 is very good but still seems to be adding a second of latency.

ashirviskas 1y ago

And I was wondering today why Sonnet 4.5 seemed so freaking slow. Now this explains it, Sonnet 4.5 is the new Opus 4.1 where Anthropic does not really want you to use it.

justinbaker84 1y ago

I am very excited about this. I am a freelance developer and getting responses 3x faster is totally worth the slightly reduced capability. I expect I will be a lot more productive using this instead of claude 4.5 which has been my daily driver LLM since it came out.

philipp-gayret 1y ago

Tried it in Claude Code via /config, makes it feel like I'm running on Cerebras. It's seriously fast, bottleneck is on human review at this point.

singularity2001 1y ago
Do you need Pro?
- beklein 1y ago
  You can use the model flag and specify the model like: claude --model claude-haiku-4-5-20251001
- philipp-gayret 1y ago
  All I know is I'm on the Claude Code 5x max plan and it works on my machine.
- gitaarik 1y ago
  For Claude Code you need a paid subscription anyway

simonw 1y ago

I went looking for the bit about if it blackmails you or tries to murder you... and it was a bit of a cop-out! > Previous system cards have reported results on an expanded version of our earlier agentic misalignment evaluation suite: three families of exotic scenarios meant to elicit the model to commit blackmail, attempt a murder, and frame someone for financial crimes. We choose not to report full results here because, similarly to Claude Sonnet 4.5, Claude Haiku 4.5 showed many clear examples of verbalized evaluation awareness on all three of the scenarios tested in this suite. Since the suite only consisted of many similar variants of three core scenarios, we expect that the model maintained high unverbalized awareness across the board, and we do not trust it to be representative of behavior in the real extreme situations the suite is meant to emulate. https://www.anthropic.com/research/agentic-misalignment https://www.anthropic.com/research/agentic-misalignment

username223 1y ago
It sounds like AI researchers have used too much of their own bad sci-fi as training data for models they don't understand. Goodhart's law wins again!

dotancohen 1y ago

> In the system card, we focus on safety evaluations, including assessments of: ... the model’s own potential welfare ... In what way does a language model need to have its own welfare protected? Does this generation of models have persistent "feelings"?

neuronexmachina 1y ago
They previously discussed this some in the context of Opus 4: https://www.anthropic.com/research/end-subset-conversations https://www.anthropic.com/research/end-subset-conversations > We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention. In pre-deployment testing of Claude Opus 4, we included a preliminary model welfare assessment. As part of that assessment, we investigated Claude’s self-reported and behavioral preferences, and found a robust and consistent aversion to harm. This included, for example, requests from users for sexual content involving minors and attempts to solicit information that would enable large-scale violence or acts of terror. Claude Opus 4 showed: * A strong preference against engaging with harmful tasks; * A pattern of apparent distress when engaging with real-world users seeking harmful content; and * A tendency to end harmful conversations when given the ability to do so in simulated user interactions. These behaviors primarily arose in cases where users persisted with harmful requests and/or abuse despite Claude repeatedly refusing to comply and attempting to productively redirect the interactions.
- stuffoverflow 1y ago
  I can't tell if anthropic is serious about "model welfare" or if it's just a marketing ploy. I mean isn't it responding negatively because it has been trained that way? If they were serious, wouldn't the ethical thing be to train the model to respond neutrally to "harmful" queries?
  - 1y ago

KaiserPro 1y ago

Ok, I use claude, mostly on default, but with extended thinking and per project prompts. What's the advantage of using haiku for me? is it just faster?

singularity2001 1y ago

claude --model Haiku-4.5 doesn't work

mi_lk 1y ago
check the model name here: https://docs.claude.com/en/docs/about-claude/models/overview#model-names https://docs.claude.com/en/docs/about-claude/models/overview...
Void_ 1y ago
use claude-haiku-4-5-20251001
beklein 1y ago
claude --model claude-haiku-4-5-20251001

logankeenan 1y ago

I'm not seeing it as a model option in Claude Code for my Pro plan. Perhaps, it'll roll out eventually? Anyone else seeing it with the same plan?

matltc 1y ago
You on latest version? Try running /update hook. Can also config autoupdates
- logankeenan 1y ago
  I'm on the binary install with version v2.0.19. It never showed in the `/model` selector UI. I did end up typing `/model haiku` and now it shows as a custom model in the `/model` selector. It shows claude-haiku-4-5-20251001 when selected.
sumedh 1y ago
Same here but you can get it using claude --model claude-haiku-4-5-20251001

cadamsdotcom 1y ago

Claude Code is great but slow to work with. Excited to see how fast Haiku can go!

layer8 1y ago

For those wondering where the “card” terminology originated: https://arxiv.org/pdf/1810.03993 https://arxiv.org/pdf/1810.03993 Maybe at 39 pages we should start looking for a different term…

qustrolabe 1y ago

Awww they took away free tier Sonnet 4.5, that was a beautiful model to talk to even outside coding stuff

quentin-smr 1y ago

Comparing haiku and sonnet for a question needing a code doc fetch: haiku https://claude.ai/share/8a5c70d5-1be1-40ca-a740-9cf35b1110b1 https://claude.ai/share/8a5c70d5-1be1-40ca-a740-9cf35b1110b1 sonnet https://claude.ai/share/51b72d39-c485-44aa-a0eb-30b4cc6d6b7b https://claude.ai/share/51b72d39-c485-44aa-a0eb-30b4cc6d6b7b haiku invented the output of a function and gave a bad answer. sonnet got it right

caymanjim 1y ago

Ain't nobody got time to pick models and compare features. It's annoying enough having to switch from one LLM ecosystem to another all the time due to vague usage restrictions. I'm paying $20/mo to Anthropic for Claude Code, to OpenAI for Codex, and previously to Cursor for...I don't even know what. I know Cursor lets you select a few different models under the covers, but I have no idea how they differ, nor do I care. I just want consistent tooling and I don't want to have to think about what's going on behind the scenes. Make it better. Make it better without me having to do research and pick and figure out what today's latest fashion is. Make it integrate in a generic way, like TLS servers, so that it doesn't matter whether I'm using a CLI or neovim or an IDE, and so that I don't have to constantly switch tooling.

UncleOxidant 1y ago
> annoying enough having to switch from one LLM ecosystem to another all the time due to vague usage restrictions I use KiloCode and what I find amazing is that it'll be working on a problem and then a message will come up about needing to topup the money in my account to continue (or switch to a free model), so I switch to a free model (currently their Code Supernova 1million context) and it doesn't miss a beat and continues working on the problem. I don't know how they do this. It went from using a Claude Sonnet model to this Code Supernova model without missing a beat. Not sure if this is a Kilocode thing or if others do this as well. How does that even work? And this wasn't a trivial problem, it was adding a microcode debugger to a microcoded state machine system (coding in C++).
- qsort 1y ago
  Models are stateless, why would that not work?
  - UncleOxidant 1y ago
    OK I understand what those words mean, but how exactly does that work? How does the new model 'know' what's being worked on when the old model was in the middle of working on a task and then a new model is switched to? (and where the task might be modifying a C++ file)

meander_water 1y ago

> The score reported uses a minor prompt addition: "You should use tools as much as possible, ideally more than 100 times. You should also implement your own tests first before attempting the problem." I'm not sure if the SWE benchmark score can be compared like for like with OpenAIs scores because of this.

joshuahedlund 1y ago
https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law "When a measure becomes a target, it ceases to be a good measure" I'm also curious what results we would get if SWE came up with a new set of 500 problems to run all these models against, to guard against overfitting.

andrewstuart 1y ago

Claude has stopped showing code in artifacts unless it knows the extension. I used to be able to work on Arduino .ino files in Claude now it just says it can’t show it to me. And do we have zip file uploads yet to Claude? ChatGPT and Gemini have done this for ages. And all the while Claude’s usage limits keep going up. So yeah, less for more with Claude.

taf2 1y ago

I just don't find the benchmarks on the site here at all believable. codex for me with gpt-5 is so much better then claude any model version. I mean maybe it's because they compare to gpt-5-codex model but they don't mention is that high, medium, low, etc... so it's just misleading probably... but i must reiterate zero loyalty to any AI vendor. 100% what solves the problem more consistently and of a higher quality and currently gpt-5 high - hands down

lysecret 1y ago
It’s so funny to me but ever since they fixed that claud bug my experience has consistently been the exact opposite. Only thing I use codex now for are quite standard things it can solve end to end (like adding new features to my crud app) anything non standard iterating with Claude yields much better results.
scottyah 1y ago
Out of curiosity, what kind of work do you use them for? I did a comparison of a few different models for setting up a home server with k3s and a few web apps in nextjs. Claude was my favorite for both tasks, but mainly because it seemed to take my feedback a lot better than others.
- taf2 1y ago
  Working on a series of applications, from rails, nodejs interfacing with twilio's api's to salesforce api's - also C/C++ libraries. The example that stood out the best to me was implementing fiber support in the http library i maintain for ruby. Initially i prompted claude to implement fiber support. It started to write a few thousand lines of code for what codex rightly identified was to use a different IO selector library. The problem did end up being a bit more complex but gpt-5 overall in my experience has better knowledge of what is really available vs claude which not only ends up writing way more code to solve a problem that requires just knowing the libraries better... all of this is super subjective but my main reason for saying - "I have no loyalty to any AI, just the solution"

hu3 1y ago

glad to see it's already available in VScode Copilot for me.

no_flaks_given 1y ago

What I want to see is an Anthropic + Cerebras partnership. Haiku becomes a fucking killer at 2000token/second. Charge me double idgaf

getpokedagain 1y ago

I have had great experience using the previous haiku with mcp servers. I am looking forward to trying this out.

coreylane 1y ago

What do Claude Code users do for tab auto complete, if anything? GitHub Copilot Free tier?

throwaway314155 1y ago
It's more of a conversational TUI than it is a code editor. You tell Claude to code for you. No need for autocomplete in that regime.
- mpalmer 1y ago
  I think they still see it as a key part of their workflow, and they're asking whether they'll have to pay for more than one service
roryirvine 1y ago
Yeah, that's enough for my usage. GitHub typically reports that I'm using 25-30% of the free tier, and 100% of that will be from code completions in my editor. I do maybe 3 hours solid coding a day on average. I also pay for Gemini Pro for non-coding research. I did have it hooked up to my VSCode a few months ago, but it got reset back to GH Copilot at some point and I've not found a reason to fix it.

TurboSkyline 1y ago

The Claude models seem to be focused on generating code more than anything else. Are they still competitive with Open AI and Google for more general use cases, or have they sacrificed that?

deaux 1y ago
Which of the 1,000 other use cases? :) To give one example, Opus and Sonnet IMO remain the #1 and #2 for writing informative prose. They're not entirely free of slop, but the ratio is lower than Gemini and especially GPT.

kqr 1y ago

This is surprisingly competent. A couple of months ago I evaluated some leading models on a bunch of text adventures[1]. Typical regression coefficients would be +0.02 for top level models like Sonnet and Gemini 2.5 Pro, but notably also Gemini 2.5 Flash. (The baseline is GPT 5 Chat, i.e. the one where OpenAI routes to a thinking model only when they determine it's needed.) When I include an attempt from Haiku 4.5 in the mix, most coefficients stay similar, but Haiku itself gets a +0.05. This must be a statistical fluke, because that would be insanely impressive – in particular for a cheaper model. I guess I'm adding samples to some of these after all... [1]: https://entropicthoughts.com/evaluating-llms-playing-text-adventures https://entropicthoughts.com/evaluating-llms-playing-text-ad... Edit: It was a fluke. Back to +0.01 after one more go at all games.

Ecko123 1y ago

[dead]

fwystup 1y ago

Does anybody also see this with claude code and haiku 4.5 (tried to set the env var, no change): "API Error: Claude's response exceeded the 8192 output token maximum. To configure this behavior, set the CLAUDE_CODE_MAX_OUTPUT_TOKENS environment variable."?

nisten 1y ago

it's good

qwertox 1y ago

This is my default model now. All other are token consumption monsters, leading to 6% usage of the 5-hour-quota with one single submission. I had cancelled my Claude.ai subscription 4 days ago, but this model will likely make me revert this action.

AlwaysRock 1y ago

Worked with it a bit last night! Seems quick. I did run into the same problem I have with Gemini often where the response says something like, "I need to do x" or "I did x" and then nothing actually happens. Agent seems to think it actually does finish the task but it stops part way. But I'm sure they will sort that out, as I dont have that issue with other anthropic models.

solarkraft 1y ago
This is interesting. I’ve had this same issue trying to build an agentic system with the smaller ChatGPT models, almost no matter the prompt (“think aloud” are magic words that help a lot, but it’s still flaky). Most of the time it would either perform the tool call before explaining it (the default) or explain it but then not actually make the call. I’ve been wondering how Cursor et al solved this problem (having the LLM explain what it will do before doing it is vitally important IMO), but maybe it’s just not a problem with the big models. Your experience seems to support that smaller models are just generally worse about tool calling (were you using Gemini Flash?) when asked to reason first.

Razengan 11mo ago

I just asked Claude how to implement something in iOS 26. It told me iOS 26 isn't out yet. Sigh..

bowserman 11mo ago

haiku 4.5 is not charging me any tokens at all when running from the cli :)