4 ms·

The benchmarks are cool and all but 1M context on an Opus-class model is the real headline here imo. Has anyone actually pushed it to the limit yet? Long contex

by dmk 8mo ago

The benchmarks are cool and all but 1M context on an Opus-class model is the real headline here imo. Has anyone actually pushed it to the limit yet? Long context has historically been one of those "works great in the demo" situations.

pants2 8mo ago
Paying $10 per request doesn't have me jumping at the opportunity to try it!
- schappim 8mo ago
  The only way to not go bankrupt is to use a Claude Code Max subscription…
  - dmk 8mo ago
    Yeah, just had to upgrade to Max 20x yesterday because of hitting the limits every day and the extra usage gets expensive very fast.
- cedws 8mo ago
  Makes me wonder: do employees at Anthropic get unmetered access to Claude models?
  - swader999 8mo ago
    It's like when you work at McDonald's and get one free meal a day. Lol, of course they get access to the full model way before we do...
    - wiredpancake 8mo ago
      [dead]
  - ajam1507 8mo ago
    Seems quite obvious that they do, within reason.
  - danw1979 8mo ago
    Boris Cherny, creator of Claude Code, posted about how he used Claude a month ago. He’s got half a dozen Opus sessions on the burners constantly. So yes, I expect it’s unmetered. https://x.com/bcherny/status/2007179832300581177 https://x.com/bcherny/status/2007179832300581177
  - _dark_matter_ 8mo ago
    Don't most jobs have unmetered access? I know mine does
awestroke 8mo ago
Opus 4.5 starts being lazy and stupid at around the 50% context mark in my opinion, which makes me skeptical that this 1M context mode can produce good output. But I'll probably try it out and see
nomel 8mo ago
Has a "N million context window" spec ever been meaningful? Very old, very terrible, models "supported" 1M context window, but would lose track after two small paragraphs of context into a conversation (looking at you early Gemini).
- libraryofbabel 8mo ago
  Umm, Sonnet 4.5 has a 1m context window option if you are using it through the api, and it works pretty well. I tend not to reach for it much these days because I prefer Opus 4.5 so much that I don't mind the added pain of clearing context, but it's perfectly usable. I'm very excited I'll get this from Opus now too.
  - nomel 8mo ago
    If you're getting on along with 4.5, then that suggests you didn't actually need the large context window, for your use. If that's true, what's the clear tell that it's working well? Am I misunderstanding? Did they solve the "lost in the middle" problem? Proof will be in the pudding, I suppose. But that number alone isn't all that meaningful for many (most?) practical uses. Claude 4.5 often starts reverting bug fixes ~50k tokens back, which isn't a context window length problem. Things fall apart much sooner than the context window length for all of my use cases (which are more reasoning related). What is a good use case? Do those use cases require strong verification to combat the "lost in the middle" problems?

by dmk 8mo ago

pants2 8mo ago

Paying $10 per request doesn't have me jumping at the opportunity to try it!

schappim 8mo ago
The only way to not go bankrupt is to use a Claude Code Max subscription…
- dmk 8mo ago
  Yeah, just had to upgrade to Max 20x yesterday because of hitting the limits every day and the extra usage gets expensive very fast.
cedws 8mo ago
Makes me wonder: do employees at Anthropic get unmetered access to Claude models?
- swader999 8mo ago
  It's like when you work at McDonald's and get one free meal a day. Lol, of course they get access to the full model way before we do...
  - wiredpancake 8mo ago
    [dead]
- ajam1507 8mo ago
  Seems quite obvious that they do, within reason.
- danw1979 8mo ago
  Boris Cherny, creator of Claude Code, posted about how he used Claude a month ago. He’s got half a dozen Opus sessions on the burners constantly. So yes, I expect it’s unmetered. https://x.com/bcherny/status/2007179832300581177 https://x.com/bcherny/status/2007179832300581177
- _dark_matter_ 8mo ago
  Don't most jobs have unmetered access? I know mine does

awestroke 8mo ago

Opus 4.5 starts being lazy and stupid at around the 50% context mark in my opinion, which makes me skeptical that this 1M context mode can produce good output. But I'll probably try it out and see

nomel 8mo ago

Has a "N million context window" spec ever been meaningful? Very old, very terrible, models "supported" 1M context window, but would lose track after two small paragraphs of context into a conversation (looking at you early Gemini).

libraryofbabel 8mo ago
Umm, Sonnet 4.5 has a 1m context window option if you are using it through the api, and it works pretty well. I tend not to reach for it much these days because I prefer Opus 4.5 so much that I don't mind the added pain of clearing context, but it's perfectly usable. I'm very excited I'll get this from Opus now too.
- nomel 8mo ago
  If you're getting on along with 4.5, then that suggests you didn't actually need the large context window, for your use. If that's true, what's the clear tell that it's working well? Am I misunderstanding? Did they solve the "lost in the middle" problem? Proof will be in the pudding, I suppose. But that number alone isn't all that meaningful for many (most?) practical uses. Claude 4.5 often starts reverting bug fixes ~50k tokens back, which isn't a context window length problem. Things fall apart much sooner than the context window length for all of my use cases (which are more reasoning related). What is a good use case? Do those use cases require strong verification to combat the "lost in the middle" problems?