3 ms·
Agreed, I appreciate the transparency (and Anthropic isn't normally very transparent). It's also great to know because I will change how I approach long contex
by freedomben 6mo ago
Agreed, I appreciate the transparency (and Anthropic isn't normally very transparent). It's also great to know because I will change how I approach long contexts knowing it struggles more with them.
- RobinL 6mo agoCould this be because they've found the 1m context uneconomical (ie costs too much to serve, or burns through users quota too quickly causing complaints), and so they're no longer targeting it as a goal
- Someone1234 6mo agoOpus 4.7 is also worse at 256K context. Go look at page 195 and page 196. It is across the board regression, not just 1M context.
- RobinL 6mo agoThanks, interesting. Does this make it more surprising that the other benchmarks have improved? I'm not sure I understand the benchmarks well enough - but I'm wondering whether with agentic workflows it's possible to get away with a smaller more focussed context (and hence lower cost) whilst achieving the same or better performance, because of agentic model's ability to decide what the put in context as they work
- timvb 6mo agowhat's all this mean in real world use?