3 ms·
This comment makes so much sense relative to what I've seen with Claude's 1M context window. It reliably fails to succeed a task with a prompt where I just stuf
by phillipcarter 3y ago
This comment makes so much sense relative to what I've seen with Claude's 1M context window. It reliably fails to succeed a task with a prompt where I just stuff in a big blob of data in the middle as context. But when I use emebddings to only select a small relevant subset of that data, it always passes the task.
- gamegoblin 3y agoYes, Claude 1M is using all sorts of approximation tricks to get that 1M context window. IMO this is actually quite deceptive marketing.
- dpflan 3y agoYes, that's the point now for competing AI research-for-profit companies, whatever metric is technical and sounds important, is going to be used in marketing and valuation determinations. It will be explored for research I'm sure, and then determine its product viability. It's nice competition, but agree, that it can be deceptive.
- l1n 3y agoClaude's context is 100K not 1M [1]. If you're somehow shoving in a million tokens that could explain the issue you're having! [1] https://www.anthropic.com/index/100k-context-windows https://www.anthropic.com/index/100k-context-windows
- gamegoblin 3y agoMisremembered, the main thrust of the comment still stands, the 100K context window isn't "real", it would be absurdly expensive to do it for real. They are using a lot of approximation tricks to get there.
- phillipcarter 3y agoYep, mistype on my end as well. Claude just fails to process the request if you get above 100k tokens (I've done that, heh).