4 ms·
with ever increasing context size — Claude 3 Sonnet, 204,800 token context size, 500 page document — sometimes not opting for chunking but at the same time opti
by constantinum 2y ago
with ever increasing context size — Claude 3 Sonnet, 204,800 token context size, 500 page document — sometimes not opting for chunking but at the same time optimizing for costs and latency might be a better solution, strategies like: Summary extraction and single pass extraction works well[1]
[1] - https://docs.unstract.com/editions/cloud_edition#summary-extraction-and-single-pass-extraction-reduce-cost-by-up-to-6x-per-document-extraction https://docs.unstract.com/editions/cloud_edition#summary-ext...
- J_Shelby_J 2y agoIn theory. In practice, context size degrades performance.