2 ms·
There are tricks you can do to better utilize the smaller context window, such as sub-summaries and attention tricks. That's how there are already products on t
by pmarreck 3y ago
There are tricks you can do to better utilize the smaller context window, such as sub-summaries and attention tricks. That's how there are already products on the market that consume entire big PDF's and let you query them. Granted, a larger context window would still work better, but it's possible to do.
- yawnxyz 3y agoit's using "overlapping chunking" methods and it usually works for generic PDFs. It really falls apart on technical documents, SOPs and research articles where you need to get context from chunks way above. Using vector DBs also doesn't work well bc you have to twiddle around with window size / overlappy-ness, which changes depending on what kind of paper you're uploading. It's a mess and takes too long
- marcopicentini 3y agoThe problem is that making a summary of a text of 100k token costs 2$ using Davinci.