3 ms·
If you chunk the document well (which is surprisingly tricky with messy documents), then you can do summary via map reduce or other techniques. To be clear: I
by phren0logy 2y ago
If you chunk the document well (which is surprisingly tricky with messy documents), then you can do summary via map reduce or other techniques.
To be clear: I don't trust it to provide an accurate summary; it's not meant to replace reading the documents. But it may help to find relevant parts of a document later, or get a reasonable overview of documents before starting a manual review. I expect (but will have to see) if it's better than just opening up a random PDF and starting there.
I recently had a case with over 24k pages of records, some of which were in PDFs that were thousands of pages long. The ability to do semantic search rather than keyword search was useful for when I said to myself, "Didn't I see something about that before? Where was that?"
- sroussey 2y agoI’ve found summarizing this way is a great idea but the summarizing models are not great themselves. Would even prefer an embedding with the summary as maybe the embedding would be better.