3 ms·
Yeah that's the issue I was thinking of, how to get it to summarise large documents. Has anyone any ideas?
by nprateem 4y ago
Yeah that's the issue I was thinking of, how to get it to summarise large documents. Has anyone any ideas?
- londons_explore 4y agoPeople have had some success with the following process: Divide your 30,000 word document into a hundred 300 word chuncks. For each chunk, give as input: Please summarize the following text into 50 words: [chunk] Join all the outputs together, and you now have a shorter document. Repeat the process recursively. You can improve the results by doing the process again, but this time giving some context: Please summarize the following text, an extract of a document about [1st attempt at a summary], into 50 words: [chunk]
- londons_explore 4y agoYou can also use "Please suggest a section title for the following text". Then that title can be used in the 2nd round, for example using a query of the form "The following is an extract from the Introduction section of a document about The benefits and disadvantages of nuclear power in sweden:"
- generalizations 4y agoI imagine you could do even better by finetuning the neural net on the document before asking for the recursive summary. Then it has all the information to work with, albeit in a compressed form.