4 ms·
Current LLMs can not summarize books even if they are in their training set. It is only possible if the training set also includes human summaries.
by macilacilove 3y ago
Current LLMs can not summarize books even if they are in their training set. It is only possible if the training set also includes human summaries.
- apwell23 3y ago> Current LLMs can not summarize books even if they are in their training set Why is this? due to legal reasons?
- avandekleut 3y agoThe context windows aren’t large enough, as I understand it. It might be possible via a chain-of-summarization, though.
- macilacilove 3y agoMost importantly due to the context length not being long enough. If the context length was long enough, it is possible that they could do it with clever training. I only trained much smaller language models though.
- dTal 3y ago"Even if" is an odd phrasing. Why would you expect that to be possible? LLMs don't have direct access to their training set. Having a book in the training set just means they emit text in the style of the book - it's not like a database they can query. But they can summarize very well, if the text is front of them.
- macilacilove 3y agoI mean, currently you have to wait for a human to summarize a book, before you can read an llm rephrasing of that summary. Or you have to buy the book and pass it to the llm employing some tricks to avoid reunning out of context length.