3 ms·
Presumably one can test it with documents one has already read and knows before. If the summaries of the test documents are good, future summaries will probably
by ronsor 3y ago
Presumably one can test it with documents one has already read and knows before. If the summaries of the test documents are good, future summaries will probably be OK too.
- lambdaba 3y agoMaybe having it summarize a fiction book (outside of training data)?
- zmmmmm 3y ago> If the summaries of the test documents are good, future summaries will probably be OK too But that is exactly what is problematic with hallucinations. It's a rare / exceptional behaviour that triggers extreme departure from reality. So you can't estimate the extremes by extrapolating from common / moderate observations. You would have to test a lot of previous documents to be confident, and even then there would be a residual risk.
- sitkack 3y agoAs with anyone, if you are going to take action on something, double check it.