3 ms·
The information is probably lost, training on very long inputs is expensive, we usually train on short inputs and only send longer ones at the end. So the autho
by visarga 2y ago
The information is probably lost, training on very long inputs is expensive, we usually train on short inputs and only send longer ones at the end. So the author name would not appear together with all the chunks unless it appears on every page in the header, assuming the headers are not stripped out first.
- nthingtohide 2y agoI am wondering the same techniques we use to augment chunks for RAG also the same technique used to augment training data with rich metadata. A preprocessing step which ensures metadata rich text for creating foundational models.