"Specifically, we treat each original sentence in the document as an index and the document itself as the object of the index, allowing the LLM to memorize docu

"Specifically, we treat each original sentence in the document as an index and the document itself as the object of the index, allowing the LLM to memorize documents and build indexes through self-supervised learning" This is clever, I haven't seen an effective way to train an LLM to search a document yet and I can imagine this being very effective. I suppose this relies on the over fitting you get when fine tuning on a very small dataset.