This is interesting. I'm curious about how much (and what) these LLMs memorize verbatim. Does anyone know any more thorough papers on this topic? For example,

This is interesting. I'm curious about how much (and what) these LLMs memorize verbatim. Does anyone know any more thorough papers on this topic? For example, this could be tested on every verse in bible and lots of other text that is certainly in the training data: books in project gutenberg, wikipedia articles, etc. Edit: this (and its references) looks like a good place to start: https://arxiv.org/abs/2407.17817v1 https://arxiv.org/abs/2407.17817v1