Yeah, RETRO [0] embeds all an entire question/prompt, and searches for similar text passages with k-NN, then does further processing. This can kind of be unders

Yeah, RETRO [0] embeds all an entire question/prompt, and searches for similar text passages with k-NN, then does further processing. This can kind of be understood as attention on paragraphs. This preprint instead does k-NN and calls it attention on single tokens. So not the same. But similar. [0] https://jalammar.github.io/illustrated-retrieval-transformer/ https://jalammar.github.io/illustrated-retrieval-transformer...