4 ms·
Did anyone else read the generations? They’re… really poor quality, right? Im not sure whether I’m misunderstanding though.
by mbforbes 4y ago
Did anyone else read the generations? They’re… really poor quality, right? Im not sure whether I’m misunderstanding though.
- ShamelessC 4y agoI don't think they are generations, but rather samples from The Pile that are semantically close to the input. Actually, as far as I can tell - the RETRO arch itself isn't trained in this article. It focuses more on how to build the retrieval system with a fast KNN index over all of the Pile.