3 ms·
Nice catch. GPT-3 at least was trained on “Books2” which is very widely suspected of containing all of Libgen and Zlibrary. If the questions are in Libgen, th
by cameldrv 3y ago
Nice catch. GPT-3 at least was trained on “Books2” which is very widely suspected of containing all of Libgen and Zlibrary. If the questions are in Libgen, this whole paper is invalid.
- pants2 3y agoConsidering that textbooks are probably the single highest quality source of training data for LLMs, I would be very surprised if OpenAI wasn't buying and scanning textbooks for their own training data (including books that aren't even in Books2).
- kaycey2022 3y agoIt’s highly unlikely that they will spend hundreds of thousands of dollars on buying their own copies when it’s not even remotely clear that doing so will be enough to answer the copyright violation cases they are facing.