2 ms·
Well yes. This is precisely the type of info that OpenAI is keeping secret, because it could turn out that it's smart because its huge OR smart because it is RL
by jiggawatts 3y ago
Well yes. This is precisely the type of info that OpenAI is keeping secret, because it could turn out that it's smart because its huge OR smart because it is RLHF-ed to death. Knowing which is the secret sauce might be sufficient for a large org to reproduce it.
E.g.: if the secret is that it has been trained on tons of "books", then Google could just throw Google Books at Bard 3.
- sillysaurusx 3y agoOr books3. :) (I put together a dataset of 190,000 books that I called books3, which llama eventually trained on. Usage rights are a big interest of mine, primarily because there’s a weird disconnect of people trying to claim copyright over models when the underlying data obviously wasn’t copyrightable.)
- selestify 3y agoA model (if trained right) will not include the training texts verbatim, and will arguably be a big enough transformation over the source material to be copyrightable in its own right. IANAL, and I’m not saying how this is how it’ll play out in the courts, but I don’t see why this is a “weird disconnect”
- _ea1k 3y agoI suspect the challenge is avoiding cases where answers are overfit. Can you get it to give you the full text of a chapter in a book that it has read, for example? It'd be easy to imagine models that accidentally made this possible.
- yyyk 3y agoThanks for your work! I had already downloaded the entire dataset. But I wonder what was the criteria behind book selection? I found it difficult to understand at places*. * Sometimes a book has several very slightly different versions. A lot of Fantasy. Sometimes many parts of a series are included but not all of it (I can imagine an argument for a sample, or for the series in its entirity). Commentaries about important philosophical arguments like Rawls' 'A theory of justice', but not the work itself.
- sillysaurusx 3y agoIt was bibliotik. Embarrassingly I didn’t even think that duplicate books could be a problem. The llama folks had to dedupe the books themselves. Someday I’d like to do that too and release a cleaned version. Basically, the-eye.eu was at one point hosting all of bibliotik, so I downloaded all the epubs and converted them to text. I still have those epubs (incidentally thanks to Carmack, who through a convoluted process managed to save the them and send them to me via snail mail) and I’ve been considering releasing them so that you can filter the books yourself.
- MacsHeadroom 3y agoWe know for a fact that GPT-4's RLHF makes the model worse by incurring an "alignment tax." Microsoft Research and others had access to GPT-4 pre-RLHF and all reported that it was far more capable then and that it lost capability as RLHF/Alignment training checkpoints came in.
- jdironman 3y agoIs the GPT-0314 model (via API) the one before RLHF?
- MrNeon 3y agoHa ha ha ha. No. OpenAI will never make pre-alignment models available.
- astrange 3y agoI think sama said this week they might release GPT3.0 DaVinci. That's a pre-alignment model.
- bugglebeetle 3y agoI would bet a boatload of money that all the 3-letter folks do, in fact, have access to the pre-alignment models, maybe with whatever minimal RLHF is needed to make them usable in various interfaces. The public offerings are intended to make the unaligned models more powerful by gathering training data.
- why_only_15 3y agocode-davinci-002, the gpt-3.5 base model, was available for a while until it was removed recently. Access is still available for researchers. Various researchers and other entities have access to the GPT-4 base model as well.