4 ms·
https://crfm.stanford.edu/2023/03/13/alpaca.html https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be
by famouswaffles 4y ago
https://crfm.stanford.edu/2023/03/13/alpaca.html https://crfm.stanford.edu/2023/03/13/alpaca.html
They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci)
I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.
- whimsicalism 4y agoI am curious how you are expected to prove in a court of law that the weights you have are derived from some licensed weights if I take just a few gradient steps from the original weights.
- hgsgm 4y agoSame way that court determines whether my book is a paraphrase of yours. Substantial similarity.
- whimsicalism 4y agoI'm curious how they will be doing that for weights though. Euclidean distance?
- sanxiyn 4y agoHere is an example I just made up: in the network, include a token that is never produced by the tokenizer, and watermark the token's embedding. It won't be touched by normal fine tuning and will remain the same. I am sure there are better methods.
- bryananderson 4y agoI am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.
- clbrmbr 4y agoJust because they cannot be copyrighted doesn’t mean that they cannot be considered a trade secret. The agreement for using the product could done with an NDA. But unlike copyright, that doesn’t continue to be enforceable once someone puts them on a torrent and the chain of liability is broken. /notalawyereither
- dchest 4y agoYeah, this happened with RC4 cipher which used to be a trade secret until someone posted it on Usenet: https://en.wikipedia.org/wiki/RC4#History https://en.wikipedia.org/wiki/RC4#History
- throwaway1851 4y agoWell, trade secret law requires the owner of the trade secret to take steps to keep it... secret. Facebook has done the opposite, and they haven’t made the people they distributed the weights to sign an NDA.
- ronsor 4y agoAnd Facebook employees openly discuss the torrent magnet links.
- Der_Einzige 4y agoGod damn I hope this is where IP law lands. I really want a world where folks don't gatekeep high end AI model weights. Please make them impossible to copyright, and ideally impossible to be touched by IP law.
- hgsgm 4y agoWhy? I mean, do you feel differently about every other patented invention or IP? That would be weird.
- bogwog 4y agoThat seems like wishful thinking. The safe thing to do is to wait until the lawsuits come out and we get a real answer. Or if you’re not risk averse, then go for it and take one for the team!
- jpeter 4y agoThey probably did but didn't release them for the public.
- sebzim4500 4y agoWhy would they withold those ones but the the 65B ones? Seems like a weird choice IMO.