4 ms·
Companies have hijacked the open source concept to mean downloadable blob and we follow them as I see in the comments. It’s a real shame.
by JTyQZSnP3cQGa8B 2y ago
Companies have hijacked the open source concept to mean downloadable blob and we follow them as I see in the comments. It’s a real shame.
- raverbashing 2y agoBetter a downloadable blob than a non-downloadable one
- kortilla 2y agoCool, that’s not open source though. That’s like a chef giving you chicken instead of beef and calling it vegetarian.
- ineedasername 2y agoI’d say it’s more like eating Chicken Cordon Bleu and then asking the chef for a recipe, who replies, “Certainly! Step 1) Acquire Chicken Cordon Bleu, preferably cooked. Step 2) if uncooked, cook. Otherwise, consume
- aiono 2y agoParent doesn't argue about that. How is this relevant?
- nicce 2y agoThis is the problem - we accept this approach and then they don't have to do any effort to correctly publish them in open means.
- frabcus 2y agoNo, it's not, as it means nobody is pushing for actually open models. A truly open model has open code that gathers pre-training data, open pre-training data, open RLHF data, open RLAIF data generated from its open constitution and so on. The binary blob is the last thing I'd want - as a heavy user of LLMs I'm actually more interested in the detail of what all training data is in full, than I am the binary blob.
- tuananh 2y agoIBM Granite is actually open https://www.ibm.com/granite https://www.ibm.com/granite
- __m 2y agoat least for the pre-training data there are some open source torrent clients [0]. [0] https://news.ycombinator.com/item?id=42971446 https://news.ycombinator.com/item?id=42971446
- ks2048 2y agoHere's a real open one: https://allenai.org/olmo https://allenai.org/olmo
- danielbln 2y agoI remember the NVIDIA Linux kernel binary blob driver discussions from the early-mid 2000s. Who knew we had an open source driver all along...
- soraminazuki 2y agoWhy was the free software movement a thing when Windows was open source all along, haha.
- CTDOCodebases 2y agoIn that case Linus needs to make a retraction.
- JimDabell 2y agoSomething isn’t open-source because you get everything that went into making it. Something is open-source if you can change it (relatively) easily. The GPL and open-source definition both refer to “the preferred form for making modifications”. The preferred form for modification in the Nvidia driver’s case is the source code. The preferred form for modification in this case is the weights. Open-source as a concept doesn’t really correspond well with LLMs but to the extent that it does, access to the training data is not required because that training data is not the preferred form for making modifications.
- fragmede 2y agoPreferred by whom? Sharing models isn't open source, and we're just going to have to keep having this argument. Letting us download the model is a very nice thing for Facebook to do, but you don't get to call it open source if you're not showing us the source! Explicitly, if we can't see the forced alignment, where the model gets its refusal to talk about Tiananmen Square or how to make meth or it The Information is a reputable news source, then it's not open. The preferred form of modification is to take the data, and train it. That some people have been able to take the model and tweak it, doesn't make it preferable.
- a-dub 2y agothis is correct. "open source" means everything required to recreate from scratch and improve. not "here's a massive binary, an interpreter script and permission."
- ncr100 2y agoFeeling my inner Klingon, "Where is the honor in releasing a binary blob and calling it .. open source. Pfah!"
- userbinator 2y agoHow can you even "open source" an AI model without all of the, presumably copyrighted and extremely voluminous, training data?
- simondotau 2y agoYou can't. But that's not an excuse to misuse the label.
- thephyber 2y agoYou would open source the procedure and reference where the data came from. If there is any non-open source content used in training, then the project couldn’t qualify as “open source”. But this thread is about misuse of the term as applied to the weights package. Those of us who know what open source means should not continue to dilute the term by calling these LLMs by that term.
- deleted 2y ago[deleted]
- a-dub 2y agothat's how you know when you actually have agi, when you have something that you don't have to shovel in every written word known to man to make it work, but rather can seed it with a few dense public domain knowledge compendia and have it derive everything else for itself from those first principles- possibly going through several stages of from scratch training and regeneration.
- guappa 2y agoI'm guilty of this. I didn't publicly shame a coworker who installed fb's model and said "it's open source" just for the sake of peace.
- palata 2y agoIt's started with abusing the term "AI", I don't see a reason why they would not abuse "open source". I guess it's what happens to language when a concept becomes mainstream: people use it wrong, but if enough people do it, it becomes the new meaning? But I agree, it's a real shame.
- beeflet 2y agoI dislike when people like RMS get semantic and gatekeep words like "free software", but this is the end product of a world without gatekeepers. People just use words in a way that's convenient to them.
- zimpenfish 2y ago> People just use words in a way that's convenient to them. Literally how language has always worked and evolved, though.
- mplanchard 2y agoLanguage has always been a push and pull between evolving (descriptivist) and correct (prescriptivist) usage. Neither side is going anywhere.
- zimpenfish 2y ago> correct (prescriptivist) usage Oof, I know there's a bunch of linguists and grammarians who are going to mock you for that bracket.
- mplanchard 2y agoWhy? Prescriptivist/prescriptivism is afaik the usual term. Proscriptivist is the other, but quite rare, so rare that my phone dictionary says it’s a misspelling.
- 2y ago
- mmoustafa 2y agoit's closed source and open outcome
- silisili 2y agoSo is asking ChatGPT to write your application, then open sourcing said application IMO. I see both sides here, but I don't think it's a hill worth dying on. The 'open source' part in this case is just not currently easily modifyable. That may not always be the case.
- TheDong 2y agoThis is still to be determined, based on whether the output of ChatGPT is copyrightable by ChatGPT, copyrightable by the requester, or something else. I think the two plausible answers are: 1. The person prompting (for example telling chatgpt 'please produce a fizzbuzz program') owns the copyright. The creativity lies in the prompt, and the chatgpt transformation is not transformative or meaningful. 2. The output of ChatGPT is derivative of the training data, and so the copyright is owned by all of the copyright holders of the input training data, i.e. everyone, and it's a glowing radioactive bomb of code in terms of copyright that cannot be used or licensed meaningfully in open source terms. There are existing things like 1, where for example if someone takes a picture, and then uses photoshop to edit it, possibly with the "AI erase" tool thingy, they still own the photo's copyright. Photoshop transformed their prompt (a photo), but adobe doesn't get any copyright, nor do any of the test files adobe used to create their AI tool. I don't think AI is like that, but it hasn't gone to court as far as I know, so no one really knows.
- aerzen 2y agoit's open weights
- llm_trw 2y agoAn llm isn't software any more than a matrix is. What do you think an open source matrix should look like?
- stonogo 2y agoIs this question in good faith? The way generated code and data should be open sourced is by releasing the tools and configuration used to generate it. There's never been much confusion around this, to my knowledge. I'm not even necessarily advocating that these things should be released, but the term "open source" has a pretty well-understood meaning that is being equivocated here.
- entropi 2y agoA compiled executable is not any less software than the source code. But the point of open source code is not the ability to see the CPU instructions though, is it? Its about reproducibility and modifiability. Compiled executables (and their licences) lack that. The same as these downloadable blobs.
- llm_trw 2y agoYou make the start of a good point, but miss most of it. You can absolutely have open source machine code. The issue is and always has been that you need to have access to the same level of abstraction as the people writing the source code. The GPL specifically bans transpilers as a way to get around this. In ML there is _no_ level of abstraction other than the raw weights. Everything else is support machinery no different to an compiler, and os, or a physical computer to run the code on. Linux isn't closed source because they don't ship a C compiler with their code. Why should llama models be any different?
- liampulles 2y agoThank you for pointing this out, I was not thinking clearly about this
- blackeyeblitzar 2y agoMost of these claimed “open” models are not open source. Some of them are open weights. But even some of the ones that share weights are not really open - they force a restricted license on you. To be open source I think they need to share training data and training code under an OSI approved license. AI2 has a model called OLMo that is actually open source. They share the training data, training source code, and many other things: https://allenai.org/blog/olmo2 https://allenai.org/blog/olmo2 They also released an app recently, to do local inference on your phone with a small truly open source model: https://allenai.org/blog/olmoe-app https://allenai.org/blog/olmoe-app
- torginus 2y agoWhile I do agree with your point - I wonder what information companies could release that'd be immediately useful to you. It's not like they understand what the weights mean either and if they released the code and dataset used to create it, you probably couldn't recreate it, owning the fact that you don't own tens of thousands of GPUs. If a software's source is released without all the documentation, commit history, bug tracker data etc., it's still considered open source, yet you couldn't recreate it without that information.