5 ms·
Llama had already become a defacto standard for LLMs, between all the fine-tunes and llama.cpp. Giving it a wider license really cements it as a standard while
by andy99 3y ago
Llama had already become a defacto standard for LLMs, between all the fine-tunes and llama.cpp. Giving it a wider license really cements it as a standard while making everybody use it on Meta's terms. It's an "open source" strategy where all the benefits accrue back to Meta instead of the community but that's permissive enough to placate most people.
Personally I feel like it's a dangerous precedent because it shifts open source from a community concept to something that a company lets you have with a bunch of conditions.
- brucethemoose2 3y agoStable Diffusion is a good example of this progression. There are new models/papers/backends coming out all the time (including SDXL, which is similar to the original SD), but the community is so entrenched in Stable Diffusion 1.5 and the old Stability AI PyTorch implementation that moving to anything else hasn't really happened yet. That being said, I think Llama won't stick around as the "de facto" standard for over a year unless Meta keeps releasing better foundational models.
- pavlov 3y agoWhy wouldn’t they? Meta seems to have the money, the hardware, the talent, and the executive motivation to keep producing better models.
- nullc 3y ago> but the community is so entrenched in Stable Diffusion 1.5 and the old Stability AI PyTorch implementation that moving to anything else hasn't really happened yet. It's not just inertia, the later models (even 1.5 over 1.4) are heavily censored. And at least in the case of 1.5 it appears to cause obviously inferior output even for content which is in no way explicit. My understanding is the facebook published Llama 2 chat fine tunes are more or less the most censored LLM yet, refusing to even discuss negative sentiments. I bet we'll see other people's RLHF fine tunes on the base model being used as a point for more developent. It's an unfortunate development that it appears that the primary commercial "value add" is building models that don't do what the users unambiguously direct them to do. It's hard to make a model smarter, but easier to lobotomize one.
- brucethemoose2 3y ago> the later models (even 1.5 over 1.4) are heavily censored. And at least in the case of 1.5 it appears to cause obviously inferior output even for content which is in no way explicit. 1.5 is better than 1.4, but 2.0/2.1 are indeed heavily lobotomized. The jury is still out on SDXL, but I think its much better. But the second thing I alluded to, the backend, is going to be more of an issue. The chat finetunes of Llama 2 are kinda irrelevant because the community will come up with their own very uncensored finetunes.
- nullc 3y agoWell automatic1111 does SDXL and the model is clearly very censored-- now perhaps the censorship doesn't handicap its performance as much as it did with 2.0... that's harder to gauge without an uncensored comparison point.
- ldjkfkdsjnv 3y agoAny open source is good open source, meta has contributed react and pytorch to the world. They should be applauded for their contributions to software. They have done far more than any other big tech company.
- tonmoy 3y agoI was with you until the last sentence. Google has given us Android, Kubernetes, Golang, Tensorflow (not to mention Angular and Chromium)
- mrtranscendence 3y ago> Kubernetes Which they never apologized for!
- speed_spread 3y agoOh yeah, thanks for the 21st century BASIC, Google.
- sanxiyn 3y agoApple did LLVM and WebKit, which seem far more important than React and PyTorch.
- remcob 3y agoWebKit was more or less ‘stolen’ from KDE’s KHTML and KJS. They forked it, developed it in secret, and did not publish anything until it diverged so far from source that it could not be merged back in. While technically legal, it definitely goes against the spirit of open source. LLVM is a better story: also not invented at Apple, but they hired the core developers to continue working on it.
- herbst 3y agoI honestly don't see how react made the world better by abstracting even more of the beautiful html and CSS Logik to messy dependency heavy JavaScript.
- palata 3y ago> it shifts open source from a community concept to something that a company lets you have with a bunch of conditions. Because you think that Protobuf, Android, Chromium, and all the other open source projects by Big Tech are there for the community? It's about control. Not saying it's fundamentally bad, but you can't deny that the fact that Android exists means that it is very difficult to get adoption for an alternative "open" mobile OS, and Google benefits greatly from it. Same for Chromium, and basically everything. When Big Tech open sources something, it's a strategic decision. Not philanthropy.
- Shawnj2 3y agoThe LLM world has been really good at bootstrapping models so I’m sure this will be used to create something exactly or more powerful than it without using any of its existing code.
- psd1 3y agoProtobuf seems like the odd one out. I understood that googlers created it for their own benefit and tossed it over the wall because "why not". Am I mistaken?
- palata 3y agoI don't know about the actual decision, but I can see the result: Protobuf is ubiquitous. Now take it like this: if Protobuf was not open source, wouldn't there be an alternative? I would say probably, since there are already alternatives (just not as popular as Protobuf). So from Google's perspective, not open sourcing Protobuf does not really give them an edge: alternatives exist. However, by open sourcing it, they end up with a ton of devs who learn Protobuf outside of Google. That makes it easier for Google technology to be adopted if they release something new, and when they hire a new engineer, it's a win if he already knows Protobuf. Same applies to gRPC and others.
- littlestymaar 3y ago> Personally I feel like it's a dangerous precedent because it shifts open source from a community concept to something that a company lets you have with a bunch of conditions. It's not really a unique precedent, Chromium and Android fit the same model, but it's still very much better than closed-source (electron and custom Android ROMs are something you couldn't have otherwise).