5 ms·

Maybe by bureaucrats in the OSI. Meta through the Llama models have done more for open source LLMs than just about anyone else, which the community recognises.

Maybe by bureaucrats in the OSI. Meta through the Llama models have done more for open source LLMs than just about anyone else, which the community recognises. The perfect is the enemy of the good, as usual.

echelon 2y ago
Open source as it stands is insufficient to capture all parts of the pipeline. With code, you just compile and run it. Models require carefully curated training data (lots of it), training code, final weights, production inference code, etc. The danger is the world gets drunk on Meta's models, but then Meta pulls the plug. Meta is doing the world a lot of good with Llama, but we'll be left in a precarious position should they stop being so generous. The next generation of LLMs might make Llama completely pointless. Meta thinks a great deal about the strategy in this space. They license other weights and models as CC-BY-NC and other non-free licenses because they know that they have the SOTA models across several categories and that there's no other competition in the marketplace. They want to retain their advantage when it benefits them. So what happens when the LLM space shifts and Meta no longer needs to appear to be open source? What if the value switches over to diffusion models and away from LLMs, which is an area where Meta shines? Llama is a gift we can't replicate or properly repair and extend. And we need to be careful.
- rat9988 2y ago
  We will have their latest model. Better than none.
  - ddingus 2y ago
    Maybe. The point is cultivating dependency under the color of "open source" is not generally wise. Users of the "open source" LLM do not have all the software freedoms normally associated with "open source." They cannot rebuild it and more. Important pieces are missing the "open" part. Someone says, "here, take this awesome math library, it is open." Then users find out there is a brittle binary blob needed for the whole thing to work. Nobody will want to build on or with it because that blob works, until it doesn't and when it does work, people are not sure what it does exactly. Makes the whole thing a lot more like free beer. In the scheme of things right now, particularly given both the pace of change, and up coming corruption due to feeding models their own output, anyone's latest may just not be relevant that long.
PeterCorless 2y ago
Other industries have seen themselves "poisoned" by vendor-specific definitions of networking protocols, programming paradigms and languages, and so on. The concern is real. You can't just say "Well, I like this product so I approve of the vendor's actions." The whole crux is to get an objective definition of what "open AI" is, and how it needs to behave. If others agree (or disagree) with the vendor that doesn't mean the vendor is permitted to make its unilateral de facto declaration a de jure consensus decision. That's not how standards work.
- Chance-Device 2y ago
  The gist of the article, which is roundly negative towards Meta and Mark Zuckerberg personally simply because he’s an easy target and they want to score some cheap points, is that they/he are actively causing harm by not releasing the data, training methods and everything else that goes into the creation and use of the models. So, the choice is give up everything that gives you any advantage, and hence any incentive to develop anything in the first place, or you’re the bad guy. I’m not sure how they think the world works. But I agree that we should be clear about our standards. So I’d suggest that those with these expectations adopt a new brand other than “open source”. Maybe “puritan approved”.
  - fragmede 2y ago
    model available or open weights are right there. Nothing puritanical about it. the standards for calling something open source are relatively simple - you have to share the source to begin to call it open source. they have not done so, and asking them to use a different name for it when they're not sharing the source really doesn't seem like too much to ask. Except it's Mark and Meta, which are household names, and I'm some Internet rando, so my voice is smaller, hence the accusations of bullying. If someone decides for you that your new name is shithead, and everyone goes along with it because that person's bigger than you, thats bullying.
    - Chance-Device 2y ago
      Meta is telling everyone to call you a shithead? That’s really the equivalent of what’s happening here?
koolala 2y ago
Please call it Open Weights and not Open Source. Open Souce AI will be nothing like Open Weight AI. Open Source AI will be able to reference and share its underlying truth data. Imagine an Open Source AI trained on all GPL code or a Public Library and it could guide you directly to the middle of the source of any code / book it knows.
- Chance-Device 2y ago
  I agree with the other poster who said something to the effect that the model is open source and the released weights are, well, open weight. But the distinction is so trivial that I think it highlights the stupidity of this whole thing.
  - koolala 2y ago
    Trivial? ."the model is open source and the released weights are, well, open weight." I'm not sure why you don't want open source model data or think open source is trivial. This sentence makes no distinction or point.
  - Onawa 2y ago
    The distinction in this case is decidedly not trivial. Open weights are great, we can fine tune them and mold them to our needs. But we don't have the training code, training data, etc, to be able to reproduce or tweak things at a more fundamental level. "But not everyone is going to spend the money or time to train their own models from scratch!" I hear you say. And there's some truth to that. But if we had truly open source LLMs, then AuroraGPT development would be fast tracked and we would have a fully government funded scientific frontier model instead of only fine-tuning models.
    - spunker540 2y ago
      I believe the training and inference code is open sourced as well, just not the training data itself, and I think we all agree, data != source.
gtsop 2y ago
I don't care for the "bureaucrats in the OSI" nor the definition of open source, but your argument is complete nonsense. The fact that llama is not open source does not make it less useful or less appreciated. Nor should anything be considered open source because it is useful or appreciated

echelon 2y ago

Open source as it stands is insufficient to capture all parts of the pipeline. With code, you just compile and run it. Models require carefully curated training data (lots of it), training code, final weights, production inference code, etc. The danger is the world gets drunk on Meta's models, but then Meta pulls the plug. Meta is doing the world a lot of good with Llama, but we'll be left in a precarious position should they stop being so generous. The next generation of LLMs might make Llama completely pointless. Meta thinks a great deal about the strategy in this space. They license other weights and models as CC-BY-NC and other non-free licenses because they know that they have the SOTA models across several categories and that there's no other competition in the marketplace. They want to retain their advantage when it benefits them. So what happens when the LLM space shifts and Meta no longer needs to appear to be open source? What if the value switches over to diffusion models and away from LLMs, which is an area where Meta shines? Llama is a gift we can't replicate or properly repair and extend. And we need to be careful.

rat9988 2y ago
We will have their latest model. Better than none.
- ddingus 2y ago
  Maybe. The point is cultivating dependency under the color of "open source" is not generally wise. Users of the "open source" LLM do not have all the software freedoms normally associated with "open source." They cannot rebuild it and more. Important pieces are missing the "open" part. Someone says, "here, take this awesome math library, it is open." Then users find out there is a brittle binary blob needed for the whole thing to work. Nobody will want to build on or with it because that blob works, until it doesn't and when it does work, people are not sure what it does exactly. Makes the whole thing a lot more like free beer. In the scheme of things right now, particularly given both the pace of change, and up coming corruption due to feeding models their own output, anyone's latest may just not be relevant that long.

PeterCorless 2y ago

Other industries have seen themselves "poisoned" by vendor-specific definitions of networking protocols, programming paradigms and languages, and so on. The concern is real. You can't just say "Well, I like this product so I approve of the vendor's actions." The whole crux is to get an objective definition of what "open AI" is, and how it needs to behave. If others agree (or disagree) with the vendor that doesn't mean the vendor is permitted to make its unilateral de facto declaration a de jure consensus decision. That's not how standards work.

Chance-Device 2y ago
The gist of the article, which is roundly negative towards Meta and Mark Zuckerberg personally simply because he’s an easy target and they want to score some cheap points, is that they/he are actively causing harm by not releasing the data, training methods and everything else that goes into the creation and use of the models. So, the choice is give up everything that gives you any advantage, and hence any incentive to develop anything in the first place, or you’re the bad guy. I’m not sure how they think the world works. But I agree that we should be clear about our standards. So I’d suggest that those with these expectations adopt a new brand other than “open source”. Maybe “puritan approved”.
- fragmede 2y ago
  model available or open weights are right there. Nothing puritanical about it. the standards for calling something open source are relatively simple - you have to share the source to begin to call it open source. they have not done so, and asking them to use a different name for it when they're not sharing the source really doesn't seem like too much to ask. Except it's Mark and Meta, which are household names, and I'm some Internet rando, so my voice is smaller, hence the accusations of bullying. If someone decides for you that your new name is shithead, and everyone goes along with it because that person's bigger than you, thats bullying.
  - Chance-Device 2y ago
    Meta is telling everyone to call you a shithead? That’s really the equivalent of what’s happening here?

koolala 2y ago

Please call it Open Weights and not Open Source. Open Souce AI will be nothing like Open Weight AI. Open Source AI will be able to reference and share its underlying truth data. Imagine an Open Source AI trained on all GPL code or a Public Library and it could guide you directly to the middle of the source of any code / book it knows.

Chance-Device 2y ago
I agree with the other poster who said something to the effect that the model is open source and the released weights are, well, open weight. But the distinction is so trivial that I think it highlights the stupidity of this whole thing.
- koolala 2y ago
  Trivial? ."the model is open source and the released weights are, well, open weight." I'm not sure why you don't want open source model data or think open source is trivial. This sentence makes no distinction or point.
- Onawa 2y ago
  The distinction in this case is decidedly not trivial. Open weights are great, we can fine tune them and mold them to our needs. But we don't have the training code, training data, etc, to be able to reproduce or tweak things at a more fundamental level. "But not everyone is going to spend the money or time to train their own models from scratch!" I hear you say. And there's some truth to that. But if we had truly open source LLMs, then AuroraGPT development would be fast tracked and we would have a fully government funded scientific frontier model instead of only fine-tuning models.
  - spunker540 2y ago
    I believe the training and inference code is open sourced as well, just not the training data itself, and I think we all agree, data != source.

gtsop 2y ago

I don't care for the "bureaucrats in the OSI" nor the definition of open source, but your argument is complete nonsense. The fact that llama is not open source does not make it less useful or less appreciated. Nor should anything be considered open source because it is useful or appreciated