3 ms·

...says the owner of it. Now seriously, by Llama being "sort of" open source, it does not seem to be something someone can fork and develop/evolve it without M

...says the owner of it. Now seriously, by Llama being "sort of" open source, it does not seem to be something someone can fork and develop/evolve it without Meta, right? If one day Meta comes and says "we are closing Llama and evolving it in a proprietary mode from now on" would this Llama indie scene continue to exist? If this is the case wouldn't this be considered a dumping strategy by Meta, to cut revenue streams from other platforms (Gemini/OpenAI/Anthropic) and contain their growth?

RicoElectrico 2y ago
The models can be fine-tuned which is good enough.
- mupuff1234 2y ago
  Not good enough to be considered open source.
  - bunderbunder 2y ago
    Realistically the problem here might just be that the concept of open source doesn't really fit machine learning models very well, and we should stop trying to force it. Sharing the end product, but not the tools and resources used to produce it, is how open source has always worked. If I develop software for a commercial operating system using a commercial toolchain, and distribute the source code under GPL, we would call that software open source. Others who get the code don't automatically get the ability to develop it themselves, but that's kind of beside the point. I don't have the rights to publicly redistribute those tools, anyway; the only part I can put under an open source license is the part for which I have copyright. Training data for a LLM like Llama works similarly when it comes to copyright law. They don't own copyright and/or redistribution rights for all of it, so they can't make it open, even if they want to. If that seems unsatisfying, that's because it is. Unfortunately, though, I don't think the Free Software community is going to get very far by continuing to try to fight today's openness and digital sovereignty battles using tactics and doctrine that were developed in the 20th century.
    - koolala 2y ago
      It does fit it. Perfectly. It's incredible. Like an Internet of all Human Knowledge released before 1965. OpenAI could of done this. The battle to me is just people respecting ideas instead of saying they are impossible or unnecessary because what we have is good enough.
- islewis 2y ago
  "good enough" is incredibly subjective here. Maybe good enough for you, but there are many things that are not possible with either the dataset or the weights being available.
  - talldayo 2y ago
    And some things are impossible even with both the dataset and weights. Say you wanted to train the same model as is released, using Meta's hypothetically released training data. You also need to know the starting parameters, the specific hardware and it's quirks during training, the order the data is trained in as well as any other preprocessing techniques used to treat the text. Considering how ludicrously expensive it would be to even attempt a ground-up retrain (as well as how it might be impossible), weights are enough for 99% of people.
- koolala 2y ago
  Good-enough? Please please type out what a truely open source ai model with open weights and open data would be like. I picture it like a Tower of Babel! Very far from "Good-enough"!

RicoElectrico 2y ago

The models can be fine-tuned which is good enough.

mupuff1234 2y ago
Not good enough to be considered open source.
- bunderbunder 2y ago
  Realistically the problem here might just be that the concept of open source doesn't really fit machine learning models very well, and we should stop trying to force it. Sharing the end product, but not the tools and resources used to produce it, is how open source has always worked. If I develop software for a commercial operating system using a commercial toolchain, and distribute the source code under GPL, we would call that software open source. Others who get the code don't automatically get the ability to develop it themselves, but that's kind of beside the point. I don't have the rights to publicly redistribute those tools, anyway; the only part I can put under an open source license is the part for which I have copyright. Training data for a LLM like Llama works similarly when it comes to copyright law. They don't own copyright and/or redistribution rights for all of it, so they can't make it open, even if they want to. If that seems unsatisfying, that's because it is. Unfortunately, though, I don't think the Free Software community is going to get very far by continuing to try to fight today's openness and digital sovereignty battles using tactics and doctrine that were developed in the 20th century.
  - koolala 2y ago
    It does fit it. Perfectly. It's incredible. Like an Internet of all Human Knowledge released before 1965. OpenAI could of done this. The battle to me is just people respecting ideas instead of saying they are impossible or unnecessary because what we have is good enough.
islewis 2y ago
"good enough" is incredibly subjective here. Maybe good enough for you, but there are many things that are not possible with either the dataset or the weights being available.
- talldayo 2y ago
  And some things are impossible even with both the dataset and weights. Say you wanted to train the same model as is released, using Meta's hypothetically released training data. You also need to know the starting parameters, the specific hardware and it's quirks during training, the order the data is trained in as well as any other preprocessing techniques used to treat the text. Considering how ludicrously expensive it would be to even attempt a ground-up retrain (as well as how it might be impossible), weights are enough for 99% of people.
koolala 2y ago
Good-enough? Please please type out what a truely open source ai model with open weights and open data would be like. I picture it like a Tower of Babel! Very far from "Good-enough"!