4 ms·

As is often the case, the article gets the difference between open source and open weights wrong. Also, they don't seem to understand that Apple's variant of t

As is often the case, the article gets the difference between open source and open weights wrong. Also, they don't seem to understand that Apple's variant of the MIT licence is GPL compatible. > This is Apple's variant of MIT. They've added wording around patents, which is why it gets it own shortname. Apple did not give this license a name, so we've named it "Apple MIT License". It is free and GPL compatible. https://fedoraproject.org/wiki/Licensing/Apple_MIT_License https://fedoraproject.org/wiki/Licensing/Apple_MIT_License

gattilorenz 3y ago
Also many HN commenters don’t get the distinction between open source and open weights… not very surprising that ars technica also doesn’t
blackeyeblitzar 3y ago
I’m shocked that Ars got this wrong and is helping Apple openwash their work. But I’m seeing this same utterly basic mistake with other tech media as well, like Venture Beat: https://venturebeat.com/ai/apple-releases-openelm-small-open-source-ai-models-designed-to-run-on-device/ https://venturebeat.com/ai/apple-releases-openelm-small-open...
- Hugsun 3y ago
  I love this term, openwashing. It stumbles off the tongue.
  - squigz 3y ago
    It's invention of terms like this that make it hard for people to take some activist groups seriously...
    - mensetmanusman 3y ago
      Openwashthepolice
    - blackeyeblitzar 3y ago
      Why such a negative take? Open washing is similar to green washing, a practice where companies pretend to be environmentally friendly for marketing purposes. To me the term makes immediate sense. What am I missing?
  - blackeyeblitzar 3y ago
    Any alternative suggestions? Its similar to green washing and other such practices.
- gilgoomesh 3y ago
  There's no open washing — Apple have released the code (both training and inference), the weights and a research paper explaining. It's about as open as you could expect from them.
  - blackeyeblitzar 3y ago
    I didn’t see training source code, but more like a promise to share insights. Am I missing something? Regardless another reason this isn’t open source is because it used a proprietary license instead of an OSI approved open source license.
whiplash451 3y ago
The concept of open-source for a million-dollar scale LLM is not very useful, especially if you don't provide the training set as well. Open weights with a permissive license is much more useful, especially for small and midsize companies.
- gattilorenz 3y ago
  The distinction between the two terms is what is useful. Imagine if people only referred to open source software as "free software", and there's no distinction made whether the software is free as in beer or free as in freedom.
  - taneq 3y ago
    Maybe both are useful? There's "open source" in the FSF sense, which isn't always as useful when talking about modern neural networks (remember when papers used to be published with "open source" Python code for initializing and training the model, but no training data or weights?) Then there's "in the spirit of open source" where the weights and training data are also GPL'd. And there's the whole range in between. Having the training data available is nice, but for large models, having weights provided under a GPL-style license (or even a MIT-style permissive license) is far better in terms of "being able to modify the program" than having the training data that you don't have enough compute resources to use. The distinction between the two, though, is also useful. (I've even seen a few posters here argue that it's not really 'free software' even if everything's provided and GPL'd, if it would take more compute to re-train than they have available, which frankly I think is silly. Free-as-in-freedom software was never guaranteed to be cheap to build or run, and nobody owes you CPU time.)
    - simion314 3y ago
      >Maybe both are useful? There's "open source" in the FSF sense, FSF uses the term free/libre software and they do not like the vague term of open source. The problem is that in English language free also means costs zero and you can get confused with frewware. This is why OP is correct in pointing out wrong term usage, you should not call a GPL software Freeware or a freeware acall it "Free software", it is wrong and causes confusion, So for models would be great we could know if a model is actually open source, open weights with no restrictions or open weights with restrictions on top.
gilgoomesh 3y ago
The lack of distinction is confusing but Apple have released both the training source and the inference source + models/weights at the same time — so at least both are true.
- nicce 3y ago
  Is the training data public as well?
  - raxxorraxor 3y ago
    This time Apple could say that they didn't release it for privacy reasons and this time it would be correct for once.
  - GeekyBear 3y ago
    Yes > RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text documents coming from 84 CommonCrawl snapshots and processed using the CCNet pipeline. https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data > Dolma Dataset: an open dataset of 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials. https://allenai.github.io/dolma/ https://allenai.github.io/dolma/

gattilorenz 3y ago

Also many HN commenters don’t get the distinction between open source and open weights… not very surprising that ars technica also doesn’t

blackeyeblitzar 3y ago

I’m shocked that Ars got this wrong and is helping Apple openwash their work. But I’m seeing this same utterly basic mistake with other tech media as well, like Venture Beat: https://venturebeat.com/ai/apple-releases-openelm-small-open-source-ai-models-designed-to-run-on-device/ https://venturebeat.com/ai/apple-releases-openelm-small-open...

Hugsun 3y ago
I love this term, openwashing. It stumbles off the tongue.
- squigz 3y ago
  It's invention of terms like this that make it hard for people to take some activist groups seriously...
  - mensetmanusman 3y ago
    Openwashthepolice
  - blackeyeblitzar 3y ago
    Why such a negative take? Open washing is similar to green washing, a practice where companies pretend to be environmentally friendly for marketing purposes. To me the term makes immediate sense. What am I missing?
- blackeyeblitzar 3y ago
  Any alternative suggestions? Its similar to green washing and other such practices.
gilgoomesh 3y ago
There's no open washing — Apple have released the code (both training and inference), the weights and a research paper explaining. It's about as open as you could expect from them.
- blackeyeblitzar 3y ago
  I didn’t see training source code, but more like a promise to share insights. Am I missing something? Regardless another reason this isn’t open source is because it used a proprietary license instead of an OSI approved open source license.

whiplash451 3y ago

The concept of open-source for a million-dollar scale LLM is not very useful, especially if you don't provide the training set as well. Open weights with a permissive license is much more useful, especially for small and midsize companies.

gattilorenz 3y ago
The distinction between the two terms is what is useful. Imagine if people only referred to open source software as "free software", and there's no distinction made whether the software is free as in beer or free as in freedom.
- taneq 3y ago
  Maybe both are useful? There's "open source" in the FSF sense, which isn't always as useful when talking about modern neural networks (remember when papers used to be published with "open source" Python code for initializing and training the model, but no training data or weights?) Then there's "in the spirit of open source" where the weights and training data are also GPL'd. And there's the whole range in between. Having the training data available is nice, but for large models, having weights provided under a GPL-style license (or even a MIT-style permissive license) is far better in terms of "being able to modify the program" than having the training data that you don't have enough compute resources to use. The distinction between the two, though, is also useful. (I've even seen a few posters here argue that it's not really 'free software' even if everything's provided and GPL'd, if it would take more compute to re-train than they have available, which frankly I think is silly. Free-as-in-freedom software was never guaranteed to be cheap to build or run, and nobody owes you CPU time.)
  - simion314 3y ago
    >Maybe both are useful? There's "open source" in the FSF sense, FSF uses the term free/libre software and they do not like the vague term of open source. The problem is that in English language free also means costs zero and you can get confused with frewware. This is why OP is correct in pointing out wrong term usage, you should not call a GPL software Freeware or a freeware acall it "Free software", it is wrong and causes confusion, So for models would be great we could know if a model is actually open source, open weights with no restrictions or open weights with restrictions on top.

gilgoomesh 3y ago

The lack of distinction is confusing but Apple have released both the training source and the inference source + models/weights at the same time — so at least both are true.

nicce 3y ago
Is the training data public as well?
- raxxorraxor 3y ago
  This time Apple could say that they didn't release it for privacy reasons and this time it would be correct for once.
- GeekyBear 3y ago
  Yes > RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text documents coming from 84 CommonCrawl snapshots and processed using the CCNet pipeline. https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data > Dolma Dataset: an open dataset of 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials. https://allenai.github.io/dolma/ https://allenai.github.io/dolma/