10 ms·

Llama and ChatGPT Are Not Open-Source

143 points by webmaven 3y ago

smoldesu 3y ago
Previously: https://news.ycombinator.com/item?id=36820122 https://news.ycombinator.com/item?id=36820122 https://news.ycombinator.com/item?id=36806536 https://news.ycombinator.com/item?id=36806536 https://news.ycombinator.com/item?id=36791671 https://news.ycombinator.com/item?id=36791671 https://news.ycombinator.com/item?id=36783019 https://news.ycombinator.com/item?id=36783019
- version_five 3y ago
  To be fair, the title is the same but this article is really about a study of the openness characteristics of different AI models, which is new. This link had the comparison of different models: https://opening-up-chatgpt.github.io/ https://opening-up-chatgpt.github.io/
  - version_five 3y ago
    Also, there's an error in that table, the first model "Bloomz" does not have an open source license, on the HF page for the model it shows a RAIL license which is a restrictive license that limits you to "approved" uses. The license link goes to the github repo where the code is released under Apache 2.0. I didn't review the rest but I'd say it calls the ranking into question.
littlestymaar 3y ago
Can we please stop using terminology related to code for something that's not code?
- hprotagonist 3y ago
  sometimes it’s hard to see a difference.
- aaur0 3y ago
  Exactly - I wrote something on similar lines here : https://medium.com/@anandandbeyond/rethinking-open-source-for-the-era-of-ai-model-sharing-5b1e770a799d https://medium.com/@anandandbeyond/rethinking-open-source-fo...
- Havoc 3y ago
  Agreed it's annoying, but on a certain level also not entirely wrong. In a software 2.0 [0] world the weights are functionally the code in that it is what gets you from input to output. Open weights or something similar would be better though [0] https://karpathy.medium.com/software-2-0-a64152b37c35 https://karpathy.medium.com/software-2-0-a64152b37c35
  - charcircuit 3y ago
    Weights are configuration.
    - alanbernstein 3y ago
      Weights almost entirely encapsulate the behavior of the model. Whatever structural analogy you want to use, they are closest in spirit to the core functional algorithm of traditional software.
    - dns_snek 3y ago
      Are symbols in your source file merely a configuration file for the compiler/interpreter?
      - charcircuit 3y ago
        No, but I would consider the command line flags to be configuration.
- choxi 3y ago
  There’s always been fuzziness between data and code, it’s kind of the core concept of Turing machines.
- sashb 3y ago
  What is "code" anyways? https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theorem https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...
- raincole 3y ago
  Except it's code. Weight decides how the model works so it's code. Code is just you telling the computer to work. var a = b * 3 + 2; You're telling me the * 3 + 2 part isn't code and only var a = b is code?
jehb 3y ago
Of note, if you're interested in helping participate in the discussion of what an open source AI would actually look like, the Open Source Initiative is looking for your help: https://opensource.org/deepdive/ https://opensource.org/deepdive/
- version_five 3y ago
  In there somewhere in there where they are asking for input? It presents itself that way but it seems the only way to "help" is to propose a presentation under their call for speakers: https://sessionize.com/deepdiveai https://sessionize.com/deepdiveai Is there another way to contribute?
  - jehb 3y ago
    As I read the process, at this point they're primarily looking for "conversation starters" (my phrase, not theirs) in the form of presentations, as well as attendees to discuss at a number of in-person events this year (I'll be at the one at All Things Open in Raleigh NC in October). I think once they've got something a little more substantial to review, they'll have a more robust "public comment"-type process. I'm sure you could also offer a meta-comment on the process itself.
bick_nyers 3y ago
I personally do not want the companies to release training data (at least for a while) because then it gives people leverage to neuter it. I don't want a sanitized LLM, and I don't have $60M lying around to train my own. Copyrighted material, sexual content, political opinions, throw it all in and release it please! Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no. Maybe I just got added to a list somewhere for having this opinion.
- version_five 3y ago
  100% agree. People that want training data released mostly just want to find something to attack, it has nothing to do with transparency.
- threeseed 3y ago
  > reducing bias in the models is a noble goal It's also a necessary goal in order for these models to be more broadly adopted. We've seen too many examples of bias in the training data set manifesting in ways that actively discriminate against people. Which is unethical and in many places illegal. And having copyrighted material and sexual content in your model will simply open you up to lawsuits as is happening right now between authors and OpenAI. Not sure that is a position most startups want to be in.
  - bick_nyers 3y ago
    College students don't want to be criminally charged with theft for pirating an $800 college textbook either. I'm not giving business advice, I'm giving humanity and knowledge propagation advice. As I mentioned, reducing bias is good so long as it doesn't introduce more bias elsewhere. The ultimate goal of course being a 100% bias free model.
  - version_five 3y ago
    Nah, these are foundation models. Companies want to be able to put in guard rails that are applicable to their application, not start with a model lobotomized according to US tech company values. The censorship is about telling people how to think, like it always is, not for the good of the people using the models.
aaur0 3y ago
I wrote a small piece sharing my views on the topic https://medium.com/@anandandbeyond/rethinking-open-source-for-the-era-of-ai-model-sharing-5b1e770a799d https://medium.com/@anandandbeyond/rethinking-open-source-fo...
- version_five 3y ago
  And posted the link to your subscribers only medium page twice in this thread
  - barbariangrunge 3y ago
    devils advocate: if it wasn't subscribers only, google's crawler would be stealing it right now to train an ai with, ie, illegally creating derivative works based on their articles
    - accrual 3y ago
      Maybe there's a market in the future for "AI-restricted" content. You can send your payment/tokens/etc. to access the content and the user is bound by a no-AI-clause. Because this content would be rare and can't be found via models, it offers a niche market.
    - selcuka 3y ago
      Google's crawler can still crawl it. The regwall is only for humans: https://archive.is/GmVKl https://archive.is/GmVKl Also the article seems to be written by an LLM anywway, so it wouldn't matter in the end.
  - aaur0 3y ago
    I wanted to delete the other link, but hackernews doesn't allow me to do so. Also, I need to figure out why the link is subscriber-only. It was not intentional.
flangola7 3y ago
Blackhats: "Imma pretend I didn't see that"
1vuio0pswjnm7 3y ago
"Mark Dingemanse, a coauthor of this report, had a particularly strong assessment of the Llama 2 model: "Meta using the term `open source' for this is positively misleading: There is no source to be seen, the training data is entirely undocumented, and beyond the glossy charts the technical documentation is really rather poor. We do not know why Meta is so intent on getting everyone into this model, but the history of this company's choices does not inspire confidence. Users beware.""
- aalimov_ 3y ago
  Does anyone have an idea of what (if anything) is being implied by the last two sentences?
  - smoldesu 3y ago
    Bad company scary, I guess. Normally I'm sympathetic to those types, but given how Llama works without a network connection I'm not sure what I have to fear. The "history of this company's choices" has generally been pretty good with AI, so it sounds like speculation on their part.
    - pedrosorio 3y ago
      See my sibling comment. Very unlikely the author has any familiarity with Meta's history of releasing open source software for ML/AI.
  - pedrosorio 3y ago
    > He is an associate professor in Language and Communication at the Centre for Language Studies of Radboud University Nijmegen. Dingemanse obtained a MA degree in African Languages and Cultures at Leiden University in 2006, and a PhD degree in arts in 2011 at Radboud University Nijmegen https://en.wikipedia.org/wiki/Mark_Dingemanse https://en.wikipedia.org/wiki/Mark_Dingemanse This co-author? Given the relevance of their academic background, I'd take their opinion on "the dangers of using LLMs released by the untrustworthy Meta" with a grain of salt.
deleted 3y ago
[deleted]
RcouF1uZ4gsC 3y ago
I think we have forgotten the foundation of open source which was free software which was about empower the user. Instead of all these complicated criteria, there are only 2 criteria that get to the heart of the question: 1. Can you run this on your own hardware 2. Can you use this directly or tweak this as a community to make porn without a bunch of barriers This gets to the heart of the matter: Can you run the software without having to be dependent on the company, and can you modify the software to do something that the company may not want you to do with it. If it is able to pass both tests, it is functionally "open source" By that criteria, ChatGPT is definitely not open but Llama is closer
- hmcq6 3y ago
  You're describing freeware. Open source means the source code is available.
distantsounds 3y ago
Yes, we know. Now can we please stop treating them as developer-friendly tools and more as a hostile theft of intellectual property?
- __loam 3y ago
  It's shocking to me how many people in tech feel completely entitled to intellectual property that took someone years to master a skill to make. But talk about releasing a proprietary codebase and suddenly they want the lawyers involved because that actually threatens their livelihood.
  - idle_zealot 3y ago
    Is that actually a commonly held position? I've seen IP abolishonosts here, and I've seen people argue the merits of proprietary software, but I don't get the impression that those are generally the same people.
    - asynchronous 3y ago
      They’re straw-manning, programmers are the best sharers in the world. Open source software has lead the drive for open source learning and information in general.
      - raincole 3y ago
        https://news.ycombinator.com/item?id=36900844 https://news.ycombinator.com/item?id=36900844 Right here the comment says they prefers Meta to keep the secret sauce as long as it allows they to (inderectly) access the copyrighted material etc. And the replies below are generally positive. So it's not a straw-man, at least in this thread, at all.
        Kamq 3y ago
        It should be noted, that they're explicitly taking this position because they think it will result in more capabilities. Not because of a respect of intellectual property.

143 points by webmaven 3y ago

Previously: https://news.ycombinator.com/item?id=36820122 https://news.ycombinator.com/item?id=36820122 https://news.ycombinator.com/item?id=36806536 https://news.ycombinator.com/item?id=36806536 https://news.ycombinator.com/item?id=36791671 https://news.ycombinator.com/item?id=36791671 https://news.ycombinator.com/item?id=36783019 https://news.ycombinator.com/item?id=36783019

version_five 3y ago
To be fair, the title is the same but this article is really about a study of the openness characteristics of different AI models, which is new. This link had the comparison of different models: https://opening-up-chatgpt.github.io/ https://opening-up-chatgpt.github.io/
- version_five 3y ago
  Also, there's an error in that table, the first model "Bloomz" does not have an open source license, on the HF page for the model it shows a RAIL license which is a restrictive license that limits you to "approved" uses. The license link goes to the github repo where the code is released under Apache 2.0. I didn't review the rest but I'd say it calls the ranking into question.

littlestymaar 3y ago

Can we please stop using terminology related to code for something that's not code?

hprotagonist 3y ago
sometimes it’s hard to see a difference.
aaur0 3y ago
Exactly - I wrote something on similar lines here : https://medium.com/@anandandbeyond/rethinking-open-source-for-the-era-of-ai-model-sharing-5b1e770a799d https://medium.com/@anandandbeyond/rethinking-open-source-fo...
Havoc 3y ago
Agreed it's annoying, but on a certain level also not entirely wrong. In a software 2.0 [0] world the weights are functionally the code in that it is what gets you from input to output. Open weights or something similar would be better though [0] https://karpathy.medium.com/software-2-0-a64152b37c35 https://karpathy.medium.com/software-2-0-a64152b37c35
- charcircuit 3y ago
  Weights are configuration.
  - alanbernstein 3y ago
    Weights almost entirely encapsulate the behavior of the model. Whatever structural analogy you want to use, they are closest in spirit to the core functional algorithm of traditional software.
  - dns_snek 3y ago
    Are symbols in your source file merely a configuration file for the compiler/interpreter?
    - charcircuit 3y ago
      No, but I would consider the command line flags to be configuration.
choxi 3y ago
There’s always been fuzziness between data and code, it’s kind of the core concept of Turing machines.
sashb 3y ago
What is "code" anyways? https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theorem https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...
raincole 3y ago
Except it's code. Weight decides how the model works so it's code. Code is just you telling the computer to work. var a = b * 3 + 2; You're telling me the * 3 + 2 part isn't code and only var a = b is code?

jehb 3y ago

Of note, if you're interested in helping participate in the discussion of what an open source AI would actually look like, the Open Source Initiative is looking for your help: https://opensource.org/deepdive/ https://opensource.org/deepdive/

version_five 3y ago
In there somewhere in there where they are asking for input? It presents itself that way but it seems the only way to "help" is to propose a presentation under their call for speakers: https://sessionize.com/deepdiveai https://sessionize.com/deepdiveai Is there another way to contribute?
- jehb 3y ago
  As I read the process, at this point they're primarily looking for "conversation starters" (my phrase, not theirs) in the form of presentations, as well as attendees to discuss at a number of in-person events this year (I'll be at the one at All Things Open in Raleigh NC in October). I think once they've got something a little more substantial to review, they'll have a more robust "public comment"-type process. I'm sure you could also offer a meta-comment on the process itself.

bick_nyers 3y ago

I personally do not want the companies to release training data (at least for a while) because then it gives people leverage to neuter it. I don't want a sanitized LLM, and I don't have $60M lying around to train my own. Copyrighted material, sexual content, political opinions, throw it all in and release it please! Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no. Maybe I just got added to a list somewhere for having this opinion.

version_five 3y ago
100% agree. People that want training data released mostly just want to find something to attack, it has nothing to do with transparency.
threeseed 3y ago
> reducing bias in the models is a noble goal It's also a necessary goal in order for these models to be more broadly adopted. We've seen too many examples of bias in the training data set manifesting in ways that actively discriminate against people. Which is unethical and in many places illegal. And having copyrighted material and sexual content in your model will simply open you up to lawsuits as is happening right now between authors and OpenAI. Not sure that is a position most startups want to be in.
- bick_nyers 3y ago
  College students don't want to be criminally charged with theft for pirating an $800 college textbook either. I'm not giving business advice, I'm giving humanity and knowledge propagation advice. As I mentioned, reducing bias is good so long as it doesn't introduce more bias elsewhere. The ultimate goal of course being a 100% bias free model.
- version_five 3y ago
  Nah, these are foundation models. Companies want to be able to put in guard rails that are applicable to their application, not start with a model lobotomized according to US tech company values. The censorship is about telling people how to think, like it always is, not for the good of the people using the models.

aaur0 3y ago

I wrote a small piece sharing my views on the topic https://medium.com/@anandandbeyond/rethinking-open-source-for-the-era-of-ai-model-sharing-5b1e770a799d https://medium.com/@anandandbeyond/rethinking-open-source-fo...

version_five 3y ago
And posted the link to your subscribers only medium page twice in this thread
- barbariangrunge 3y ago
  devils advocate: if it wasn't subscribers only, google's crawler would be stealing it right now to train an ai with, ie, illegally creating derivative works based on their articles
  - accrual 3y ago
    Maybe there's a market in the future for "AI-restricted" content. You can send your payment/tokens/etc. to access the content and the user is bound by a no-AI-clause. Because this content would be rare and can't be found via models, it offers a niche market.
  - selcuka 3y ago
    Google's crawler can still crawl it. The regwall is only for humans: https://archive.is/GmVKl https://archive.is/GmVKl Also the article seems to be written by an LLM anywway, so it wouldn't matter in the end.
- aaur0 3y ago
  I wanted to delete the other link, but hackernews doesn't allow me to do so. Also, I need to figure out why the link is subscriber-only. It was not intentional.

flangola7 3y ago

Blackhats: "Imma pretend I didn't see that"

1vuio0pswjnm7 3y ago

"Mark Dingemanse, a coauthor of this report, had a particularly strong assessment of the Llama 2 model: "Meta using the term `open source' for this is positively misleading: There is no source to be seen, the training data is entirely undocumented, and beyond the glossy charts the technical documentation is really rather poor. We do not know why Meta is so intent on getting everyone into this model, but the history of this company's choices does not inspire confidence. Users beware.""

aalimov_ 3y ago
Does anyone have an idea of what (if anything) is being implied by the last two sentences?
- smoldesu 3y ago
  Bad company scary, I guess. Normally I'm sympathetic to those types, but given how Llama works without a network connection I'm not sure what I have to fear. The "history of this company's choices" has generally been pretty good with AI, so it sounds like speculation on their part.
  - pedrosorio 3y ago
    See my sibling comment. Very unlikely the author has any familiarity with Meta's history of releasing open source software for ML/AI.
- pedrosorio 3y ago
  > He is an associate professor in Language and Communication at the Centre for Language Studies of Radboud University Nijmegen. Dingemanse obtained a MA degree in African Languages and Cultures at Leiden University in 2006, and a PhD degree in arts in 2011 at Radboud University Nijmegen https://en.wikipedia.org/wiki/Mark_Dingemanse https://en.wikipedia.org/wiki/Mark_Dingemanse This co-author? Given the relevance of their academic background, I'd take their opinion on "the dangers of using LLMs released by the untrustworthy Meta" with a grain of salt.

deleted 3y ago

[deleted]

RcouF1uZ4gsC 3y ago

I think we have forgotten the foundation of open source which was free software which was about empower the user. Instead of all these complicated criteria, there are only 2 criteria that get to the heart of the question: 1. Can you run this on your own hardware 2. Can you use this directly or tweak this as a community to make porn without a bunch of barriers This gets to the heart of the matter: Can you run the software without having to be dependent on the company, and can you modify the software to do something that the company may not want you to do with it. If it is able to pass both tests, it is functionally "open source" By that criteria, ChatGPT is definitely not open but Llama is closer

hmcq6 3y ago
You're describing freeware. Open source means the source code is available.

distantsounds 3y ago

Yes, we know. Now can we please stop treating them as developer-friendly tools and more as a hostile theft of intellectual property?

__loam 3y ago
It's shocking to me how many people in tech feel completely entitled to intellectual property that took someone years to master a skill to make. But talk about releasing a proprietary codebase and suddenly they want the lawyers involved because that actually threatens their livelihood.
- idle_zealot 3y ago
  Is that actually a commonly held position? I've seen IP abolishonosts here, and I've seen people argue the merits of proprietary software, but I don't get the impression that those are generally the same people.
  - asynchronous 3y ago
    They’re straw-manning, programmers are the best sharers in the world. Open source software has lead the drive for open source learning and information in general.
    - raincole 3y ago
      https://news.ycombinator.com/item?id=36900844 https://news.ycombinator.com/item?id=36900844 Right here the comment says they prefers Meta to keep the secret sauce as long as it allows they to (inderectly) access the copyrighted material etc. And the replies below are generally positive. So it's not a straw-man, at least in this thread, at all.
      - Kamq 3y ago
        It should be noted, that they're explicitly taking this position because they think it will result in more capabilities. Not because of a respect of intellectual property.