5 ms·

I find the snyde remarks around open source in the paper and announcement rather off putting. As the ecosystem evolves, we urge the corporate AI community to m

by artninja1988 3y ago

I find the snyde remarks around open source in the paper and announcement rather off putting. As the ecosystem evolves, we urge the corporate AI community to move beyond demanding to be taken seriously as a player in open source for models that are not actually open, and avoid preaching with a PR statement that can be interpreted as uniformed at best or malicious at worst.

silentsanctuary 3y ago
Which remarks are you referring to?
- artninja1988 3y ago
  The synde remarks at metas llama license that doesn't allow companies with 700 million monthly active users to use it, while this model also doesn't have a really 'open' license itself and also this paragraph: >As the ecosystem evolves, we urge the wider AI community to move beyond simplistic ’open vs. closed’ debates, and avoid either exaggerating or minimising potential harms, as we believe a nuanced, collaborative approach to risks and benefits is essential. At Google DeepMind we’re committed to developing high-quality evaluations and invite the community to join us in this effort for a deeper understanding of AI systems.
  - tomComb 3y ago
    Well, given that that restriction added to the meta-llama license is aimed at Google, is petty, and goes against open source norms, I think it’s reasonable that they should feel this way about it.
  - lordswork 3y ago
    How is this a snide remark? It's factual and prevented their team from benchmarking against Llama 2.
  - trisfromgoogle 3y ago
    Quick question -- can you tell me where you got that quote? It's not in the main blog or any of the launch communications that I can see.
    - artninja1988 3y ago
      The quote is from the technical report https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf https://storage.googleapis.com/deepmind-media/gemma/gemma-re...
      - trisfromgoogle 3y ago
        Ah, thanks for clarifying! It's a good flag, though I wouldn't classify it as a snide comment personally. I'd be interested in hearing what you find snide or offensive about it -- do you think we shouldn't be trying to bring the whole community along for evals/safety/etc, regardless of open/closed?
trisfromgoogle 3y ago
It would be great to understand what you mean by this -- we have a deep love for open source and the open developer ecosystem. Our open source team also released a blog today describing the rationale and approach for open models and continuing AI releases in the open ecosystem: https://opensource.googleblog.com/2024/02/building-open-models-responsibly-gemini-era.html https://opensource.googleblog.com/2024/02/building-open-mode... Thoughts and feedback welcome, as always.
- artninja1988 3y ago
  The statement on you not being able to use LLaMA 2 to benchmark is also false and highly misleading see https://x.com/BlancheMinerva/status/1760302091166241163?s=20 https://x.com/BlancheMinerva/status/1760302091166241163?s=20
  - lordswork 3y ago
    If, on the Llama 2 version release date, the monthly active users [...] is greater than 700 million monthly active users [...] you are not authorized to exercise any of the rights under this Agreement I would guess this is Google being careful to not be burned by this lame clause in the Llama 2 license.
    - not2b 3y ago
      It's aimed directly at them (and OpenAI and Microsoft) so they have to honor it if they don't want a legal battle. But there's nothing stopping others from doing benchmarking.
  - lucubratory 3y ago
    For the reference of people seeing this now: The tweet that person linked has now been deleted and the scientist who tweeted it has acknowledged they were wrong and retracted their claim, as all good scientists should.
- mrob 3y ago
  If you truly love Open Source, you should update the the language you use to describe your models so it doesn't mislead people into thinking it has something to do with Open Source. Despite being called "Open", the Gemma weights are released under a license that is incompatible with the Open Source Definition. It has more in common with Source-Available Software, and as such it should be called a "Weights-Available Model".
jppittma 3y ago
Working at google is like this, where no matter how much you try to do the right thing you're always under attack.

silentsanctuary 3y ago

Which remarks are you referring to?

artninja1988 3y ago
The synde remarks at metas llama license that doesn't allow companies with 700 million monthly active users to use it, while this model also doesn't have a really 'open' license itself and also this paragraph: >As the ecosystem evolves, we urge the wider AI community to move beyond simplistic ’open vs. closed’ debates, and avoid either exaggerating or minimising potential harms, as we believe a nuanced, collaborative approach to risks and benefits is essential. At Google DeepMind we’re committed to developing high-quality evaluations and invite the community to join us in this effort for a deeper understanding of AI systems.
- tomComb 3y ago
  Well, given that that restriction added to the meta-llama license is aimed at Google, is petty, and goes against open source norms, I think it’s reasonable that they should feel this way about it.
- lordswork 3y ago
  How is this a snide remark? It's factual and prevented their team from benchmarking against Llama 2.
- trisfromgoogle 3y ago
  Quick question -- can you tell me where you got that quote? It's not in the main blog or any of the launch communications that I can see.
  - artninja1988 3y ago
    The quote is from the technical report https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf https://storage.googleapis.com/deepmind-media/gemma/gemma-re...
    - trisfromgoogle 3y ago
      Ah, thanks for clarifying! It's a good flag, though I wouldn't classify it as a snide comment personally. I'd be interested in hearing what you find snide or offensive about it -- do you think we shouldn't be trying to bring the whole community along for evals/safety/etc, regardless of open/closed?

trisfromgoogle 3y ago

It would be great to understand what you mean by this -- we have a deep love for open source and the open developer ecosystem. Our open source team also released a blog today describing the rationale and approach for open models and continuing AI releases in the open ecosystem: https://opensource.googleblog.com/2024/02/building-open-models-responsibly-gemini-era.html https://opensource.googleblog.com/2024/02/building-open-mode... Thoughts and feedback welcome, as always.

artninja1988 3y ago
The statement on you not being able to use LLaMA 2 to benchmark is also false and highly misleading see https://x.com/BlancheMinerva/status/1760302091166241163?s=20 https://x.com/BlancheMinerva/status/1760302091166241163?s=20
- lordswork 3y ago
  If, on the Llama 2 version release date, the monthly active users [...] is greater than 700 million monthly active users [...] you are not authorized to exercise any of the rights under this Agreement I would guess this is Google being careful to not be burned by this lame clause in the Llama 2 license.
  - not2b 3y ago
    It's aimed directly at them (and OpenAI and Microsoft) so they have to honor it if they don't want a legal battle. But there's nothing stopping others from doing benchmarking.
- lucubratory 3y ago
  For the reference of people seeing this now: The tweet that person linked has now been deleted and the scientist who tweeted it has acknowledged they were wrong and retracted their claim, as all good scientists should.
mrob 3y ago
If you truly love Open Source, you should update the the language you use to describe your models so it doesn't mislead people into thinking it has something to do with Open Source. Despite being called "Open", the Gemma weights are released under a license that is incompatible with the Open Source Definition. It has more in common with Source-Available Software, and as such it should be called a "Weights-Available Model".

jppittma 3y ago

Working at google is like this, where no matter how much you try to do the right thing you're always under attack.