6 ms·

Why is this being called an open source model? This is a proprietary model that has been leaked on the internet, and will remain so until Mistral releases it of

Why is this being called an open source model? This is a proprietary model that has been leaked on the internet, and will remain so until Mistral releases it officially. Like Llama 1 they won't care about personal use, but no corp is going to touch this.

codetrotter 3y ago
Presumably because it is planned to be released as open source
- jallmann 3y ago
  Open source is really about reproducibility. Most of these model releases are better described as "open weights" because we don't know how exactly they were trained.
  - andy99 3y ago
    No, open source is about software freedom, see debian free software guidelines from which the "open source definition" derives. https://wiki.debian.org/DebianFreeSoftwareGuidelines https://wiki.debian.org/DebianFreeSoftwareGuidelines Between these and FSF you've got pretty much all the accepted pontificating about free / open source software. Reproducibility is not mentioned because it's not really a consideration for software. Model weights aren't software so there's not an automatic correspondence between the freedoms, but the essential one you might think you need the training data for is freedom to modify and inspect the source. Modification is fine tuning which you're free to do if you have the weights. And the model weights + code fully define a system that can be interrogated to give a practitioner relevant info about how the model works (within our understanding) that the training data isn't needed or relevant for. I don't see that any freedom on use or inspection is violated by not having the data. It could be nice to have it of course, but it's more about using it to learn how, not exercising any freedom. Incidentally, the big freedom that's usually violated is freedom of discrimination against field of endeavor. LLAMA et al list uses and industries they restrict from using them and because of that are not "open".
    - JumpCrisscross 3y ago
      > open source is about software freedom, see debian free software guidelines from which the "open source definition" The Open Software Foundation ironically screwed the pooch on this one. Open source commonly means source available, more of less. Free software, as in “'free speech,' not as in 'free beer',” is the cumbersome construction for what open source aspired to mean [1]. [1] https://www.gnu.org/philosophy/free-sw.en.html https://www.gnu.org/philosophy/free-sw.en.html
      - andy99 3y ago
        Why do you say they screwed up? OSI definition is pretty clear. I think the naming is a challenge because in English "open" gives the impression that the key point is that you can see it, as opposed to anything about freedom. Is that what you mean? I have heard it said that open source is sort of a "commercial friendly" version of free software that de-emphasizes user freedom. I think some groups push for that (like Meta is trying to redefine what open source means wrt AI weights). But the OSI defined freedoms basically match what FSF pushes.
        JumpCrisscross 3y ago
        > Why do you say they screwed up? OSI definition is pretty clear They didn't screw up, they screwed the pooch on open source != source available. The Open Group's members--from IBM to Huawei [1]--started calling the latter open source, which set a precedent that's stuck. [1] https://en.wikipedia.org/wiki/The_Open_Group#Member_Forums_and_Consortia https://en.wikipedia.org/wiki/The_Open_Group#Member_Forums_a...
    - monocasa 3y ago
      > Model weights aren't software so there's not an automatic correspondence between the freedoms, but the essential one you might think you need the training data for is freedom to modify and inspect the source. Models aren't just the weights, but also the list of operations to perform using those weights. I haven't heard a good definition that allows for neural network models to not be software, but allows any other table lookup heavy signal processing algorithm to be software. > Modification is fine tuning which you're free to do if you have the weights. And the model weights + code fully define a system that can be interrogated to give a practitioner relevant info about how the model works (within our understanding) that the training data isn't needed or relevant for. I don't see that any freedom on use or inspection is violated by not having the data. Mistral wouldn't constrain themselves to fine tuning if they have a big enough change, they would go back to their build pipeline. This argument sounds a lot like 'there's nothing stopping you from patching the binary, so that's basically as good as source'.
  - dizhn 3y ago
    Would you point to some completely open models if they exist?
- monocasa 3y ago
  I don't think so, they're earlier release they called "open source" but never released anything other than the binary.
- 2devnull 3y ago
  People really abuse this term. Like using “natural” to market stuff. But arsenic is “all natural”! It’s “open source” so it must be beneficial. “rm -rf c” is “open source” too
  - TeMPOraL 3y ago
    > Like using “natural” to market stuff. Also, because people are idiotically afraid of E-numbers, some manufacturers figured they can find whatever fruit or bean is naturally rich in the relevant E-compound, and use that in the process, allowing them to replace E-whatever with ${cute plant name} in the ingredient list (at some loss of process efficiency).
GaggiX 3y ago
Well I remember Tencent using the leaked NovelAI model in "Different Dimension Me".
colordrops 3y ago
Where do you get the idea that corps aren't touching LLama...
- int_19h 3y ago
  It's already on Azure, and has been for several months: https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/introducing-llama-2-on-azure/ba-p/3881233 https://techcommunity.microsoft.com/t5/ai-machine-learning-b...
snovv_crash 3y ago
The weights aren't created by humans so they aren't copyrightable - at least in theory.
- monocasa 3y ago
  That has nothing to do with whether it's open source or not. Software wasn't for sure copyrightable before 1976 in the US, but there was plenty of closed source code that simply only ever distributed the binaries.
  - epistasis 3y ago
    It's open source in the sense that you can take the existing weights, apply any one of dozens of techniques for adapting the model to your data, and anybody you give your model to can do the same. This right isn't protected by copyright, it's protected by the lack of copyright, and how executables are directly modifiable. So the AGPL won't be possible, but it's a lot like permissive licenses, without attribution.
    - monocasa 3y ago
      That's like saying an executable that I never gave out the source to is "open source" because there's nothing stopping you from binary patching it. Also, the license to the model assumes copyright (Apache 2) and works by granting you permissions based on that copyright.
      - epistasis 3y ago
        It's very very different from patching a binary, because these models don't work like a simple compilation of understandable source code into incomprehensible weights. The source training data is even less comprehensible and useful than its encapsulation into weights. A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. It's usually far more useful than the source training data, because the work has been done.

codetrotter 3y ago

Presumably because it is planned to be released as open source

jallmann 3y ago
Open source is really about reproducibility. Most of these model releases are better described as "open weights" because we don't know how exactly they were trained.
- andy99 3y ago
  No, open source is about software freedom, see debian free software guidelines from which the "open source definition" derives. https://wiki.debian.org/DebianFreeSoftwareGuidelines https://wiki.debian.org/DebianFreeSoftwareGuidelines Between these and FSF you've got pretty much all the accepted pontificating about free / open source software. Reproducibility is not mentioned because it's not really a consideration for software. Model weights aren't software so there's not an automatic correspondence between the freedoms, but the essential one you might think you need the training data for is freedom to modify and inspect the source. Modification is fine tuning which you're free to do if you have the weights. And the model weights + code fully define a system that can be interrogated to give a practitioner relevant info about how the model works (within our understanding) that the training data isn't needed or relevant for. I don't see that any freedom on use or inspection is violated by not having the data. It could be nice to have it of course, but it's more about using it to learn how, not exercising any freedom. Incidentally, the big freedom that's usually violated is freedom of discrimination against field of endeavor. LLAMA et al list uses and industries they restrict from using them and because of that are not "open".
  - JumpCrisscross 3y ago
    > open source is about software freedom, see debian free software guidelines from which the "open source definition" The Open Software Foundation ironically screwed the pooch on this one. Open source commonly means source available, more of less. Free software, as in “'free speech,' not as in 'free beer',” is the cumbersome construction for what open source aspired to mean [1]. [1] https://www.gnu.org/philosophy/free-sw.en.html https://www.gnu.org/philosophy/free-sw.en.html
    - andy99 3y ago
      Why do you say they screwed up? OSI definition is pretty clear. I think the naming is a challenge because in English "open" gives the impression that the key point is that you can see it, as opposed to anything about freedom. Is that what you mean? I have heard it said that open source is sort of a "commercial friendly" version of free software that de-emphasizes user freedom. I think some groups push for that (like Meta is trying to redefine what open source means wrt AI weights). But the OSI defined freedoms basically match what FSF pushes.
      - JumpCrisscross 3y ago
        > Why do you say they screwed up? OSI definition is pretty clear They didn't screw up, they screwed the pooch on open source != source available. The Open Group's members--from IBM to Huawei [1]--started calling the latter open source, which set a precedent that's stuck. [1] https://en.wikipedia.org/wiki/The_Open_Group#Member_Forums_and_Consortia https://en.wikipedia.org/wiki/The_Open_Group#Member_Forums_a...
  - monocasa 3y ago
    > Model weights aren't software so there's not an automatic correspondence between the freedoms, but the essential one you might think you need the training data for is freedom to modify and inspect the source. Models aren't just the weights, but also the list of operations to perform using those weights. I haven't heard a good definition that allows for neural network models to not be software, but allows any other table lookup heavy signal processing algorithm to be software. > Modification is fine tuning which you're free to do if you have the weights. And the model weights + code fully define a system that can be interrogated to give a practitioner relevant info about how the model works (within our understanding) that the training data isn't needed or relevant for. I don't see that any freedom on use or inspection is violated by not having the data. Mistral wouldn't constrain themselves to fine tuning if they have a big enough change, they would go back to their build pipeline. This argument sounds a lot like 'there's nothing stopping you from patching the binary, so that's basically as good as source'.
- dizhn 3y ago
  Would you point to some completely open models if they exist?
monocasa 3y ago
I don't think so, they're earlier release they called "open source" but never released anything other than the binary.
2devnull 3y ago
People really abuse this term. Like using “natural” to market stuff. But arsenic is “all natural”! It’s “open source” so it must be beneficial. “rm -rf c” is “open source” too
- TeMPOraL 3y ago
  > Like using “natural” to market stuff. Also, because people are idiotically afraid of E-numbers, some manufacturers figured they can find whatever fruit or bean is naturally rich in the relevant E-compound, and use that in the process, allowing them to replace E-whatever with ${cute plant name} in the ingredient list (at some loss of process efficiency).

GaggiX 3y ago

Well I remember Tencent using the leaked NovelAI model in "Different Dimension Me".

colordrops 3y ago

Where do you get the idea that corps aren't touching LLama...

int_19h 3y ago
It's already on Azure, and has been for several months: https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/introducing-llama-2-on-azure/ba-p/3881233 https://techcommunity.microsoft.com/t5/ai-machine-learning-b...

snovv_crash 3y ago

The weights aren't created by humans so they aren't copyrightable - at least in theory.

monocasa 3y ago
That has nothing to do with whether it's open source or not. Software wasn't for sure copyrightable before 1976 in the US, but there was plenty of closed source code that simply only ever distributed the binaries.
- epistasis 3y ago
  It's open source in the sense that you can take the existing weights, apply any one of dozens of techniques for adapting the model to your data, and anybody you give your model to can do the same. This right isn't protected by copyright, it's protected by the lack of copyright, and how executables are directly modifiable. So the AGPL won't be possible, but it's a lot like permissive licenses, without attribution.
  - monocasa 3y ago
    That's like saying an executable that I never gave out the source to is "open source" because there's nothing stopping you from binary patching it. Also, the license to the model assumes copyright (Apache 2) and works by granting you permissions based on that copyright.
    - epistasis 3y ago
      It's very very different from patching a binary, because these models don't work like a simple compilation of understandable source code into incomprehensible weights. The source training data is even less comprehensible and useful than its encapsulation into weights. A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. It's usually far more useful than the source training data, because the work has been done.