3 ms·

That has nothing to do with whether it's open source or not. Software wasn't for sure copyrightable before 1976 in the US, but there was plenty of closed sourc

That has nothing to do with whether it's open source or not. Software wasn't for sure copyrightable before 1976 in the US, but there was plenty of closed source code that simply only ever distributed the binaries.

epistasis 3y ago
It's open source in the sense that you can take the existing weights, apply any one of dozens of techniques for adapting the model to your data, and anybody you give your model to can do the same. This right isn't protected by copyright, it's protected by the lack of copyright, and how executables are directly modifiable. So the AGPL won't be possible, but it's a lot like permissive licenses, without attribution.
- monocasa 3y ago
  That's like saying an executable that I never gave out the source to is "open source" because there's nothing stopping you from binary patching it. Also, the license to the model assumes copyright (Apache 2) and works by granting you permissions based on that copyright.
  - epistasis 3y ago
    It's very very different from patching a binary, because these models don't work like a simple compilation of understandable source code into incomprehensible weights. The source training data is even less comprehensible and useful than its encapsulation into weights. A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. It's usually far more useful than the source training data, because the work has been done.
    - monocasa 3y ago
      > It's very very different from patching a binary, because these models don't work like a simple compilation of understandable source code into incomprehensible weights. The source training data is even less comprehensible and useful than its encapsulation into weights. Tons of binary patching works that way. For instance from the gameshark days it was relatively common for a patch that worked for unknown reasons, but who's discovery was tooling assisted and simply displayed a desired effect (and commonly a lot of undesired effects that weren't clear). > A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. None of that changes whether it's open source or not. I guarantee you that mistral has some code (and a lot o data) laying around that created this model. > It's usually far more useful than the source training data, because the work has been done. For certain operations maybe. I guarantee that Mistral wouldn't constrain themselves to fine-tuning if they had a major change to make to the model.
      - epistasis 3y ago
        No, patching binaries is not really similar at all, because it's viewed as an inferior method to having the source code and modifying that. With these large models, the weights really are quite useful objects in themselves, the very starting points of future modifications, and importantly the desired points of modifications Practioners in the field call these models "open source" and write open source software to run them and use them to power open source software.
        monocasa 3y ago
        > No, patching binaries is not really similar at all, because it's viewed as an inferior method to having the source code and modifying that. > With these large models, the weights really are quite useful objects in themselves, the very starting points of future modifications, and importantly the desired points of modifications For fine tuning. You go back to the training pipeline to make major changes. > Practioners in the field call these models "open source" Some do. In contrast to the accepted definitions of open source. > and write open source software to run them and use them to power open source software. There's plenty of open source code to run closed source binaries.
- sroussey 3y ago
  It’s very likely protected by EU copyright.

epistasis 3y ago

It's open source in the sense that you can take the existing weights, apply any one of dozens of techniques for adapting the model to your data, and anybody you give your model to can do the same. This right isn't protected by copyright, it's protected by the lack of copyright, and how executables are directly modifiable. So the AGPL won't be possible, but it's a lot like permissive licenses, without attribution.

monocasa 3y ago
That's like saying an executable that I never gave out the source to is "open source" because there's nothing stopping you from binary patching it. Also, the license to the model assumes copyright (Apache 2) and works by granting you permissions based on that copyright.
- epistasis 3y ago
  It's very very different from patching a binary, because these models don't work like a simple compilation of understandable source code into incomprehensible weights. The source training data is even less comprehensible and useful than its encapsulation into weights. A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. It's usually far more useful than the source training data, because the work has been done.
  - monocasa 3y ago
    > It's very very different from patching a binary, because these models don't work like a simple compilation of understandable source code into incomprehensible weights. The source training data is even less comprehensible and useful than its encapsulation into weights. Tons of binary patching works that way. For instance from the gameshark days it was relatively common for a patch that worked for unknown reasons, but who's discovery was tooling assisted and simply displayed a desired effect (and commonly a lot of undesired effects that weren't clear). > A set of weights is a pre-trained local minima in the model space is a both executable and modifiable. None of that changes whether it's open source or not. I guarantee you that mistral has some code (and a lot o data) laying around that created this model. > It's usually far more useful than the source training data, because the work has been done. For certain operations maybe. I guarantee that Mistral wouldn't constrain themselves to fine-tuning if they had a major change to make to the model.
    - epistasis 3y ago
      No, patching binaries is not really similar at all, because it's viewed as an inferior method to having the source code and modifying that. With these large models, the weights really are quite useful objects in themselves, the very starting points of future modifications, and importantly the desired points of modifications Practioners in the field call these models "open source" and write open source software to run them and use them to power open source software.
      - monocasa 3y ago
        > No, patching binaries is not really similar at all, because it's viewed as an inferior method to having the source code and modifying that. > With these large models, the weights really are quite useful objects in themselves, the very starting points of future modifications, and importantly the desired points of modifications For fine tuning. You go back to the training pipeline to make major changes. > Practioners in the field call these models "open source" Some do. In contrast to the accepted definitions of open source. > and write open source software to run them and use them to power open source software. There's plenty of open source code to run closed source binaries.
sroussey 3y ago
It’s very likely protected by EU copyright.