4 ms·

I agree with the other poster who said something to the effect that the model is open source and the released weights are, well, open weight. But the distinctio

I agree with the other poster who said something to the effect that the model is open source and the released weights are, well, open weight. But the distinction is so trivial that I think it highlights the stupidity of this whole thing.

koolala 2y ago
Trivial? ."the model is open source and the released weights are, well, open weight." I'm not sure why you don't want open source model data or think open source is trivial. This sentence makes no distinction or point.
Onawa 2y ago
The distinction in this case is decidedly not trivial. Open weights are great, we can fine tune them and mold them to our needs. But we don't have the training code, training data, etc, to be able to reproduce or tweak things at a more fundamental level. "But not everyone is going to spend the money or time to train their own models from scratch!" I hear you say. And there's some truth to that. But if we had truly open source LLMs, then AuroraGPT development would be fast tracked and we would have a fully government funded scientific frontier model instead of only fine-tuning models.
- spunker540 2y ago
  I believe the training and inference code is open sourced as well, just not the training data itself, and I think we all agree, data != source.
  - skeledrew 2y ago
    Actually, in the case of models, training data is functionally ~= code, as the final item cannot be reproduced without it. And, as the open source definition states in its criteria 2: "Intermediate forms such as the output of a preprocessor or translator are not allowed."[0]. The intermediate output in this case being the weights. [0] https://opensource.org/osd https://opensource.org/osd
    - deleted 2y ago
      [deleted]
    - spunker540 2y ago
      I don’t think open source authors have a responsibility to also open source their binary. And if they do happen to provide a binary they aren’t also required to open source their compiler. Usually just the source code. I personally feel if they opened source only the code and not any model weights, it would still be fair to call it open source because “open source” refers to the code itself, not artifacts produced by the code, or possible inputs to the code.
      - skeledrew 2y ago
        It's a matter of intention. The intention of open source is to allow for anyone to be able to sufficiently reproduce some given artifact, at no/reasonable charge (as stated in the definition). Yes, the source code for a binary must be made available, if is to follow the OSD. No, the compiler doesn't need to be open sourced, unless it's an integral part of the program (a compiler would also have to be made available if there's none generally available for the source language). This intention necessarily translates to models. In order to reproduce a program, the source code and a relevant compiler is what is required. In order to reproduce a model, the architecture (source code), training data and initial parameters are required.

koolala 2y ago

Trivial? ."the model is open source and the released weights are, well, open weight." I'm not sure why you don't want open source model data or think open source is trivial. This sentence makes no distinction or point.

Onawa 2y ago

The distinction in this case is decidedly not trivial. Open weights are great, we can fine tune them and mold them to our needs. But we don't have the training code, training data, etc, to be able to reproduce or tweak things at a more fundamental level. "But not everyone is going to spend the money or time to train their own models from scratch!" I hear you say. And there's some truth to that. But if we had truly open source LLMs, then AuroraGPT development would be fast tracked and we would have a fully government funded scientific frontier model instead of only fine-tuning models.

spunker540 2y ago
I believe the training and inference code is open sourced as well, just not the training data itself, and I think we all agree, data != source.
- skeledrew 2y ago
  Actually, in the case of models, training data is functionally ~= code, as the final item cannot be reproduced without it. And, as the open source definition states in its criteria 2: "Intermediate forms such as the output of a preprocessor or translator are not allowed."[0]. The intermediate output in this case being the weights. [0] https://opensource.org/osd https://opensource.org/osd
  - deleted 2y ago
    [deleted]
  - spunker540 2y ago
    I don’t think open source authors have a responsibility to also open source their binary. And if they do happen to provide a binary they aren’t also required to open source their compiler. Usually just the source code. I personally feel if they opened source only the code and not any model weights, it would still be fair to call it open source because “open source” refers to the code itself, not artifacts produced by the code, or possible inputs to the code.
    - skeledrew 2y ago
      It's a matter of intention. The intention of open source is to allow for anyone to be able to sufficiently reproduce some given artifact, at no/reasonable charge (as stated in the definition). Yes, the source code for a binary must be made available, if is to follow the OSD. No, the compiler doesn't need to be open sourced, unless it's an integral part of the program (a compiler would also have to be made available if there's none generally available for the source language). This intention necessarily translates to models. In order to reproduce a program, the source code and a relevant compiler is what is required. In order to reproduce a model, the architecture (source code), training data and initial parameters are required.