5 ms·

I agree… This is not open source. If you publish a wrapper that represent just 0.01% of your code and your product and hide all the rest as closed source that i

by nazka 4y ago

I agree… This is not open source. If you publish a wrapper that represent just 0.01% of your code and your product and hide all the rest as closed source that is considered open source now? Then any closed source providing an API become open source with this bogus definition. I guess now Windows is open source too, just miss the 99.99% of the rest of the code. It’s absurd and it damages the open source world and its real definition.

p1esk 4y ago
No, this is open source. The model code and everything needed to verify their results is provided. No one promised the training code, and the training code is not needed to reproduce the results.
- kaoD 4y ago
  Going back to my original comment: if I release a giant binary blob (say, a Windows image) and an open source runtime to run it (say, VirtualBox), does that mean I can say Windows is open source? I don't think so, and to me it seems that this situation is a perfect parallel. This is FSF's freedom 1: "The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this." By that definition I don't think you can say this model is open source then. You could say the inference code is open source, and that's technically true (and the only thing the repo claims TBH), but calling this open source AI model is misleading. Open source inference code is outright uninteresting. It's a few hundred lines of glue code to run a giant binary blob which does the heavy lifting, i.e. the actual computing, i.e. the actual software, i.e. the software whose source we're actually interested in. EDIT: Actually I just noticed they didn't even open source checkpoints+tokenizer.
  - p1esk 4y ago
    Did you even look at the repo before commenting? The model code is literally open source: https://github.com/facebookresearch/llama/blob/main/llama/model.py https://github.com/facebookresearch/llama/blob/main/llama/mo... Even the checkpoints are provided - for free! All you have to do is ask. Someone at Facebook spent a ton of money to train a state of the art model, open sourced the code and even provides checkpoints free of charge, and you still complain? The level of entitlement is off the charts…
    - kaoD 4y ago
      > All you have to do is ask. That makes it literally not open source. Can I redistribute it freely? I don't think so, I might be wrong. If not, it's not open source. > and you still complain? But I don't complain about Facebook releasing it in whichever terms they want. I complain that people call it open source when it clearly isn't. > The level of entitlement is off the charts… How is pointing that it's not open source entitlement?
      - p1esk 4y ago
        Again, I have to point you to the repo code. Here’s the licence: https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE The model code is literally open source. Provided and licensed.
        throwaway1851 4y ago
        The disconnect here is that the model code isn’t the important part. If you read the paper, LLaMA is all about the training data and the training process (which required a ”massive quantity” of compute, to quote the paper). Without releasing the significant part of their work, what are they really releasing?
        p1esk 4y ago
        Exactly - massive amount of compute required to reproduce the training process, and that’s why almost no one is going to try training this model from scratch. Most people will use this model as is, and a small number of people will finetune it on their own data - those people most likely already have their own training scripts, because finetuning is much easier than training from scratch and requires different hyper parameters. I don’t know why they haven’t released the training code - I agree it would be nice if they did - but the important thing here is they created something valuable, open sourced it, and even provided model weights - for free. Let’s appreciate it. Even if they did publish their training code - that’s not enough to train the model. You also need their dataset. Would you still claim “the model is not open source” because the dataset is not available? Bottom line: the model has been open sourced. The training code hasn’t - and isn’t needed by most users of this model.

by nazka 4y ago

p1esk 4y ago

No, this is open source. The model code and everything needed to verify their results is provided. No one promised the training code, and the training code is not needed to reproduce the results.

kaoD 4y ago
Going back to my original comment: if I release a giant binary blob (say, a Windows image) and an open source runtime to run it (say, VirtualBox), does that mean I can say Windows is open source? I don't think so, and to me it seems that this situation is a perfect parallel. This is FSF's freedom 1: "The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this." By that definition I don't think you can say this model is open source then. You could say the inference code is open source, and that's technically true (and the only thing the repo claims TBH), but calling this open source AI model is misleading. Open source inference code is outright uninteresting. It's a few hundred lines of glue code to run a giant binary blob which does the heavy lifting, i.e. the actual computing, i.e. the actual software, i.e. the software whose source we're actually interested in. EDIT: Actually I just noticed they didn't even open source checkpoints+tokenizer.
- p1esk 4y ago
  Did you even look at the repo before commenting? The model code is literally open source: https://github.com/facebookresearch/llama/blob/main/llama/model.py https://github.com/facebookresearch/llama/blob/main/llama/mo... Even the checkpoints are provided - for free! All you have to do is ask. Someone at Facebook spent a ton of money to train a state of the art model, open sourced the code and even provides checkpoints free of charge, and you still complain? The level of entitlement is off the charts…
  - kaoD 4y ago
    > All you have to do is ask. That makes it literally not open source. Can I redistribute it freely? I don't think so, I might be wrong. If not, it's not open source. > and you still complain? But I don't complain about Facebook releasing it in whichever terms they want. I complain that people call it open source when it clearly isn't. > The level of entitlement is off the charts… How is pointing that it's not open source entitlement?
    - p1esk 4y ago
      Again, I have to point you to the repo code. Here’s the licence: https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE The model code is literally open source. Provided and licensed.
      - throwaway1851 4y ago
        The disconnect here is that the model code isn’t the important part. If you read the paper, LLaMA is all about the training data and the training process (which required a ”massive quantity” of compute, to quote the paper). Without releasing the significant part of their work, what are they really releasing?
        p1esk 4y ago
        Exactly - massive amount of compute required to reproduce the training process, and that’s why almost no one is going to try training this model from scratch. Most people will use this model as is, and a small number of people will finetune it on their own data - those people most likely already have their own training scripts, because finetuning is much easier than training from scratch and requires different hyper parameters. I don’t know why they haven’t released the training code - I agree it would be nice if they did - but the important thing here is they created something valuable, open sourced it, and even provided model weights - for free. Let’s appreciate it. Even if they did publish their training code - that’s not enough to train the model. You also need their dataset. Would you still claim “the model is not open source” because the dataset is not available? Bottom line: the model has been open sourced. The training code hasn’t - and isn’t needed by most users of this model.