5 ms·

why does Yann LeCun advertise that it is open source then?

by hackernewds 4y ago

mertd 4y ago
I assume you could train one like this provided that you bring your own training data and computing resources.
p1esk 4y ago
Because it is open source: https://github.com/facebookresearch/llama https://github.com/facebookresearch/llama
- kaoD 4y ago
  It feels like open source as a term became obsolete(-ish). Is it open source (in spirit) if only the inference code is open source? I mean, technically it might be, but not being able to train it myself is basically against FSF's freedom 1 if you consider the model the software. That repo is basically releasing a binary (the weights) and an open source runtime to run them.
  - nazka 4y ago
    I agree… This is not open source. If you publish a wrapper that represent just 0.01% of your code and your product and hide all the rest as closed source that is considered open source now? Then any closed source providing an API become open source with this bogus definition. I guess now Windows is open source too, just miss the 99.99% of the rest of the code. It’s absurd and it damages the open source world and its real definition.
    - p1esk 4y ago
      No, this is open source. The model code and everything needed to verify their results is provided. No one promised the training code, and the training code is not needed to reproduce the results.
      - kaoD 4y ago
        Going back to my original comment: if I release a giant binary blob (say, a Windows image) and an open source runtime to run it (say, VirtualBox), does that mean I can say Windows is open source? I don't think so, and to me it seems that this situation is a perfect parallel. This is FSF's freedom 1: "The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this." By that definition I don't think you can say this model is open source then. You could say the inference code is open source, and that's technically true (and the only thing the repo claims TBH), but calling this open source AI model is misleading. Open source inference code is outright uninteresting. It's a few hundred lines of glue code to run a giant binary blob which does the heavy lifting, i.e. the actual computing, i.e. the actual software, i.e. the software whose source we're actually interested in. EDIT: Actually I just noticed they didn't even open source checkpoints+tokenizer.
        p1esk 4y ago
        Did you even look at the repo before commenting? The model code is literally open source: https://github.com/facebookresearch/llama/blob/main/llama/model.py https://github.com/facebookresearch/llama/blob/main/llama/mo... Even the checkpoints are provided - for free! All you have to do is ask. Someone at Facebook spent a ton of money to train a state of the art model, open sourced the code and even provides checkpoints free of charge, and you still complain? The level of entitlement is off the charts…
        kaoD 4y ago
        > All you have to do is ask. That makes it literally not open source. Can I redistribute it freely? I don't think so, I might be wrong. If not, it's not open source. > and you still complain? But I don't complain about Facebook releasing it in whichever terms they want. I complain that people call it open source when it clearly isn't. > The level of entitlement is off the charts… How is pointing that it's not open source entitlement?
        4y ago
  - homarp 4y ago
    Would you say that Doom is not open source because the WAD are not freely available?
    - kaoD 4y ago
      That's what I meant when I said the open source term kinda got obsolete. With AI the model is the program. I don't even have the training code or the checkpoint so I cannot modify it as the freedom 1 requires. As I see it the weights are more similar to LLVM IR. It got "programmed" by gradient descent. At most you can say the inference code is open source (see sibling thread).
      - p1esk 4y ago
        The confusion here is about what constitutes a “model” - a model source code, model parameters, code to train the model, dataset used to train the model, code used to evaluate the performance of the model? I would probably say that the first two constitute a model. I agree that only the first one (model code) is strictly open source, while the second (model parameters) is not, because it’s not available to everyone.

why does Yann LeCun advertise that it is open source then?

mertd 4y ago

I assume you could train one like this provided that you bring your own training data and computing resources.

p1esk 4y ago

Because it is open source: https://github.com/facebookresearch/llama https://github.com/facebookresearch/llama

kaoD 4y ago
It feels like open source as a term became obsolete(-ish). Is it open source (in spirit) if only the inference code is open source? I mean, technically it might be, but not being able to train it myself is basically against FSF's freedom 1 if you consider the model the software. That repo is basically releasing a binary (the weights) and an open source runtime to run them.
- nazka 4y ago
  I agree… This is not open source. If you publish a wrapper that represent just 0.01% of your code and your product and hide all the rest as closed source that is considered open source now? Then any closed source providing an API become open source with this bogus definition. I guess now Windows is open source too, just miss the 99.99% of the rest of the code. It’s absurd and it damages the open source world and its real definition.
  - p1esk 4y ago
    No, this is open source. The model code and everything needed to verify their results is provided. No one promised the training code, and the training code is not needed to reproduce the results.
    - kaoD 4y ago
      Going back to my original comment: if I release a giant binary blob (say, a Windows image) and an open source runtime to run it (say, VirtualBox), does that mean I can say Windows is open source? I don't think so, and to me it seems that this situation is a perfect parallel. This is FSF's freedom 1: "The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this." By that definition I don't think you can say this model is open source then. You could say the inference code is open source, and that's technically true (and the only thing the repo claims TBH), but calling this open source AI model is misleading. Open source inference code is outright uninteresting. It's a few hundred lines of glue code to run a giant binary blob which does the heavy lifting, i.e. the actual computing, i.e. the actual software, i.e. the software whose source we're actually interested in. EDIT: Actually I just noticed they didn't even open source checkpoints+tokenizer.
      - p1esk 4y ago
        Did you even look at the repo before commenting? The model code is literally open source: https://github.com/facebookresearch/llama/blob/main/llama/model.py https://github.com/facebookresearch/llama/blob/main/llama/mo... Even the checkpoints are provided - for free! All you have to do is ask. Someone at Facebook spent a ton of money to train a state of the art model, open sourced the code and even provides checkpoints free of charge, and you still complain? The level of entitlement is off the charts…
        kaoD 4y ago
        > All you have to do is ask. That makes it literally not open source. Can I redistribute it freely? I don't think so, I might be wrong. If not, it's not open source. > and you still complain? But I don't complain about Facebook releasing it in whichever terms they want. I complain that people call it open source when it clearly isn't. > The level of entitlement is off the charts… How is pointing that it's not open source entitlement?
        4y ago
- homarp 4y ago
  Would you say that Doom is not open source because the WAD are not freely available?
  - kaoD 4y ago
    That's what I meant when I said the open source term kinda got obsolete. With AI the model is the program. I don't even have the training code or the checkpoint so I cannot modify it as the freedom 1 requires. As I see it the weights are more similar to LLVM IR. It got "programmed" by gradient descent. At most you can say the inference code is open source (see sibling thread).
    - p1esk 4y ago
      The confusion here is about what constitutes a “model” - a model source code, model parameters, code to train the model, dataset used to train the model, code used to evaluate the performance of the model? I would probably say that the first two constitute a model. I agree that only the first one (model code) is strictly open source, while the second (model parameters) is not, because it’s not available to everyone.