6 ms·

Look, absolutely zero people in the world should trust any tech company when they say they care about or will keep commitments to the open-source ecosystem in a

by advael 2y ago

Look, absolutely zero people in the world should trust any tech company when they say they care about or will keep commitments to the open-source ecosystem in any capacity. Nevertheless, it is occasionally strategic for them to do so, and there can be ancillary benefits for said ecosystem in those moments where this is the best play for them to harm their competitors For now, Meta seems to release Llama models in ways that don't significantly lock people into their infrastructure. If that ever stops being the case, you should fork rather than trust their judgment. I say this knowing full well that most of the internet is on AWS or GCP, most brick and mortar businesses use Windows, and carrying a proprietary smartphone is essentially required to participate in many aspects of the modern economy. All of this is a mistake. You can't resist all lock-in. The players involved effectively run the world. You should still try where you can, and we should still be happy when tech companies either slip up or make the momentary strategic decision to make this easier

ori_b 2y ago
> If that ever stops being the case, you should fork rather than trust their judgment. Fork what? The secret sauce is in the training data and infrastructure. I don't think either of those is currently open.
- quasse 2y ago
  I'm just a lowly outsider to the AI space, but calling these open source models seems kind of like calling a compiled binary open source. If you don't have a way to replicate what they did to create the model, it seems more like freeware than open source.
  - advael 2y ago
    As an ML researcher, I agree. Meta doesn't include adequate information to replicate the models, and from the perspective of fundamental research, the interest that big tech companies have taken in this field has been a significant impediment to independent researchers, despite the fact that they are undeniably producing groundbreaking results in many respects, due to this fundamental lack of openness This should also make everyone very skeptical of any claim they are making, from benchmark results to the legalities involved in their training process to the prospect of future progress on these models. Without being able to vet their results against the same datasets they're using, there is no way to verify what they're saying, and the credulity that otherwise smart people have been exhibiting in this space has been baffling to me As a developer, if you have a working Llama model, including the source code and weights, and it's crucial for something you're building or have already built, it's still fundamentally a good thing that Meta isn't gating it behind an API and if they went away tomorrow, you could still use, self-host, retrain, and study the models
    - warkdarrior 2y ago
      The model is public, so you can at least verify their benchmark claims.
      - advael 2y ago
        Generally speaking, no. An important part of a lot of benchmarks in ML research is generalization. What this means is that it's often a lot easier to get a machine learning model to memorize the test cases in a benchmark than it is to train it to perform a general capability the benchmark is trying to test for. For that reason, the dataset is important, as if it includes the benchmark test cases in some way, it invalidates the test When AI research was still mostly academic, I'm sure a lot of people still cheated, but there was somewhat less incentive to, and norms like publishing datasets made it easier to verify claims made in research papers. In a world where people don't, and there's significant financial incentive to lie, I just kind of assume they're lying
    - celdon25 2y ago
      Which option would be better? A) Release the data, and if it ends up causing a privacy scandal, at least you can actually call it open this time. B) Neuter the dataset, and the model All I ever see in these threads is a lot of whining and no viable alternative solutions (I’m fine with the idea of it being a hard problem, but when I see this attitude from “researchers” it makes me less optimistic about the future) > and the credulity that otherwise smart people have been exhibiting in this space has been baffling to me Remove the “otherwise” and you’re halfway to understanding your error.
      - wanderingbort 2y ago
        > Release the data, and if it ends up causing a privacy scandal... We can't prove that a model like llama will never produce a segment of its training data set verbatim. Any potential privacy scandal is already in motion. My cynical assumption is that Meta knows that competitors like OpenAI have PR-bombs in their trained model and therefore would never opensource the weights.
      - 2y ago
  - celdon25 2y ago
    > it seems more like freeware than open source. What would you have them do instead? Specifically?
    - ori_b 2y ago
      Release the training set and the code that was used to train the model, or stop calling it open source. If you can't fork it and take the project in your own direction, it's not open source.
  - wongarsu 2y ago
    > If you don't have a way to replicate what they did to create the model, it seems more like freeware Isn't that a bit like arguing that a linux kernel driver isn't open source if I just give you a bunch of GPL-licensed source code that speaks to my device, but no documentation how my device works? If you take away the source code you have no way to recreate it. But so far that never caused anyone to call the code not open-source. The closest is the whole GPL3 Tivoization debate and that was very divisive. The heart of the issue is that open source is kind of hard to define for anything that isn't software. As a proxy we could look at Stallman's free software definition. Free software shares a common history with open source and in most open source software is free/libre, and the other way around, so this might be a useful proxy. So checking the four software freedoms: - The freedom to run the program as you wish, for any purpose: For most purposes. There's that 700M user restriction, also Meta forbids breaking the law and requires you to follow their acceptable use policy. - The freedom to study how the program works, and change it so it does your computing as you wish: yes. You can change it by fine tuning it, and the weights allow you to figure out how it works. At least as well as anyone knows how any large neural network works, but it's not like Meta is keeping something from you here - The freedom to redistribute copies so you can help your neighbor: Allowed, no real asterisks - The freedom to distribute copies of your modified versions to others: Yes So is it Free Software™? Not really, but it is pretty close.
    - advael 2y ago
      The model is "open-source" for the purpose of software engineering, and it's "closed data" for the purpose of AI research. These are separate issues and it's not necessary to conflate them under one term
- JKCalhoun 2y ago
  A good point. Forgive me, I am AI naive, is there some way to harness Llama to train ones own actually-open AI?
  - advael 2y ago
    Kinda. Since you can self-host the model on a linux machine, there's no meaningful way for them to prevent you from having the trained weights. You can use this to bootstrap other models, or retrain on your own datasets, or fine-tune from the starting point of the currently-working model. What you can't do is be sure what they trained it on
    - QuercusMax 2y ago
      How open is it really though? If you're starting from their weights, do you actually have legal permission to use derived models for commercial purposes? If it turns out that Meta used datasets they didn't have licenses to use in order to generate the model, then you might be in a big heap of mess.
      - advael 2y ago
        From a legal perspective, yea. If we end up having any legal protection against training AI models, legal liability will be a huge mess for everyone involved. From an engineering perspective, if all you need is the pretrained weights, there's not a clear way Facebook could show up and break your product from a technological perspective, as compared to if the thing is relying on, say, an OpenAI API key rather than a self-hosted Llama instance
  - ein0p 2y ago
    I could be wrong but most “model” licenses prohibit the use of the models to improve other models
    - JKCalhoun 2y ago
      That's a good point. I expect it is ultimately unenforceable though. I'm describing training a model for myself, not for sale or public consumption.
- logicchains 2y ago
  They actually did open source the infrastructure library they developed. They don't open source the data but they describe how they gathered/filtered it.
ladzoppelin 2y ago
Is forking really possible with an LLM or one the size of future Lama versions, have they even released the weights and everything? Maybe I am just negative about it because I feel Meta is the worst company ever invented and feel this will hurt society in the long run just like Facebook.
- lawlessone 2y ago
  > have they even released the weights? Isn't that what the model is? just a collection weights?
- pmarreck 2y ago
  When you run `ollama pull llama3.1:70b`, which you can literally do right now (assuming ollama.com is installed and you're not afraid of the terminal), and it downloads a 40 gigabyte model, that is the weights! I'd consider the ability to admit when even your most hated adversary is doing something right, a hallmark of acting smarter. Now, they haven't released the training data with the model weights. THAT plus the training tooling would be "end to end open source". Apple actually did that very thing recently, and it flew under almost everyone's radar for some reason: https://x.com/vaishaal/status/1813956553042711006?s=46&t=qWag1P6krA-5zkoLCeptNw https://x.com/vaishaal/status/1813956553042711006?s=46&t=qWa...
  - mym1990 2y ago
    Doing something right vs doing something that seems right but has a hidden self interest that is harmful in the long run can be vastly different things. Often this kind of strategy will allow people to let their guard down, and those same people will get steamrolled down the road, left wondering where it all went wrong. Get smarter.
    - pmarreck 2y ago
      How in the heck is an open source model that is free and open today going to lock me down, down the line? This is nonsense. You can literally run this model forever if you use NixOS (or never touch your windows, macos or linux install again). Zuck can't come back and molest it. Ever. The best I can tell is that their self-interest here is more about gathering mindshare. That's not a terrible motive; in fact, that's a pretty decent one. It's not the bully pressing you into their ecosystem with a tit-for-tat; it's the nerd showing off his latest and going "Here. Try it. Join me. Join us."
holoduke 2y ago
In tech you can trust the underdogs. Once they turn into dominant players they turn evil. 99% of the cases.

by advael 2y ago

ori_b 2y ago

> If that ever stops being the case, you should fork rather than trust their judgment. Fork what? The secret sauce is in the training data and infrastructure. I don't think either of those is currently open.

quasse 2y ago
I'm just a lowly outsider to the AI space, but calling these open source models seems kind of like calling a compiled binary open source. If you don't have a way to replicate what they did to create the model, it seems more like freeware than open source.
- advael 2y ago
  As an ML researcher, I agree. Meta doesn't include adequate information to replicate the models, and from the perspective of fundamental research, the interest that big tech companies have taken in this field has been a significant impediment to independent researchers, despite the fact that they are undeniably producing groundbreaking results in many respects, due to this fundamental lack of openness This should also make everyone very skeptical of any claim they are making, from benchmark results to the legalities involved in their training process to the prospect of future progress on these models. Without being able to vet their results against the same datasets they're using, there is no way to verify what they're saying, and the credulity that otherwise smart people have been exhibiting in this space has been baffling to me As a developer, if you have a working Llama model, including the source code and weights, and it's crucial for something you're building or have already built, it's still fundamentally a good thing that Meta isn't gating it behind an API and if they went away tomorrow, you could still use, self-host, retrain, and study the models
  - warkdarrior 2y ago
    The model is public, so you can at least verify their benchmark claims.
    - advael 2y ago
      Generally speaking, no. An important part of a lot of benchmarks in ML research is generalization. What this means is that it's often a lot easier to get a machine learning model to memorize the test cases in a benchmark than it is to train it to perform a general capability the benchmark is trying to test for. For that reason, the dataset is important, as if it includes the benchmark test cases in some way, it invalidates the test When AI research was still mostly academic, I'm sure a lot of people still cheated, but there was somewhat less incentive to, and norms like publishing datasets made it easier to verify claims made in research papers. In a world where people don't, and there's significant financial incentive to lie, I just kind of assume they're lying
  - celdon25 2y ago
    Which option would be better? A) Release the data, and if it ends up causing a privacy scandal, at least you can actually call it open this time. B) Neuter the dataset, and the model All I ever see in these threads is a lot of whining and no viable alternative solutions (I’m fine with the idea of it being a hard problem, but when I see this attitude from “researchers” it makes me less optimistic about the future) > and the credulity that otherwise smart people have been exhibiting in this space has been baffling to me Remove the “otherwise” and you’re halfway to understanding your error.
    - wanderingbort 2y ago
      > Release the data, and if it ends up causing a privacy scandal... We can't prove that a model like llama will never produce a segment of its training data set verbatim. Any potential privacy scandal is already in motion. My cynical assumption is that Meta knows that competitors like OpenAI have PR-bombs in their trained model and therefore would never opensource the weights.
    - 2y ago
- celdon25 2y ago
  > it seems more like freeware than open source. What would you have them do instead? Specifically?
  - ori_b 2y ago
    Release the training set and the code that was used to train the model, or stop calling it open source. If you can't fork it and take the project in your own direction, it's not open source.
- wongarsu 2y ago
  > If you don't have a way to replicate what they did to create the model, it seems more like freeware Isn't that a bit like arguing that a linux kernel driver isn't open source if I just give you a bunch of GPL-licensed source code that speaks to my device, but no documentation how my device works? If you take away the source code you have no way to recreate it. But so far that never caused anyone to call the code not open-source. The closest is the whole GPL3 Tivoization debate and that was very divisive. The heart of the issue is that open source is kind of hard to define for anything that isn't software. As a proxy we could look at Stallman's free software definition. Free software shares a common history with open source and in most open source software is free/libre, and the other way around, so this might be a useful proxy. So checking the four software freedoms: - The freedom to run the program as you wish, for any purpose: For most purposes. There's that 700M user restriction, also Meta forbids breaking the law and requires you to follow their acceptable use policy. - The freedom to study how the program works, and change it so it does your computing as you wish: yes. You can change it by fine tuning it, and the weights allow you to figure out how it works. At least as well as anyone knows how any large neural network works, but it's not like Meta is keeping something from you here - The freedom to redistribute copies so you can help your neighbor: Allowed, no real asterisks - The freedom to distribute copies of your modified versions to others: Yes So is it Free Software™? Not really, but it is pretty close.
  - advael 2y ago
    The model is "open-source" for the purpose of software engineering, and it's "closed data" for the purpose of AI research. These are separate issues and it's not necessary to conflate them under one term
JKCalhoun 2y ago
A good point. Forgive me, I am AI naive, is there some way to harness Llama to train ones own actually-open AI?
- advael 2y ago
  Kinda. Since you can self-host the model on a linux machine, there's no meaningful way for them to prevent you from having the trained weights. You can use this to bootstrap other models, or retrain on your own datasets, or fine-tune from the starting point of the currently-working model. What you can't do is be sure what they trained it on
  - QuercusMax 2y ago
    How open is it really though? If you're starting from their weights, do you actually have legal permission to use derived models for commercial purposes? If it turns out that Meta used datasets they didn't have licenses to use in order to generate the model, then you might be in a big heap of mess.
    - advael 2y ago
      From a legal perspective, yea. If we end up having any legal protection against training AI models, legal liability will be a huge mess for everyone involved. From an engineering perspective, if all you need is the pretrained weights, there's not a clear way Facebook could show up and break your product from a technological perspective, as compared to if the thing is relying on, say, an OpenAI API key rather than a self-hosted Llama instance
- ein0p 2y ago
  I could be wrong but most “model” licenses prohibit the use of the models to improve other models
  - JKCalhoun 2y ago
    That's a good point. I expect it is ultimately unenforceable though. I'm describing training a model for myself, not for sale or public consumption.
logicchains 2y ago
They actually did open source the infrastructure library they developed. They don't open source the data but they describe how they gathered/filtered it.

ladzoppelin 2y ago

Is forking really possible with an LLM or one the size of future Lama versions, have they even released the weights and everything? Maybe I am just negative about it because I feel Meta is the worst company ever invented and feel this will hurt society in the long run just like Facebook.

lawlessone 2y ago
> have they even released the weights? Isn't that what the model is? just a collection weights?
pmarreck 2y ago
When you run `ollama pull llama3.1:70b`, which you can literally do right now (assuming ollama.com is installed and you're not afraid of the terminal), and it downloads a 40 gigabyte model, that is the weights! I'd consider the ability to admit when even your most hated adversary is doing something right, a hallmark of acting smarter. Now, they haven't released the training data with the model weights. THAT plus the training tooling would be "end to end open source". Apple actually did that very thing recently, and it flew under almost everyone's radar for some reason: https://x.com/vaishaal/status/1813956553042711006?s=46&t=qWag1P6krA-5zkoLCeptNw https://x.com/vaishaal/status/1813956553042711006?s=46&t=qWa...
- mym1990 2y ago
  Doing something right vs doing something that seems right but has a hidden self interest that is harmful in the long run can be vastly different things. Often this kind of strategy will allow people to let their guard down, and those same people will get steamrolled down the road, left wondering where it all went wrong. Get smarter.
  - pmarreck 2y ago
    How in the heck is an open source model that is free and open today going to lock me down, down the line? This is nonsense. You can literally run this model forever if you use NixOS (or never touch your windows, macos or linux install again). Zuck can't come back and molest it. Ever. The best I can tell is that their self-interest here is more about gathering mindshare. That's not a terrible motive; in fact, that's a pretty decent one. It's not the bully pressing you into their ecosystem with a tit-for-tat; it's the nerd showing off his latest and going "Here. Try it. Join me. Join us."

holoduke 2y ago

In tech you can trust the underdogs. Once they turn into dominant players they turn evil. 99% of the cases.