4 ms·

I am not sure why people keep pretending like this is possible or keep asking it to happen. You know that distributing all the data would be illegal copyright

by stale2002 2y ago

I am not sure why people keep pretending like this is possible or keep asking it to happen. You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks. Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down, or what? Because anyone who actually supports open "anything" would understand this, instead of pedantically arguing about the definition of open source and by suggesting things that both aren't possible and would make the release of these models much less likely.

throwthrowuknow 2y ago
Perhaps a good solution would be to form a trust responsible for storing and distributing training data. It could be held in common by all contributors and allow anyone to access high quality curated data. It would need legal allowances and exemption from copyright which would be difficult to work out but perhaps some mechanism could be created where copyright holders could derive benefit from not seeking to have their works removed, not necessarily monetary. This would at least get data collection out of the shadows and allow companies and researchers to attest that they are using the data only for training and not distributing it otherwise. That could go a long way toward clarifying issues of whether or not a book or article was in a dataset, how often it appeared, how many epochs it was used in, whether it only appeared as referenced or quoted by other text, etc. With that kind of disclosure it would be possible to write tests to ensure that the models won’t simply regurgitate the material without attribution or unprompted and publish those tests to assuage fears.
fragmede 2y ago
Anyone who understands the difference between model available and open source would realize they're different things, and then also be smart enough to realize that randos arguing about the definition isn't what's going to sway someone as to wether they'd release the model or the source used to create them. You think Mark Zuckerberg, or anyone else with the $100s of million dollars to spend on training is reading these comments, seeing these dorks argue about if giving away models is open source or not, and then saying, I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna. We want words to mean things. model available != open source, If you go to a restaurant and order the beef and you get the fish, what's wrong, it's still food, just eat it. Don't be pedantic, otherwise the cook isn't going to make food anymore. If you're 10 and your parents made you dinner that might pass, but if you're an adult and go to a restaurant where you're paying with your own hard earned money, that's not how it works. They don't have to distribute the data directly, Stability AI has said they used the LAION-5B Dataset. Just like I could say I read Harry Potter without giving you a copy, and can say that it says "the boy who lived" without violating copyright.
- stale2002 2y ago
  > I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna. Its not because he said things. Instead it is that (if implemented) the things that he is advocating for would actually significantly reduce the amount of open-whatever-you-want-to-call-it models available as most of it would all get shut down. If these people don't want these models to exist and they are just using this concern trolling to get rid of all these AI models, then I wish they would just say so openly. Because that would be the result of their positions. It would be a destruction of most of the more-open AI models that we have available today and an end to the open-whatever AI industry. If thats what you want, just say it, and then we can talk about the pros and cons of a completely centralized AI ecosystem that is only controlled by OpenAI, Google, and the government and how if the destruction of the AI startup space that is reliant on the models given out by meta/ect is good or bad.
  - fragmede 2y ago
    it's not concern trolling, it's that the words used don't describe what it is. what Mark Zuckerberg has done with releasing llama3 and making the model available is frickin awesome, just don't call it open source! it's model available. Trying to paint me into a corner with a false dichotomy, that if I want it to be called model available instead of open source, and that in doing so, I want a dystopian future where megacorps have AGI and we're all at their mercy isn't going to work. I don't want the models to not be available, what I want is to see what's being used to create the models, the source, if you will, and thus the alignment side of the source, and why it won't answer "how do I make cocaine", as if I'm going to go off and somehow become Tony Montoya. We may never get that, but that's what calling it open source and have it mean that would entail. Making the models available to the plebs is a generous thing for Mark to do, but that doesn't mean we can't ask for more. Or maybe we shouldn't; and just be grateful for his benevolence, because it's not like OpenAI has done the same with ChatGPT. I will give them credit for Whisper though.

throwthrowuknow 2y ago

Perhaps a good solution would be to form a trust responsible for storing and distributing training data. It could be held in common by all contributors and allow anyone to access high quality curated data. It would need legal allowances and exemption from copyright which would be difficult to work out but perhaps some mechanism could be created where copyright holders could derive benefit from not seeking to have their works removed, not necessarily monetary. This would at least get data collection out of the shadows and allow companies and researchers to attest that they are using the data only for training and not distributing it otherwise. That could go a long way toward clarifying issues of whether or not a book or article was in a dataset, how often it appeared, how many epochs it was used in, whether it only appeared as referenced or quoted by other text, etc. With that kind of disclosure it would be possible to write tests to ensure that the models won’t simply regurgitate the material without attribution or unprompted and publish those tests to assuage fears.

fragmede 2y ago

Anyone who understands the difference between model available and open source would realize they're different things, and then also be smart enough to realize that randos arguing about the definition isn't what's going to sway someone as to wether they'd release the model or the source used to create them. You think Mark Zuckerberg, or anyone else with the $100s of million dollars to spend on training is reading these comments, seeing these dorks argue about if giving away models is open source or not, and then saying, I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna. We want words to mean things. model available != open source, If you go to a restaurant and order the beef and you get the fish, what's wrong, it's still food, just eat it. Don't be pedantic, otherwise the cook isn't going to make food anymore. If you're 10 and your parents made you dinner that might pass, but if you're an adult and go to a restaurant where you're paying with your own hard earned money, that's not how it works. They don't have to distribute the data directly, Stability AI has said they used the LAION-5B Dataset. Just like I could say I read Harry Potter without giving you a copy, and can say that it says "the boy who lived" without violating copyright.

stale2002 2y ago
> I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna. Its not because he said things. Instead it is that (if implemented) the things that he is advocating for would actually significantly reduce the amount of open-whatever-you-want-to-call-it models available as most of it would all get shut down. If these people don't want these models to exist and they are just using this concern trolling to get rid of all these AI models, then I wish they would just say so openly. Because that would be the result of their positions. It would be a destruction of most of the more-open AI models that we have available today and an end to the open-whatever AI industry. If thats what you want, just say it, and then we can talk about the pros and cons of a completely centralized AI ecosystem that is only controlled by OpenAI, Google, and the government and how if the destruction of the AI startup space that is reliant on the models given out by meta/ect is good or bad.
- fragmede 2y ago
  it's not concern trolling, it's that the words used don't describe what it is. what Mark Zuckerberg has done with releasing llama3 and making the model available is frickin awesome, just don't call it open source! it's model available. Trying to paint me into a corner with a false dichotomy, that if I want it to be called model available instead of open source, and that in doing so, I want a dystopian future where megacorps have AGI and we're all at their mercy isn't going to work. I don't want the models to not be available, what I want is to see what's being used to create the models, the source, if you will, and thus the alignment side of the source, and why it won't answer "how do I make cocaine", as if I'm going to go off and somehow become Tony Montoya. We may never get that, but that's what calling it open source and have it mean that would entail. Making the models available to the plebs is a generous thing for Mark to do, but that doesn't mean we can't ask for more. Or maybe we shouldn't; and just be grateful for his benevolence, because it's not like OpenAI has done the same with ChatGPT. I will give them credit for Whisper though.