4 ms·

I agree, except in the case of OpenAI it should be an end, and yet they fail at it spectacularly, and since Altman/Microsoft finished their takeover there is ba

by nolok 2y ago

I agree, except in the case of OpenAI it should be an end, and yet they fail at it spectacularly, and since Altman/Microsoft finished their takeover there is basically no hope of it ever coming back.

ben_w 2y ago
The OpenAI charter specifically says "safe". We're all arguing over what that even means for an AI. If you're at all risk adverse, that argument by itself should be a hint that releasing the weights of a model. For example, the last few years have seen a lot of angry comments about how "the algorithm" (for both social media feeds and Google search results) is politically biased. IIRC we don't know the full set of training data (data includes RLHF responses before anyone points me to the Wikipedia page I already read that lists some data sources) how can we be confident that any model from Facebook has not been specifically trained to drive a specific political agenda?
- littlestymaar 2y ago
  The safety argument has proven complete BS now that they commercialize the unsafe AI…
  - anon373839 2y ago
    I can’t believe anyone buys it anymore. It feels like it was just yesterday when they were begging for a pause on training “dangerous” models (where danger was defined as “anything better than our flagship product”).
    - ben_w 2y ago
      The request has become law and there aren't any models clearly better than their flagship.
  - ben_w 2y ago
    "The" safety argument? You think there's only one? "The unsafe AI"? Which one would this be? Would it be the one which so many people on this very website complain has been "lobotomised"? (No matter how much I object to that word in this context, that's what people call it).
    - littlestymaar 2y ago
      You can't have it both ways: either ChatGPT as it is now is dangerous (hence you don't open the weights but you also should not commercialize it) or it is not and there's no good reason to keep it secret. ChatGPT has clearly caused significant negative social impact (students cheating on their essays, SEO spam, etc.) and they didn't give a shit.
      - ben_w 2y ago
        > You can't have it both ways: either ChatGPT as it is now is dangerous (hence you don't open the weights but you also should not commercialize it) or it is not and there's no good reason to keep it secret. Almost every dangerous thing I can think of "has it both ways" by the standard you apply here. I can use an aircraft without being a registered pilot; I can use the police without being a trained law enforcement officer; I can use restricted drugs when supplied by a duly authorised medical professional; I can use high voltage equipment, high power lasers, high intensity RF sources, when they are encased in the appropriate safety equipment that allows them to be legally sold as a lightbulb, a DVD player, and microwave oven respectively. The weights themselves reveal any and all information found within the model, regardless of superficial attempts to prevent the model "leaking". We do not, at present, actually know how to locate information within a model as would be required to confirm that it has genuinely deleted some information rather than merely pretending — this is an active field of research. By analogy: data on a spinning hard drive. In certain file systems, if you delete a file, you only remove the pointer to it, the actual content can be un-deleted. A full overwrite of unused space is better, but owing to imprecision in the write head, even this is not actually certain to delete data, and multiple passes are used — but even this was not sufficient for the agents who oversaw the destruction of The Guardian's laptop containing a copy of the Snowden data. At present, we do not even know how to fully determine the capabilities of a set of weights, so we cannot actually tell if any given model is "safe" (not even by a restricted definition of "safe" as in "we won't get sued under GDPR Article 17"), we can only guess. And those best-guesses are what people complain about when they are upset that an AI model no longer does what it did last week. There is an argument that having open access to the weights makes it easier to perform the research necessary to even be able to answer this question. That's an important counter! But it's not even remotely where you're going with your comment. > ChatGPT has clearly caused significant negative social impact (students cheating on their essays, SEO spam, etc.) and they didn't give a shit. None of those are significant social impact. Negative, sure, but the "significant" risk isn't SEO spam, it's giving everyone their own personal Goebbels; and it's not "students cheating" because, to the extent the AI is capable of that, the tests they can cheat on with it represent something that is now fully automated to the point that it shouldn't be tested — I mean, I grew up with the maths teachers saying "you won't have a calculator on you all the time when you're an adult", but by the time I got to my exams some of the tests required a calculator, and now I regularly do ⌘-space and it does the unit conversion as well as the maths, and even for the symbolic calculus I can usually just ask Wolfram Alpha… which is on my phone, which I have on me all the time.

by nolok 2y ago

I agree, except in the case of OpenAI it should be an end, and yet they fail at it spectacularly, and since Altman/Microsoft finished their takeover there is basically no hope of it ever coming back.

ben_w 2y ago

The OpenAI charter specifically says "safe". We're all arguing over what that even means for an AI. If you're at all risk adverse, that argument by itself should be a hint that releasing the weights of a model. For example, the last few years have seen a lot of angry comments about how "the algorithm" (for both social media feeds and Google search results) is politically biased. IIRC we don't know the full set of training data (data includes RLHF responses before anyone points me to the Wikipedia page I already read that lists some data sources) how can we be confident that any model from Facebook has not been specifically trained to drive a specific political agenda?

littlestymaar 2y ago
The safety argument has proven complete BS now that they commercialize the unsafe AI…
- anon373839 2y ago
  I can’t believe anyone buys it anymore. It feels like it was just yesterday when they were begging for a pause on training “dangerous” models (where danger was defined as “anything better than our flagship product”).
  - ben_w 2y ago
    The request has become law and there aren't any models clearly better than their flagship.
- ben_w 2y ago
  "The" safety argument? You think there's only one? "The unsafe AI"? Which one would this be? Would it be the one which so many people on this very website complain has been "lobotomised"? (No matter how much I object to that word in this context, that's what people call it).
  - littlestymaar 2y ago
    You can't have it both ways: either ChatGPT as it is now is dangerous (hence you don't open the weights but you also should not commercialize it) or it is not and there's no good reason to keep it secret. ChatGPT has clearly caused significant negative social impact (students cheating on their essays, SEO spam, etc.) and they didn't give a shit.
    - ben_w 2y ago
      > You can't have it both ways: either ChatGPT as it is now is dangerous (hence you don't open the weights but you also should not commercialize it) or it is not and there's no good reason to keep it secret. Almost every dangerous thing I can think of "has it both ways" by the standard you apply here. I can use an aircraft without being a registered pilot; I can use the police without being a trained law enforcement officer; I can use restricted drugs when supplied by a duly authorised medical professional; I can use high voltage equipment, high power lasers, high intensity RF sources, when they are encased in the appropriate safety equipment that allows them to be legally sold as a lightbulb, a DVD player, and microwave oven respectively. The weights themselves reveal any and all information found within the model, regardless of superficial attempts to prevent the model "leaking". We do not, at present, actually know how to locate information within a model as would be required to confirm that it has genuinely deleted some information rather than merely pretending — this is an active field of research. By analogy: data on a spinning hard drive. In certain file systems, if you delete a file, you only remove the pointer to it, the actual content can be un-deleted. A full overwrite of unused space is better, but owing to imprecision in the write head, even this is not actually certain to delete data, and multiple passes are used — but even this was not sufficient for the agents who oversaw the destruction of The Guardian's laptop containing a copy of the Snowden data. At present, we do not even know how to fully determine the capabilities of a set of weights, so we cannot actually tell if any given model is "safe" (not even by a restricted definition of "safe" as in "we won't get sued under GDPR Article 17"), we can only guess. And those best-guesses are what people complain about when they are upset that an AI model no longer does what it did last week. There is an argument that having open access to the weights makes it easier to perform the research necessary to even be able to answer this question. That's an important counter! But it's not even remotely where you're going with your comment. > ChatGPT has clearly caused significant negative social impact (students cheating on their essays, SEO spam, etc.) and they didn't give a shit. None of those are significant social impact. Negative, sure, but the "significant" risk isn't SEO spam, it's giving everyone their own personal Goebbels; and it's not "students cheating" because, to the extent the AI is capable of that, the tests they can cheat on with it represent something that is now fully automated to the point that it shouldn't be tested — I mean, I grew up with the maths teachers saying "you won't have a calculator on you all the time when you're an adult", but by the time I got to my exams some of the tests required a calculator, and now I regularly do ⌘-space and it does the unit conversion as well as the maths, and even for the symbolic calculus I can usually just ask Wolfram Alpha… which is on my phone, which I have on me all the time.