4 ms·

Luckily that doesn't exist. It's about equivalent to spinning up a Ghostbusters unit and letting it wither. You can argue that we'll be in trouble if we encount

by andy99 2y ago

Luckily that doesn't exist. It's about equivalent to spinning up a Ghostbusters unit and letting it wither. You can argue that we'll be in trouble if we encounter ghosts, but on likelihood weighted basis there are much better uses of time.

lgas 2y ago
Probability is only half of the expected value equation. When the outcome is excessive then you should care even if the probability is low as long as it is non-zero.
- XorNot 2y ago
  Nobody knows what the outcome is though. Its literally bad science fiction being used to pay people to do... What exactly? About a technology which doesn't exist.
manmal 2y ago
Part of me thinks that’s exactly why this department has been sidelined - having such a department is necessary to create hype (“we are creating things so powerful we have to explore how to contain them”), but it doesn’t need to thrive either.
- ilrwbwrkhv 2y ago
  The real answer is they have realized internally that agi is simply not possible. Even gpt 4o doesn't understand at all any logic.
  - sigmoid10 2y ago
    That's simply not true. We now have LLMs that perform similarly or better than average humans on general reasoning and logic benchmarks. So if you were to say that they don't understand logic at all, then by that definition most humans don't either (which can be debated, but it's a different topic).
    - codexon 2y ago
      GPT4o still can't reason. It is super fancy autocomplete. https://i.imgur.com/z83umbk.jpeg https://i.imgur.com/z83umbk.jpeg Here I change a widely known riddle to the opposite answer, and I manage to make it state them both as the answer.
      - jordanpg 2y ago
        Also, even if this oft-repeated trump card is true, why are you so sure this is different from how our own brains functionally work?
        codexon 2y ago
        Because if this is how our brains really worked, then chatgpt wouldn't be beating most humans at standardized tests and then failing this absurdly easy question that even an elementary school kid could pass. It is simply regurgitating this phrase without even considering that it is stating the exact opposite of the answer it just gave, simply because most answers to this riddle on the internet say this at the end. > This riddle plays on the assumption that a surgeon is typically male, but in this case, the surgeon is the boy's mother. So from this 1 failing, you can see that it is a copy and paste machine, and it doesn't even understand that it is contradicting itself.
        jordanpg 2y ago
        Your counterexample doesn’t prove that this isn’t how our minds functionally work, except better. No amount of counterexamples could.
      - Eisenstein 2y ago
        'Fancy autocomplete' is a thought-terminating cliche. You can stump a person with a riddle or a logic puzzle or an optical illusion.
        gfourfour 2y ago
FormerBandmate 2y ago
It could theoretically be possible, but it's not gonna happen from training a machine learning model off Reddit. SolidGoldMagikarp is the canary in the coal mine if you doubt it's Reddit
agucova 2y ago
I find this kind of dismissive attitude annoying. There are good arguments in the literature for why you might want to care about these risks [1, 2], and I think there's lots of room for reasonable disagreement about whether these are arguments are any good, but pretending the entire field of AI Safety is just delusional is just bad faith at this point. Especially when companies like OpenAI, Anthropic or GDM were explicitly created to build AGI, and have been talking about these risks since they were first founded. [1]: An introductory paper I like is The Alignment Problem for a Deep Learning Perspective, from Ngo et al, https://openreview.net/forum?id=fh8EYKFKns https://openreview.net/forum?id=fh8EYKFKns. [2]: A broader, less technical introduction to AI Safety that I like is Hendricks et al's An Overview of Catastrophic AI Risks, https://arxiv.org/abs/2306.12001 https://arxiv.org/abs/2306.12001
- gfourfour 2y ago
  > This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans. The first three risks are completely reasonable and people should be thinking about them. No, ChatGPT should not be diagnosing patients and giving them medicine. Yes, we should be vigilant to a flood of disinformation and revenge porn made possible by AI generated content. But when people talk about “AI safety” in this context, it’s usually in reference to the fourth category, planning for a superintelligent malicious AI that evades detection, self-replicates, etc. That’s pure science fiction at that point, and it’s not a reason to slow down development of LLMs, which yes are basically glorified chatbots and will not lead to “AGI” in this threatening sense. If I recall correctly, when steam engines started being able to go 40-50 MPH, there were people who were concerned that human beings would not be able to survive travel at such speeds because we never had experienced them. This wasn’t completely irrational, I suppose, as there are speed-induced G forces that are fatal, and they had no way of knowing the threshold back then. But once it was clear that steam locomotives weren’t in any danger of putting us over that threshold, incessant worry about locomotive-induced speeds death was kooky. “Locomotive safety” involving derailment mitigation, track crossing markings, etc. - still legitimate. But if “locomotive safety” was associated with people making claims like “we’re headed for a mass casualty event when the first locomotive hits 60 mph,” then “locomotive safety” would be marginalized. It doesn’t help that the public faces of “AI safety” include autodidactic pseudointellectuals, clearly mentally unwell people, and philosophers too deep in their own “taken to its logical conclusion…” thought experiments.

by andy99 2y ago

lgas 2y ago

Probability is only half of the expected value equation. When the outcome is excessive then you should care even if the probability is low as long as it is non-zero.

XorNot 2y ago
Nobody knows what the outcome is though. Its literally bad science fiction being used to pay people to do... What exactly? About a technology which doesn't exist.

manmal 2y ago

Part of me thinks that’s exactly why this department has been sidelined - having such a department is necessary to create hype (“we are creating things so powerful we have to explore how to contain them”), but it doesn’t need to thrive either.

ilrwbwrkhv 2y ago
The real answer is they have realized internally that agi is simply not possible. Even gpt 4o doesn't understand at all any logic.
- sigmoid10 2y ago
  That's simply not true. We now have LLMs that perform similarly or better than average humans on general reasoning and logic benchmarks. So if you were to say that they don't understand logic at all, then by that definition most humans don't either (which can be debated, but it's a different topic).
  - codexon 2y ago
    GPT4o still can't reason. It is super fancy autocomplete. https://i.imgur.com/z83umbk.jpeg https://i.imgur.com/z83umbk.jpeg Here I change a widely known riddle to the opposite answer, and I manage to make it state them both as the answer.
    - jordanpg 2y ago
      Also, even if this oft-repeated trump card is true, why are you so sure this is different from how our own brains functionally work?
      - codexon 2y ago
        Because if this is how our brains really worked, then chatgpt wouldn't be beating most humans at standardized tests and then failing this absurdly easy question that even an elementary school kid could pass. It is simply regurgitating this phrase without even considering that it is stating the exact opposite of the answer it just gave, simply because most answers to this riddle on the internet say this at the end. > This riddle plays on the assumption that a surgeon is typically male, but in this case, the surgeon is the boy's mother. So from this 1 failing, you can see that it is a copy and paste machine, and it doesn't even understand that it is contradicting itself.
        jordanpg 2y ago
        Your counterexample doesn’t prove that this isn’t how our minds functionally work, except better. No amount of counterexamples could.
    - Eisenstein 2y ago
      'Fancy autocomplete' is a thought-terminating cliche. You can stump a person with a riddle or a logic puzzle or an optical illusion.
      - gfourfour 2y ago

FormerBandmate 2y ago

It could theoretically be possible, but it's not gonna happen from training a machine learning model off Reddit. SolidGoldMagikarp is the canary in the coal mine if you doubt it's Reddit

agucova 2y ago

I find this kind of dismissive attitude annoying. There are good arguments in the literature for why you might want to care about these risks [1, 2], and I think there's lots of room for reasonable disagreement about whether these are arguments are any good, but pretending the entire field of AI Safety is just delusional is just bad faith at this point. Especially when companies like OpenAI, Anthropic or GDM were explicitly created to build AGI, and have been talking about these risks since they were first founded. [1]: An introductory paper I like is The Alignment Problem for a Deep Learning Perspective, from Ngo et al, https://openreview.net/forum?id=fh8EYKFKns https://openreview.net/forum?id=fh8EYKFKns. [2]: A broader, less technical introduction to AI Safety that I like is Hendricks et al's An Overview of Catastrophic AI Risks, https://arxiv.org/abs/2306.12001 https://arxiv.org/abs/2306.12001

gfourfour 2y ago
> This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans. The first three risks are completely reasonable and people should be thinking about them. No, ChatGPT should not be diagnosing patients and giving them medicine. Yes, we should be vigilant to a flood of disinformation and revenge porn made possible by AI generated content. But when people talk about “AI safety” in this context, it’s usually in reference to the fourth category, planning for a superintelligent malicious AI that evades detection, self-replicates, etc. That’s pure science fiction at that point, and it’s not a reason to slow down development of LLMs, which yes are basically glorified chatbots and will not lead to “AGI” in this threatening sense. If I recall correctly, when steam engines started being able to go 40-50 MPH, there were people who were concerned that human beings would not be able to survive travel at such speeds because we never had experienced them. This wasn’t completely irrational, I suppose, as there are speed-induced G forces that are fatal, and they had no way of knowing the threshold back then. But once it was clear that steam locomotives weren’t in any danger of putting us over that threshold, incessant worry about locomotive-induced speeds death was kooky. “Locomotive safety” involving derailment mitigation, track crossing markings, etc. - still legitimate. But if “locomotive safety” was associated with people making claims like “we’re headed for a mass casualty event when the first locomotive hits 60 mph,” then “locomotive safety” would be marginalized. It doesn’t help that the public faces of “AI safety” include autodidactic pseudointellectuals, clearly mentally unwell people, and philosophers too deep in their own “taken to its logical conclusion…” thought experiments.