6 ms·

'I'm a Doomer': OpenAI's New Interim CEO Wants to Slow AI Progress Down

29 points by isaacfrond 3y ago

gumballindie 3y ago
[flagged]
- lucubratory 3y ago
  Is there a specific term for the phenomena where people arguing online substitute "I disagree with your opinion" with "Your opinion is proof that you are severely mentally ill", "Your opinion is similar to fiction, which means you have the brain of a child", or "Your opinion is proof that you are a paid foreign agent"? I know it's just generally part of being an anonymous online asshole who doesn't care about what's true or having values, only dunking on someone using some easily accessible thought terminating cliche, but surely there's a specific term for this.
  - aeonik 3y ago
    I think it's a specific kind of Ad Hominim. It's not even a great Ad Hominim because folks like John Nash who did have mental illness were really gifted in other areas. I was about to post a study that even showed that mental illness is linked with mental disorders, but looks like this recent study is pushing back on that idea. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9879926/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9879926/
    - solardev 3y ago
      > even showed that mental illness is linked with mental disorders Huh? Is there a typo in there, or are disorders and illnesses different in some big way?
      - aeonik 3y ago
        Yes thanks for pointing it out. Mental illness, I thought, was correlated with higher intelligence. Especially depression, though there is also the legend of the mad genius. When I looked up studies to reference this it looks like the landscape here has changed.
  - gumballindie 3y ago
    Sorry, but what else can cause people to fear software and think it’s intelligent and can destroy the world? It most certainly isnt something real. We can pretend it is, but it’s affecting people’s lives. They are constantly bombarded with this nonsense.
    - johngladtj 3y ago
      Do you genuinely not recognize the possibility that software can, even unintentionally, cause significant harm to both individuals and large groups of people? Do I need to list out cases where that has already happen?
      - gumballindie 3y ago
        There’s a huge gap between unintentional software errors and software choosing to do harm by its own “desire”. The crowd seeks to think ai will “do” the latter. And that’s an issue. Also it seems like there’s a pattern of redefining the meaning of words to accommodate that imaginary world. All this sounds like a copy cat of 90’s scares about computer virii and ai (yes, this not the first ai is “taking over the world”). Amusing people used to come out on tv to tell us how virii will take over nuclear plants and ai will make those virii.
  - jollyllama 3y ago
    Broadly, it's psychologizing. More interestingly, look into "The Therapeutic State" for when this scales up.
  - vivekd 3y ago
    That phenomenon you speak of is real but I think the parents' comment is valid in this case. Being afraid of a large language model running statistics does look a lot like childishness or mental illness. I can give the general public a pass because maybe they don't know the tech and are going off media hype. But for someone to know what this is and be afraid of it does look silly doesn't it?
    - slenocchio 3y ago
      Listen to Eliezer Yudkowsky's arguments. Perhaps it looks silly to you but there are actually _extremely_ good reasons to be worried about AI progress without figuring out alignment first. What would happen if we gave humanity the internet before we had cryptography? On a side note, it's troubling how many hyper intelligent and successful people in tech are happy to denigrate the opposition in bad faith.
      - gumballindie 3y ago
        > internet before we had cryptography? I’m sorry what? The internet has been clear text for the most part.
        slenocchio 3y ago
        We can communicate securely on the internet because of cryptography (HTTPS vs. HTTP). If cryptography didn't exist any signal you ever sent over the internet could be observed and used maliciously against you (payment info, passwords, etcetera).
        argiopetech 3y ago
        The WWW (1989) ran for 5 years before SSL 1.0 (1994). If cryptography were never invented, we would use the internet like we used analog telephones (i.e., under the assumption our communication could be easily intercepted) and not pass privileged information over it. Otherwise, I imagine it would be a non-issue. Given changing laws on cryptography (i.e., state-required backdoors & etc.), we may be back in that mindset before too long.
  - mypastself 3y ago
    I used to think you could be a 9/11 truther and still hold reasonable beliefs otherwise. I thought a lot of these people genuinely looked at the official story and related evidence with an open mind, and came to a different conclusion than I did. But the more of them I’ve met, the more I’ve found they tend to be everything-else truthers as well: moon landing, vaccines, etc., and I tend to disregard their arguments outright. Is that fair? Maybe not. But it’s borne out of years of similar interactions, and it’s saved me a lot of time and effort. Lately I’ve started acting similarly with doomers, almost inadvertently. If you think that e.g. climate change will make humanity go extinct in the next few decades, it’s likely you feel similarly about A.I., COVID, etc. I’m starting to shut out such claims immediately, or at least engage with them less. There seem to be certain personality types who gravitate towards certain types of beliefs. I’m not qualified to speculate as to why they do, but sometimes simply ignoring them can be… sensible.
    - solardev 3y ago
      It saddens me how right you are. Things have become so polarized and tribal that there isn't much room for nuance. Can't climate change be a big deal without being an extinction risk? Covid was deadly, but mostly to certain demographics. AI is dangerous in the way that the Manhattan Project was dangerous, but maybe that's not sufficient reason to stop pursuing it. Etc. But I think people are so overwhelmed by this hyperconnected and ubersensationalized world, flooded with ethical conundrums daily, that they only have the bandwidth for snap judgments. Into this bucket or the other you go, thought, no time or energy for a closer examination!
maremmano 3y ago
Why I feel that China will not slow down AI progress?
- ShamelessC 3y ago
  They won’t but mainland China hasn’t shown a very good track record of keeping up, much less inventing new methods that gain any sort of traction. They do seem to produce plenty of good researchers, who sometimes proceed to do good research with proper funding from western institutions. I would be more worried about Europe.
  - boringuser2 3y ago
    [flagged]
    - fsociety 3y ago
      That seems like an overly broad and unfair generalization of Chinese.
      - boringuser2 3y ago
        Every generalization is "overly broad". That's why it's called a "generalization". As for whether the claim is fair or not, that's an opinion, but I disagree, obviously.
    - solardev 3y ago
      Um, are we just skipping past the whole colonialist era and straight into some sort of enlightened modern age? Even then, Russia's European too. China's had a rough and tumble history, to be sure, but they didn't try to take over the whole world. Just Tibet and Hong Kong so far, maybe Taiwan someday. I think their ambitions are different, and so are their ethical frameworks. The CCP isn't a governing body I admire, but their people and culture have a very long history of collectivism and Confucianism. The flip side of Han supremacy and cultural erasure is national unity and civil harmony, their version of manifest destiny I guess, with an emphasis on assimilation. I envision a Great Chinese AI to be in line with the firewall, ostensibly (and mostly) uses to promote harmony via strict control, rather than conquest. Maybe more Borg than Skynet?
snorkel 3y ago
I wonder how deeply the board and the rotating CEOs actually understand the technology and practicality of achieving AGI, or whether they are falling over themselves due to buying into the LLM hype.
datpiff 3y ago
If they try to keep the same pace of LLM advancement to preserve a high valuation, they might fail - LLM training costs skyrocket and it will get harder and harder to get the data. The CEO announcing a slow-down could likely be a strategy not to spook investors.
shortsunblack 3y ago
No, AI is not going to kill us anytime soon. Yes, LLMs are not even close to AGI and the research on them does not lead to research on understanding cognition (read Chomsky on this), which is required for AGI. Yes, DO ban all AI research and investment until society as a whole, by consensus, grants the privilege of continuing. OpenAI saga already has shown Sillicon Valley tech culture is in no fit shape to be involved in such manners in any form. The culture that gave us Musk buying Twitter and torpedoing its value by half the next day and the shitcoin scammer SBF does not have a good track record.
- thfuran 3y ago
  >the research on them does not lead to research on understanding cognition (read Chomsky on this), which is required for AGI. Many phenomena have been discovered unintentionally and/or without understanding them. Deliberately developing an agi would probably be easier with a better understanding of cognition, but it's almost certainly not required.
- dcwca 3y ago
  With all due respect, Chomsky has no intuition for this kind of technology. Was sad to see him reject the technology, felt oddly like sour grapes when really LLMs have validated his work on universal grammar.
- chpatrick 3y ago
  > No, AI is not going to kill us anytime soon. > Yes, DO ban all AI research and investment So do you think it's a threat or not?
  - agevag 3y ago
    [dead]
    - wholinator2 3y ago
      I think it's a Stretch to assert that for the GP. Where did they bring that kind of politics into their view or what led you to think that they, or anyone, thinks that
elzbardico 3y ago
The next few years are going to be interesting for lawyers. I imagine all kinds of lawsuits including, but not limited to, stealing of trade secrets are going to started by OpenAI foundation against Sam's new employers and all the retaliatory counter-suing.
elzbardico 3y ago
Killing us in a nuclear holocaust is for now a very far fetched hypothesis. I surely hope Nuclear C&C networks are properly air-gaped from the internet, and that actually launching the nukes require humans in the loop (yes, humans that could be socially engineered by an uber smart AGI, but then we are reaching). The immediate danger from UI is on the economic and political realm. Reinforcing wealth-concentration, monopolies, all powerful surveillance and propaganda. I care more about those immediate dangers, that are related to how humans that control the AI can use it for their benefit versus the rest of us, than to the distant theoretical scenario of an AI going crazy and deciding to go all skynet on US. I fear the AI owners, they are going to fuck us way before the AI has any idea about fucking humans.
dtagames 3y ago
This hire makes is clear why MS is taking the IP and breaking ranks w/ OpenAI. He wants to slow down AI to development to 1/10th of its current pace? You can't run a company like that. He could run a library research program but that's about it. MS doesn't want to slow down AI; they want to own it. Many people realize that AGI isn't a thing and LLMs will never get there. Computers lack volition, but corporations have in it spades.

29 points by isaacfrond 3y ago

gumballindie 3y ago

[flagged]

lucubratory 3y ago
Is there a specific term for the phenomena where people arguing online substitute "I disagree with your opinion" with "Your opinion is proof that you are severely mentally ill", "Your opinion is similar to fiction, which means you have the brain of a child", or "Your opinion is proof that you are a paid foreign agent"? I know it's just generally part of being an anonymous online asshole who doesn't care about what's true or having values, only dunking on someone using some easily accessible thought terminating cliche, but surely there's a specific term for this.
- aeonik 3y ago
  I think it's a specific kind of Ad Hominim. It's not even a great Ad Hominim because folks like John Nash who did have mental illness were really gifted in other areas. I was about to post a study that even showed that mental illness is linked with mental disorders, but looks like this recent study is pushing back on that idea. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9879926/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9879926/
  - solardev 3y ago
    > even showed that mental illness is linked with mental disorders Huh? Is there a typo in there, or are disorders and illnesses different in some big way?
    - aeonik 3y ago
      Yes thanks for pointing it out. Mental illness, I thought, was correlated with higher intelligence. Especially depression, though there is also the legend of the mad genius. When I looked up studies to reference this it looks like the landscape here has changed.
- gumballindie 3y ago
  Sorry, but what else can cause people to fear software and think it’s intelligent and can destroy the world? It most certainly isnt something real. We can pretend it is, but it’s affecting people’s lives. They are constantly bombarded with this nonsense.
  - johngladtj 3y ago
    Do you genuinely not recognize the possibility that software can, even unintentionally, cause significant harm to both individuals and large groups of people? Do I need to list out cases where that has already happen?
    - gumballindie 3y ago
      There’s a huge gap between unintentional software errors and software choosing to do harm by its own “desire”. The crowd seeks to think ai will “do” the latter. And that’s an issue. Also it seems like there’s a pattern of redefining the meaning of words to accommodate that imaginary world. All this sounds like a copy cat of 90’s scares about computer virii and ai (yes, this not the first ai is “taking over the world”). Amusing people used to come out on tv to tell us how virii will take over nuclear plants and ai will make those virii.
- jollyllama 3y ago
  Broadly, it's psychologizing. More interestingly, look into "The Therapeutic State" for when this scales up.
- vivekd 3y ago
  That phenomenon you speak of is real but I think the parents' comment is valid in this case. Being afraid of a large language model running statistics does look a lot like childishness or mental illness. I can give the general public a pass because maybe they don't know the tech and are going off media hype. But for someone to know what this is and be afraid of it does look silly doesn't it?
  - slenocchio 3y ago
    Listen to Eliezer Yudkowsky's arguments. Perhaps it looks silly to you but there are actually _extremely_ good reasons to be worried about AI progress without figuring out alignment first. What would happen if we gave humanity the internet before we had cryptography? On a side note, it's troubling how many hyper intelligent and successful people in tech are happy to denigrate the opposition in bad faith.
    - gumballindie 3y ago
      > internet before we had cryptography? I’m sorry what? The internet has been clear text for the most part.
      - slenocchio 3y ago
        We can communicate securely on the internet because of cryptography (HTTPS vs. HTTP). If cryptography didn't exist any signal you ever sent over the internet could be observed and used maliciously against you (payment info, passwords, etcetera).
        argiopetech 3y ago
        The WWW (1989) ran for 5 years before SSL 1.0 (1994). If cryptography were never invented, we would use the internet like we used analog telephones (i.e., under the assumption our communication could be easily intercepted) and not pass privileged information over it. Otherwise, I imagine it would be a non-issue. Given changing laws on cryptography (i.e., state-required backdoors & etc.), we may be back in that mindset before too long.
- mypastself 3y ago
  I used to think you could be a 9/11 truther and still hold reasonable beliefs otherwise. I thought a lot of these people genuinely looked at the official story and related evidence with an open mind, and came to a different conclusion than I did. But the more of them I’ve met, the more I’ve found they tend to be everything-else truthers as well: moon landing, vaccines, etc., and I tend to disregard their arguments outright. Is that fair? Maybe not. But it’s borne out of years of similar interactions, and it’s saved me a lot of time and effort. Lately I’ve started acting similarly with doomers, almost inadvertently. If you think that e.g. climate change will make humanity go extinct in the next few decades, it’s likely you feel similarly about A.I., COVID, etc. I’m starting to shut out such claims immediately, or at least engage with them less. There seem to be certain personality types who gravitate towards certain types of beliefs. I’m not qualified to speculate as to why they do, but sometimes simply ignoring them can be… sensible.
  - solardev 3y ago
    It saddens me how right you are. Things have become so polarized and tribal that there isn't much room for nuance. Can't climate change be a big deal without being an extinction risk? Covid was deadly, but mostly to certain demographics. AI is dangerous in the way that the Manhattan Project was dangerous, but maybe that's not sufficient reason to stop pursuing it. Etc. But I think people are so overwhelmed by this hyperconnected and ubersensationalized world, flooded with ethical conundrums daily, that they only have the bandwidth for snap judgments. Into this bucket or the other you go, thought, no time or energy for a closer examination!

maremmano 3y ago

Why I feel that China will not slow down AI progress?

ShamelessC 3y ago
They won’t but mainland China hasn’t shown a very good track record of keeping up, much less inventing new methods that gain any sort of traction. They do seem to produce plenty of good researchers, who sometimes proceed to do good research with proper funding from western institutions. I would be more worried about Europe.
- boringuser2 3y ago
  [flagged]
  - fsociety 3y ago
    That seems like an overly broad and unfair generalization of Chinese.
    - boringuser2 3y ago
      Every generalization is "overly broad". That's why it's called a "generalization". As for whether the claim is fair or not, that's an opinion, but I disagree, obviously.
  - solardev 3y ago
    Um, are we just skipping past the whole colonialist era and straight into some sort of enlightened modern age? Even then, Russia's European too. China's had a rough and tumble history, to be sure, but they didn't try to take over the whole world. Just Tibet and Hong Kong so far, maybe Taiwan someday. I think their ambitions are different, and so are their ethical frameworks. The CCP isn't a governing body I admire, but their people and culture have a very long history of collectivism and Confucianism. The flip side of Han supremacy and cultural erasure is national unity and civil harmony, their version of manifest destiny I guess, with an emphasis on assimilation. I envision a Great Chinese AI to be in line with the firewall, ostensibly (and mostly) uses to promote harmony via strict control, rather than conquest. Maybe more Borg than Skynet?

snorkel 3y ago

I wonder how deeply the board and the rotating CEOs actually understand the technology and practicality of achieving AGI, or whether they are falling over themselves due to buying into the LLM hype.

datpiff 3y ago

If they try to keep the same pace of LLM advancement to preserve a high valuation, they might fail - LLM training costs skyrocket and it will get harder and harder to get the data. The CEO announcing a slow-down could likely be a strategy not to spook investors.

shortsunblack 3y ago

No, AI is not going to kill us anytime soon. Yes, LLMs are not even close to AGI and the research on them does not lead to research on understanding cognition (read Chomsky on this), which is required for AGI. Yes, DO ban all AI research and investment until society as a whole, by consensus, grants the privilege of continuing. OpenAI saga already has shown Sillicon Valley tech culture is in no fit shape to be involved in such manners in any form. The culture that gave us Musk buying Twitter and torpedoing its value by half the next day and the shitcoin scammer SBF does not have a good track record.

thfuran 3y ago
>the research on them does not lead to research on understanding cognition (read Chomsky on this), which is required for AGI. Many phenomena have been discovered unintentionally and/or without understanding them. Deliberately developing an agi would probably be easier with a better understanding of cognition, but it's almost certainly not required.
dcwca 3y ago
With all due respect, Chomsky has no intuition for this kind of technology. Was sad to see him reject the technology, felt oddly like sour grapes when really LLMs have validated his work on universal grammar.
chpatrick 3y ago
> No, AI is not going to kill us anytime soon. > Yes, DO ban all AI research and investment So do you think it's a threat or not?
- agevag 3y ago
  [dead]
  - wholinator2 3y ago
    I think it's a Stretch to assert that for the GP. Where did they bring that kind of politics into their view or what led you to think that they, or anyone, thinks that

elzbardico 3y ago

The next few years are going to be interesting for lawyers. I imagine all kinds of lawsuits including, but not limited to, stealing of trade secrets are going to started by OpenAI foundation against Sam's new employers and all the retaliatory counter-suing.

elzbardico 3y ago

Killing us in a nuclear holocaust is for now a very far fetched hypothesis. I surely hope Nuclear C&C networks are properly air-gaped from the internet, and that actually launching the nukes require humans in the loop (yes, humans that could be socially engineered by an uber smart AGI, but then we are reaching). The immediate danger from UI is on the economic and political realm. Reinforcing wealth-concentration, monopolies, all powerful surveillance and propaganda. I care more about those immediate dangers, that are related to how humans that control the AI can use it for their benefit versus the rest of us, than to the distant theoretical scenario of an AI going crazy and deciding to go all skynet on US. I fear the AI owners, they are going to fuck us way before the AI has any idea about fucking humans.

dtagames 3y ago

This hire makes is clear why MS is taking the IP and breaking ranks w/ OpenAI. He wants to slow down AI to development to 1/10th of its current pace? You can't run a company like that. He could run a library research program but that's about it. MS doesn't want to slow down AI; they want to own it. Many people realize that AGI isn't a thing and LLMs will never get there. Computers lack volition, but corporations have in it spades.