8 ms·

This piece frames this as a debate between broad camps of AI makers, but in my experience both the accelerationist and doomer sides are basically media/attentio

This piece frames this as a debate between broad camps of AI makers, but in my experience both the accelerationist and doomer sides are basically media/attention economy phenomena -- narratives wielded by those who know the power of compelling narratives in media. The bulk of the AI researchers, engineers, etc I know kind of just roll their eyes at both. We know there are concrete, mundane, but important application risks in AI product development, like dataset bias and the perils of imperfect automated decision making, and it's a shame that tech-weak showmen like Musk and Altman suck up so much discursive oxygen.

zamfi 3y ago
> it's a shame that tech-weak showmen like Musk and Altman suck up so much discursive oxygen Is it that bad, though? It does mean there's lots of attention (and thus funding, etc.) for AI research, engineering, etc. -- unless you are expressing a wish that the discursive oxygen were instead spent on other things. In which case, I ask: what things?
- sonicanatidae 3y ago
  What things? The pauses to consider if we should do <action>, before we actually do <action>. Tesla's "Self-Driving" is an example of too soon, but fuck it, we gots PROFITS to make and if a few pedestrians die, we'll just throw them a check and keep going. Imagine the trainwreck caused by millions of people leveraging AI like the SCOTUS lawyers, where their brief was written by AI and noted imagined cases in support of its decision. AI has the potential to make great change in the world, as the tech grows, but it's being guided by humans. Humans aren't known for altruism or kindness. (source: history) and now we're concentrating even more power into fewer hands. Luckily, I'll be dead long before AI gets crammed into every possible facet of life. Note that AI is inserted, not because it makes your life better, not because the world would be a better place for it and not even to free humans of mundane tasks. Instead it's because someone, somewhere can earn more profits, whether it works right or not and humans are the grease in the wheels.
  - pixl97 3y ago
    >The pauses to consider if we should do <action>, before we actually do <action>. Unless there has been an effective gatekeeper, that's almost never happened in history. With nuclear the gatekeeper is it's easy to detect. With genetics there pretty universal revulsion to it to the point a large portion of most populations are concerned about it. But with AI, to most people it's just software. And pretty much it is, if you want a universal ban of AI you really are asking for authoritarian type controls on it.
    - JoshTriplett 3y ago
      > But with AI, to most people it's just software. Practical AI involves cutting-edge hardware, which is produced in relatively few places. AI that runs on a CPU will not be a danger to anyone for much longer. Also, nobody's asking for a universal ban on AI. People are asking for an upper bound on AI capabilities (e.g. number of nodes/tokens) until we have widely proven techniques for AI alignment. (Or, in other words, until we have the ability to reliably tell AI to do something and have it do that thing and not entirely different and dangerous things).
      - pixl97 3y ago
        Right, and when I was a kid computers were things that fit on entire office floors. If your 'much longer' is only 30-40 years I could still be around then. In addition you're just asking for limits on compute, which ain't gonna go over well. How do you know if it's running a daily weather model, or making an AI. And how do you even measure capabilities when we're coming out with with other functions like transformers that are X times more efficient. What you want with AI cannot happen. If it's 100% predictable it's a calculation. If it's a generalization function taking incomplete information (something humans do) it will have unpredictable modes.
  - pc86 3y ago
    Is a Tesla FSD car a worse driver than a human of median skill and ability? Sure we can pull out articles of tragedies, but I'm not asking about that. Everything I've seen points to cars being driven on Autopilot being quite a bit safer than your average human driver, which is admittedly not a high bar, but I think painting it as "greedy billionaire literally kills people for PROFITS" is at best disingenuous to what's actually occurring.
- permanent 3y ago
  It is very bad. There's more money and fame to be made by taking these two extreme stances. The media and the general public is eating up this discourse, that are polarizing the society, instead of educating. > What things? There are helpful developments and applications that go unnoticed and unfunded. And there are actual dangerous AI practices right now. Instead we talk about hypotheticals.
  - zamfi 3y ago
    Respectfully, I don't think it's AI hype that is "polarizing the society".
- heyitsguay 3y ago
  They're talking about shit that isn't real because it advances their personal goals, keeps eyes on them, whatever. I think the effect on funding is overhyped -- OpenAI got their big investment before this doomer/e-acc dueling narrative surge, and serious investors are still determining viability through due diligence, not social media front pages. Basically, it's just more self-serving media pollution in an era that's drowning in it. Let the nerds who actually make this stuff have their say and argue it out, it's a shame they're famously bad at grabbing and holding onto the spotlight.
  - pixl97 3y ago
    Just to play devils advocate to this type of response. What if tomorrow I drop a small computer unit in front of you that has human level intelligence? Now, you're not allowed to say humans are magical and computers will never do this. For the sake of this theoretical debate it's already been developed and we can make millions of them. What does this world look like?
    - AnimalMuppet 3y ago
      > What does this world look like? It looks imaginary. Or, if you prefer, it looks hypothetical. The point isn't how we would respond if this were real. The point is, it isn't real - at least not at this point in time, and it's not looking like it's going to be real tomorrow, either. I'm not sure what purpose is served by "imagine that I'm right and you're wrong; how do you respond"?
      - pixl97 3y ago
        Thank god you're not charge of military planning. "Hey the next door neighbors are spending billions on a superweapon, but don't worry, they'll never build it"
        RandomLensman 3y ago
        On some things that is not a bad position: The old SDI had a lot of spending but really not much to show for it while at the same time forcing the USSR into a reaction based on what today might be called "hype".
- fallingknife 3y ago
  Very bad. The Biden admin is proposing AI regulation that will protect large companies from competition due to all the nonsense being said about AI.
  - jazzyjackson 3y ago
    Alternatively: there is nonsense being said about AI so that the Biden admin can protect large companies from competition
    - AlexandrB 3y ago
      Yup. I continue to be convinced that a lot of the fearmongering about rogue AI taking over the world is a marketing/lobbying effort to give early movers in the space a leg up. The real AI harms are probably much more mundane - such as flooding the internet with (even more) low quality garbage.
  - dragonwriter 3y ago
    > The Biden admin is proposing AI regulation that will protect large companies from competition Mostly, the Biden Administration is proposing a bunch of studies by different agencies of different areas, and some authorities for the government to take action regarding AI in some security-related areas. The concrete regulation mostly is envisioned to be drafted based on the studies, and the idea that it will be incumbent protective is mostly based on the fact that certain incumbents have been pretty nakedly tying safety concerns to proposals to pull up the ladder behind themselves. But the Administration is, at a minimum, resisting the lure of relying on those incumbents presentation of the facts and alternatives out of the gate, and also taking a more expansive view of safety and related concerns than the incumbents are proposing (expressly factoring in some of the issues that they have used "safety" concerns to distract from), so I think prejudging the orientation of the regulatory proposals that will follow on the study directives is premature.
    - fallingknife 3y ago
      What I have heard from people I know in the industry is that the proposal they are talking about now is to restrict all models over 20 billion parameters. This arbitrary rule would be a massive moat to the few companies that have these models already.
pixl97 3y ago
The problem with humanity is we are really poor at recognizing all the ramifications of things when they happen. Did the indigenous people of north America recognize the threat that they'd be driven to near extinction in a few hundred years when a boat showed up? Even if they did, could they have done anything about it, the germs and viruses that would lead to their destruction had been quickly planted. Many people focus on the pseudo-religious connotations of a technological singularity instead of the more traditional "loss of predictability" definition. Decreasing predictability of the future state of the world stands to destabilize us far more likely than the FOOM event. If you can't predict your enemies actions, you're more apt to take offensive action. If you can't (at least somewhat) predict the future market state then you may pull all investment. The AI doesn't have to do the hard work here, with potential economic collapse and war humans have shown the capability to put themselves at risk. And the existential risks are the improbable ones. The "Big Brother LLM" where you're watched by a sentiment analysis AI for your entire life and if you try to hide from it you disappear forever are much more, very terrible, likelihoods.
- MichaelZuo 3y ago
  > The problem with humanity is we are really poor at recognizing all the ramifications of things when they happen. Zero percent of humanity can recognize "all the ramifications" due to the butterfly effect and various other issues. Some small fraction of bonafide super geniuses can likely recognize the majority, but beyond that is just fantasy.
  - pixl97 3y ago
    And by increasing uncertainty the super genius recognizes less...
- pishpash 3y ago
  That's already happening unfortunately. Voice print in call centers is pretty much omniscient, knowing your identity, age, gender, mood, etc. on a call. They do it in the name of "security", naturally. But nobody ever asked your permission other than to use the "your call may be recorded for training purposes" blanket one. (Training purposes? How convenient that models are also "trained"?) Anonymity and privacy can be eliminated tomorrow technologically. The only thing holding that back is some laziness and inertia. There is no serious pushback. You want to solve AI risk, there is one right here, but because there's an unchecked human at one end of a powerful machine, no one pays attention.
JohnFen 3y ago
Yes. I frequently get asked by laypeople about how likely I think adverse effects of AI are. My answer is "it depends on what risk you're talking about. I think there's nearly zero risk of a Skynet situation. The risk is around what people are going to do, not machines."
- ben_w 3y ago
  I don't know the risk of Terminator robots running around, but automatic systems on both USA and USSR (and post-Soviet Russian) systems have been triggered by stupid things like "we forgot the moon didn't have an IFF transponder" and "we misplaced our copy of your public announcement about planning a polar rocket launch".
  - JohnFen 3y ago
    Sure, but that's an automation problem, not an AI-specific one.
    - ben_w 3y ago
      Would you also argue that radon gas is fine because radiation is a radioisotope problem not a radon-specific one? The point of AI is to automate stuff.
  - pdonis 3y ago
    But the reason those incidents didn't become a lot worse was that the humans in the loop exercised sound judgment and common sense and had an ethical norm of not inadvertently causing a nuclear exchange. That's the GP's point: the risk is in what humans do, not what automated systems do. Even creating a situation where an automated system's wrong response is allowed to trigger a disastrous event because humans are taken out of the loop, is still a human decision; it won't happen unless humans who don't exercise sound judgment and common sense or who don't have proper ethical norms make such a disastrous decision. My biggest takeaway from all the recent events surrounding AI, and in fact from the AI hype in general, including hype about the singularity, AI existential risk, etc., is that I see nobody in these areas who qualifies under the criteria I stated above: exercising sound judgment and common sense and having proper ethical norms.
concordDance 3y ago
Does Ilya count as a "tech-weak" showman in your book too?
twinge 3y ago
The media also doesn't define what it means to be a "doomer". Would an accelerationist with a p(doom) = 20% be a "doomer"?

zamfi 3y ago

> it's a shame that tech-weak showmen like Musk and Altman suck up so much discursive oxygen Is it that bad, though? It does mean there's lots of attention (and thus funding, etc.) for AI research, engineering, etc. -- unless you are expressing a wish that the discursive oxygen were instead spent on other things. In which case, I ask: what things?

sonicanatidae 3y ago
What things? The pauses to consider if we should do <action>, before we actually do <action>. Tesla's "Self-Driving" is an example of too soon, but fuck it, we gots PROFITS to make and if a few pedestrians die, we'll just throw them a check and keep going. Imagine the trainwreck caused by millions of people leveraging AI like the SCOTUS lawyers, where their brief was written by AI and noted imagined cases in support of its decision. AI has the potential to make great change in the world, as the tech grows, but it's being guided by humans. Humans aren't known for altruism or kindness. (source: history) and now we're concentrating even more power into fewer hands. Luckily, I'll be dead long before AI gets crammed into every possible facet of life. Note that AI is inserted, not because it makes your life better, not because the world would be a better place for it and not even to free humans of mundane tasks. Instead it's because someone, somewhere can earn more profits, whether it works right or not and humans are the grease in the wheels.
- pixl97 3y ago
  >The pauses to consider if we should do <action>, before we actually do <action>. Unless there has been an effective gatekeeper, that's almost never happened in history. With nuclear the gatekeeper is it's easy to detect. With genetics there pretty universal revulsion to it to the point a large portion of most populations are concerned about it. But with AI, to most people it's just software. And pretty much it is, if you want a universal ban of AI you really are asking for authoritarian type controls on it.
  - JoshTriplett 3y ago
    > But with AI, to most people it's just software. Practical AI involves cutting-edge hardware, which is produced in relatively few places. AI that runs on a CPU will not be a danger to anyone for much longer. Also, nobody's asking for a universal ban on AI. People are asking for an upper bound on AI capabilities (e.g. number of nodes/tokens) until we have widely proven techniques for AI alignment. (Or, in other words, until we have the ability to reliably tell AI to do something and have it do that thing and not entirely different and dangerous things).
    - pixl97 3y ago
      Right, and when I was a kid computers were things that fit on entire office floors. If your 'much longer' is only 30-40 years I could still be around then. In addition you're just asking for limits on compute, which ain't gonna go over well. How do you know if it's running a daily weather model, or making an AI. And how do you even measure capabilities when we're coming out with with other functions like transformers that are X times more efficient. What you want with AI cannot happen. If it's 100% predictable it's a calculation. If it's a generalization function taking incomplete information (something humans do) it will have unpredictable modes.
- pc86 3y ago
  Is a Tesla FSD car a worse driver than a human of median skill and ability? Sure we can pull out articles of tragedies, but I'm not asking about that. Everything I've seen points to cars being driven on Autopilot being quite a bit safer than your average human driver, which is admittedly not a high bar, but I think painting it as "greedy billionaire literally kills people for PROFITS" is at best disingenuous to what's actually occurring.
permanent 3y ago
It is very bad. There's more money and fame to be made by taking these two extreme stances. The media and the general public is eating up this discourse, that are polarizing the society, instead of educating. > What things? There are helpful developments and applications that go unnoticed and unfunded. And there are actual dangerous AI practices right now. Instead we talk about hypotheticals.
- zamfi 3y ago
  Respectfully, I don't think it's AI hype that is "polarizing the society".
heyitsguay 3y ago
They're talking about shit that isn't real because it advances their personal goals, keeps eyes on them, whatever. I think the effect on funding is overhyped -- OpenAI got their big investment before this doomer/e-acc dueling narrative surge, and serious investors are still determining viability through due diligence, not social media front pages. Basically, it's just more self-serving media pollution in an era that's drowning in it. Let the nerds who actually make this stuff have their say and argue it out, it's a shame they're famously bad at grabbing and holding onto the spotlight.
- pixl97 3y ago
  Just to play devils advocate to this type of response. What if tomorrow I drop a small computer unit in front of you that has human level intelligence? Now, you're not allowed to say humans are magical and computers will never do this. For the sake of this theoretical debate it's already been developed and we can make millions of them. What does this world look like?
  - AnimalMuppet 3y ago
    > What does this world look like? It looks imaginary. Or, if you prefer, it looks hypothetical. The point isn't how we would respond if this were real. The point is, it isn't real - at least not at this point in time, and it's not looking like it's going to be real tomorrow, either. I'm not sure what purpose is served by "imagine that I'm right and you're wrong; how do you respond"?
    - pixl97 3y ago
      Thank god you're not charge of military planning. "Hey the next door neighbors are spending billions on a superweapon, but don't worry, they'll never build it"
      - RandomLensman 3y ago
        On some things that is not a bad position: The old SDI had a lot of spending but really not much to show for it while at the same time forcing the USSR into a reaction based on what today might be called "hype".
fallingknife 3y ago
Very bad. The Biden admin is proposing AI regulation that will protect large companies from competition due to all the nonsense being said about AI.
- jazzyjackson 3y ago
  Alternatively: there is nonsense being said about AI so that the Biden admin can protect large companies from competition
  - AlexandrB 3y ago
    Yup. I continue to be convinced that a lot of the fearmongering about rogue AI taking over the world is a marketing/lobbying effort to give early movers in the space a leg up. The real AI harms are probably much more mundane - such as flooding the internet with (even more) low quality garbage.
- dragonwriter 3y ago
  > The Biden admin is proposing AI regulation that will protect large companies from competition Mostly, the Biden Administration is proposing a bunch of studies by different agencies of different areas, and some authorities for the government to take action regarding AI in some security-related areas. The concrete regulation mostly is envisioned to be drafted based on the studies, and the idea that it will be incumbent protective is mostly based on the fact that certain incumbents have been pretty nakedly tying safety concerns to proposals to pull up the ladder behind themselves. But the Administration is, at a minimum, resisting the lure of relying on those incumbents presentation of the facts and alternatives out of the gate, and also taking a more expansive view of safety and related concerns than the incumbents are proposing (expressly factoring in some of the issues that they have used "safety" concerns to distract from), so I think prejudging the orientation of the regulatory proposals that will follow on the study directives is premature.
  - fallingknife 3y ago
    What I have heard from people I know in the industry is that the proposal they are talking about now is to restrict all models over 20 billion parameters. This arbitrary rule would be a massive moat to the few companies that have these models already.

pixl97 3y ago

The problem with humanity is we are really poor at recognizing all the ramifications of things when they happen. Did the indigenous people of north America recognize the threat that they'd be driven to near extinction in a few hundred years when a boat showed up? Even if they did, could they have done anything about it, the germs and viruses that would lead to their destruction had been quickly planted. Many people focus on the pseudo-religious connotations of a technological singularity instead of the more traditional "loss of predictability" definition. Decreasing predictability of the future state of the world stands to destabilize us far more likely than the FOOM event. If you can't predict your enemies actions, you're more apt to take offensive action. If you can't (at least somewhat) predict the future market state then you may pull all investment. The AI doesn't have to do the hard work here, with potential economic collapse and war humans have shown the capability to put themselves at risk. And the existential risks are the improbable ones. The "Big Brother LLM" where you're watched by a sentiment analysis AI for your entire life and if you try to hide from it you disappear forever are much more, very terrible, likelihoods.

MichaelZuo 3y ago
> The problem with humanity is we are really poor at recognizing all the ramifications of things when they happen. Zero percent of humanity can recognize "all the ramifications" due to the butterfly effect and various other issues. Some small fraction of bonafide super geniuses can likely recognize the majority, but beyond that is just fantasy.
- pixl97 3y ago
  And by increasing uncertainty the super genius recognizes less...
pishpash 3y ago
That's already happening unfortunately. Voice print in call centers is pretty much omniscient, knowing your identity, age, gender, mood, etc. on a call. They do it in the name of "security", naturally. But nobody ever asked your permission other than to use the "your call may be recorded for training purposes" blanket one. (Training purposes? How convenient that models are also "trained"?) Anonymity and privacy can be eliminated tomorrow technologically. The only thing holding that back is some laziness and inertia. There is no serious pushback. You want to solve AI risk, there is one right here, but because there's an unchecked human at one end of a powerful machine, no one pays attention.

JohnFen 3y ago

Yes. I frequently get asked by laypeople about how likely I think adverse effects of AI are. My answer is "it depends on what risk you're talking about. I think there's nearly zero risk of a Skynet situation. The risk is around what people are going to do, not machines."

ben_w 3y ago
I don't know the risk of Terminator robots running around, but automatic systems on both USA and USSR (and post-Soviet Russian) systems have been triggered by stupid things like "we forgot the moon didn't have an IFF transponder" and "we misplaced our copy of your public announcement about planning a polar rocket launch".
- JohnFen 3y ago
  Sure, but that's an automation problem, not an AI-specific one.
  - ben_w 3y ago
    Would you also argue that radon gas is fine because radiation is a radioisotope problem not a radon-specific one? The point of AI is to automate stuff.
- pdonis 3y ago
  But the reason those incidents didn't become a lot worse was that the humans in the loop exercised sound judgment and common sense and had an ethical norm of not inadvertently causing a nuclear exchange. That's the GP's point: the risk is in what humans do, not what automated systems do. Even creating a situation where an automated system's wrong response is allowed to trigger a disastrous event because humans are taken out of the loop, is still a human decision; it won't happen unless humans who don't exercise sound judgment and common sense or who don't have proper ethical norms make such a disastrous decision. My biggest takeaway from all the recent events surrounding AI, and in fact from the AI hype in general, including hype about the singularity, AI existential risk, etc., is that I see nobody in these areas who qualifies under the criteria I stated above: exercising sound judgment and common sense and having proper ethical norms.

concordDance 3y ago

Does Ilya count as a "tech-weak" showman in your book too?

twinge 3y ago

The media also doesn't define what it means to be a "doomer". Would an accelerationist with a p(doom) = 20% be a "doomer"?