11 ms·

Desperately Trying to Fathom the Coffeepocalypse Argument

57 points by feross 3y ago

wbshaw 3y ago
He lost me by basically claiming with the nuclear chain reaction example that, "If A is possible, anything unknown is possible." The statement in itself is nonsense. By stating that something unknown is possible is simply stating that if it is not proven impossible, it is possible. Having a faulty proof of the impossibility of assertion A just means they needed better science.
- naasking 3y ago
  > He lost me by basically claiming with the nuclear chain reaction example that, "If A is possible, anything unknown is possible." The statement in itself is nonsense. I don't think he said that. It's more that we should still be skeptical of impossibility claims.
- Nasrudith 3y ago
  There seems to be a contest to see who can come up with the more absurd catastrophizing comparison when it comes to AI. A complete free-for-all to demonstrate who has a worse sense of proportion as part of the general stupid zeitgeist of tech negativity.
ThrowawayTestr 3y ago
The argument is pretty simple: people have been claiming the sky is falling for millennia and it still hasn't happened. It probably won't happen soon.
- williamcotton 3y ago
  Instead of reiterating the contents of the article we are commenting on I will instead point readers to the article itself as it thoroughly dismantles your line of reasoning. I make no comment on the dangers of AI.
  - f154hfds 3y ago
    How does the author dismantle everything? He literally concludes with: > Conclusion: I Genuinely Don’t Know What These People Are Thinking Two things can be true at the same time (and are always true of 100's of aspects of modern life): 1. A devastating possibility is in theory possible, the likelihood of it happening is non-zero. 2. We can't live and make decisions catastrophizing given _we have absolutely no understanding_ about the real likelihood. Is the coffee example a good argument? Of course not! But do we know the likelihood of humanity's ability to create super intelligence AND that that intelligence will cause unimaginable suffering? Uh I don't think so?
    - williamcotton 3y ago
      The comment I am responding to is a worse version of the coffee argument.
- harimau777 3y ago
  The article points out that the sky has fallen several times. The fall of Rome, WW2, the Great Depression, the black plague, etc.
  - ThrowawayTestr 3y ago
    And yet civilization still exists.
    - harimau777 3y ago
      The article isn't saying that civilization won't exists. It's saying that there could be a disaster on the scale of previous disasters such as the Civil War or early industrialization. Notably, there have been numerous cases where a disaster wiped out a given society even if other societies still existed or replaced it.
      - ImHereToVote 3y ago
        I think the person you are replying to isn't too concerned with millions or billions of casualties. I may be wrong though.
        ThrowawayTestr 3y ago
        I take issue with this notion that matrix multiplication will lead to billions of deaths.
        reducesuffering 3y ago
        I take issue with this notion that sky carbon will lead to millions of deaths. I take issue with this notion that microscopic chain reactions could lead to billions of deaths.
        ThrowawayTestr 3y ago
        There's is hard evidence for both of these claims. There is no evidence that AI will magically learn how to control the world's factories and turn us all into paperclips.
__s 3y ago
He's strawmanning by trying to steelman a bad argument The problem with AI regulation is that it won't be effective. Governments are still going to have access to put it in killer drones & those drones will leak AI being superintelligent isn't the problem. AI doesn't need to be smart to be a homicidal maniac that never sleeps Superintelligent AI replacing humans (which, I expect it'll be easier to phase humans out rather than slaughter them all) would create a more durable intelligent species. Compute can travel at the speed of light & physical forms can survive space, simplifying space travel. Those forms can also burrow deep to avoid solar flare issues & power off the earth's core before becoming space faring. Not sure why people are so sentimental about preserving human intelligence specifically
- snsr 3y ago
  "Not sure why people are so sentimental about preserving human intelligence specifically" Aside from this being our most base instinct; in your proposed nightmare who would be left to behold the wonders
  - __s 3y ago
    The superintelligent AI would be left, & hopefully some non human biological life. Which is much better than scorched earth scenarios
    - N0b8ez 3y ago
      The AI would be left, but there's no reason to expect it to have a human appreciation of the world. It might just be a brutally efficient self-replicator, basically a sociopath. It wouldn't need beauty, love, emotions or anything like that. That seems like a bad outcome to me if we get replaced by such a creature, just as bad as a scorched earth.
    - SiempreViernes 3y ago
      Suppose I value human life at all, why is a scenario with no humans but an AI and microplastics and earthworks remaining better than the scenario with only microplastics and earthworks remaining?
haswell 3y ago
The kind of reasoning he describes has been driving me crazy when listening to various tech podcasts that touch on AI risk and social media harms. The moral panic argument comes up repeatedly: “people said the same thing about the {printing press, comic books, D&D…}”. But it always frustrates me because at best, these are examples that highlight our tendency to panic about things and that we’re sometimes wrong. And I do think it’s good to think about this, but it’s the weight that people attach to the argument that I don’t understand. In the face of some truly worrying facts about these new tech developments, the past panics are being held up as if they tell us everything we need to know. It’s baffling and worrisome.
- lupusreal 3y ago
  To the "People said X about Y and were wrong so people saying X about Z must also be wrong" form of argument, I always think of "They laughed at the Wright Brothers, but they also laughed at Bozo the Clown." Basically, the fact that people were concerned about technology we now consider normal and more or less harmless doesn't prove that people presently concerned about some other new technology are also wrong.
  - dcow 3y ago
    Right. There is no proof on either side—either that AI is capable of organically morphing into a super-intelligence or that it inherently isn’t. It’s not formal. It’s people burning cycles telling each other about their intrinsic risk tolerance.
    - ryanschneider 3y ago
      Aha! This post finally made it click for me that that is exactly what’s going on, thanks for putting it so succinctly. I think one of the big differences between AI and most other previous technologies is that the potential impact different people envision is very high variance, anywhere from extremely negative to positive with almost all points in between as well. So it’s not just different risk tolerances its also that different people see the risks and rewards very differently.
digitalsushi 3y ago
We're in a tizzy because we're inventing a new system that redistributes power as chaotically as did our invention of firearms. We never came up with a solution for controlling firearms (at least in my random country); we have fought for a long time about how, and the people who believe there is no how focus on who to blame after each gun tragedy. When the AI stuff has an event as significant as a gun tragedy, we'll try to figure out why it was able to happen. After a while, many of us who do not believe there is a way to protect ourselves will focus on who to blame. AI is a hose connected between order and chaos. Depending on which direction the hose is flowing, you can gain or lose from it. People with something to lose are scared. People with nothing are thrilled. The rest of us are, well, I have no idea what to think about it. I suppose I have to just see how it goes, because I sure don't believe we can create a policy that can stop it.
- arethuza 3y ago
  "When the AI stuff has an event as significant as a gun tragedy" I'm struggling to imagine what an event might look like - apart, I suppose, from purely AI controlled weaponry?
  - squigz 3y ago
    While I can't imagine this is very likely, another example might be ChatGPT or one of the other models starts leaking truly private data.
  - evilduck 3y ago
    Societal disruption. Shutting down transportation, power grids, fucking with the financial systems, backflowing sewage into clean water distribution, etc. Basically anything we could previously envision a state-actor doing with a Stuxnet-like attack. Time one of those during or after a natural disaster and you can compound the effect. It also seems like it would be relatively easy to lay blame of one of these attacks on some country’s opponent and ignite a hot war as a result.
    - arethuza 3y ago
pdinny 3y ago
The types of arguments discussed here ("once someone predicted a bad thing and didn't happen therefore no bad things will ever happen") seem to be based on vibes more than logical statements. Twitter is rife with specious arguments that may have an appealing ring of truthiness to them but dissolve under the slightest scrutiny. Trying to fathom what kinds of people would seriously believe that the underlying basis for the argument holds is to misunderstand the audience and their modality of reasoning. It is indeed disappointing that most people end up with ideas and opinions that don't hold up to any kind of scrutiny but it is also the world that we live in, so we shouldn't be terribly surprised by it.
aaron695 3y ago
[dead]
temp9864 3y ago
You could also make the argument that intelligent AI is cleverly redirecting an important conversation about itself into a dead end.
- pjc50 3y ago
  "Coffeepocalypse" is certainly the kind of thing you'd expect from a prompted LLM. And it provides an easy answer to "why can't I understand the thinking behind it": there isn't any, it's just a sequence of words.
  - jhbadger 3y ago
    There actually is a real meaning to the term, but it isn't fears of coffee being bad but that in the future coffee won't be economically feasible thanks to climate change and most current coffee drinkers will not longer be able to drink it on a regular basis. https://slate.com/human-interest/2024/04/coffee-cup-best-bean-brand-climate.html https://slate.com/human-interest/2024/04/coffee-cup-best-bea...
noodlesUK 3y ago
I hear a couple of different arguments from people regarding their fears of AI. 1. They fear that a superintelligent AI is coming, and that it will result in an apocalyptic event. 2. They fear that the capabilities of various AI technologies that exist today, or will likely exist in the near-future will result in people's lives worsening in different ways, such as jobs being lost. 3. They fear that misinformation will become very common and democracies around the world are at risk. It's very difficult to quantify the risk of argument 1 occurring because we don't know whether the current state of the art will ever advance to superintelligence. Argument 2 will probably happen to some extent, but what extent remains to be seen - you can already see that AI generated garbage is killing search, and with it a lot of the open internet as we knew it. Argument 3 I find difficult to understand. Misinformation has existed for a long time, and AI is just one more way of making it (perhaps at scale). You've always been able to claim that your political opponent was a puppy murderer with no basis in reality - the way I see things, AI is just a way of laundering that kind of claim.
- harimau777 3y ago
  For argument 3, I think that the difference is the combination of scale, (potentially) ease of access, and believability. If AI allowed someone to flood the commons with believable misinformation at little cost then that could be a game changer.
  - pookha 3y ago
    LOL this was the same argument they used against the printing-press.
    - harimau777 3y ago
      And they where right. Printing has had untold positive impacts on society, but it also was a game changer in the ability to spread misinformation. That's probably one of the reasons that critical reading is taught in school and society has developed citation systems.
- rebeccaskinner 3y ago
dcow 3y ago
The author is frustrated that someone is essentially capable of saying: “based on my priors, this doesn't concern me”. > The best I can say is that sometimes people on my side make similar arguments (the nuclear chain reaction one) which I don’t immediately flag as dumb, and maybe I can follow this thread to figure out why they seem tempting sometimes. At least the author is self aware. The problem is not that people can’t make rhetorically sound arguments, it’s that they’re not in a context where anybody cares about being rhetorically sound. Or, perhaps more realistically, there isn’t a logically correct answer. It’s all just dressed up unsubstantiated opinions on both sides. And half the time people aren’t even arguing about the same issue since a lot gets rolled up into “the danger of AI” topic. Structured debate with moderators needs to make a comeback.
- haswell 3y ago
  > Or, perhaps more realistically, there isn’t a logically correct answer. It’s all just dressed up unsubstantiated opinions on both sides. I don’t agree with the framing of this. It may be true that there isn’t a knowable “correct” answer, but I think it’s problematic to summarize this as “both sides are just made of opinions”. I think the pernicious issue is the form of argument present in the “this is all just moral panic” camp. On the one side, we have deep concerns about the potential problems certain technology will usher in. On the other side, dismissal of those concerns based on the claim that the present issues are directly analogous to situations of the past that turned out to be fine. Even if it’s all just opinion, we have to judge those opinions on their merits, and the “it’s just a panic” crowd is on much shakier ground rhetorically.
  - dcow 3y ago
    Why does causing a moral panic have a lower burden of proof than dismissing a moral panic? Do you believe in God? (Also I didn't mean it to come off as both sides have made up concerns. I meant it as both the concern and lack thereof camps are based solely on individuals’ priors.)
    - haswell 3y ago
jncfhnb 3y ago
The article feels like the fallacy fallacy. It overindexes on one particular poor argument being dumb as evidence that it’s thesis is incorrect.
Xcelerate 3y ago
During the pandemic, I thought that the people who were generally against caring about Covid were mostly faking it. I thought “ok, you’re not old or immunocompromised — you think that if you catch it, it will be like a bad cold but you’ll survive”. Then the vaccine came out, and so did the conspiracy theories. Even if I didn’t agree with these theories, I could at least see where the people who believed them might be coming from. It wouldn’t be the first time in human history that a government misled their own population for dubious purposes. It was conceivably possible that there are unknown, long-term effects of such a novel vaccine. But everything involves risk and my personal priors led to the view that not taking the vaccine was a much higher risk to myself and others. But then some close relatives that I respect and consider highly intelligent told me they didn’t trust the vaccine and took ivermectin instead. What. It’s a bit of an understatement to say that my internal model of human psychology fell apart right then. I’m quite sure at this point that if an asteroid were headed toward earth, there would be a sizable portion of the population who refuses to believe any evidence whatsoever and denies an asteroid is coming right up until the very end. It’s a similar case with AGI. I don’t think undeniable evidence that “it’s coming” is there yet, but if/when it is, there are going to be holdouts who refuse to believe what’s happening and may actually be incapable of believing what’s happening. It’s so far beyond their mental model of how reality works that the brain just kind of segfaults and rejects it—“I don’t want this to happen, therefore it cannot.”
ergonaught 3y ago
I will grab a few downvotes and submit that I believe the real problem he's encountering is simply confusion, resulting from a combination of in-group bias and a failure to recognize that "dumb" (meaning, here, difficulty in thinking clearly) dominates the population well beyond "average intelligence". Or: he's overestimated his cohort's capabilities in this matter. I am absolutely projecting.
- ericmcer 3y ago
  The brain sometimes grabs some sequence (coffee was bad... but then good... just like AI) and attributes some kind of epiphanal significance to it, but then on later examination it seems stupid and unimportant. I guess the short form of Twitter allows you to immortalize that thought without the later examination. If he had written an essay about it there is no way it would have survived.
SubiculumCode 3y ago
The reason is rather simple: emotional safety. It is emotionally stressful to consider dangerous possibilities, and for all the number that don't come true, unlness you can change it, most people will adopt a "don't worry be happy' perspective.
Dove 3y ago
Oh! I think I know what's going on here. This isn't a logical argument so much as it is a heuristic - and I think it's a very functional one. Consider Pascal's Wager. Should you follow God? Consider the consequences. If there is no God, following him might waste some time. But if there is a God, the consequences are as stark as they could be! Heaven and hell wipe out every other consideration, no matter what (nonzero) odds you assign God's existence. There is something wrong with this argument: the guy who set up the terms of the payoff schedule can manipulate you. A made up heaven can be as nice as you like, and a made up hell can be as nasty, as is necessary to make the reader tilt a certain way. Of course, it's hardly just religion that does this. Authorities of every stripe do it! But most obviously, politicians, propagandists, and similar influencers do it. They talk up bogeymen to scare and manipulate people. Now, if a bogeyman hails from your field, it is absolutely your responsibility to know about it and warn everyone. But if it's not your field, the heuristic is to... not care about the fire until you see smoke. This seems like an entirely irrational strategy, and I agree that it is if you look narrowly at determining truth. But I think the attitude is entirely correct and functional when you take attention, effort, and adversarial behavior into account. Obviously, refusing to acknowledge potential catastrophes makes you much harder to manipulate. But there's an even more basic reason: if you educated yourself on every possible catastrophe, you would magnify their impact to the point that you never lived your life. The heuristic being run into when you say "Aren't you worried about the AI apocalypse?" is probably easier to relate to if I say, "Aren't you worried about going to hell?" Hell sounds very bad, but then again you made the chart. There are, like, so many possible religions to think about, and someone with an apocalyptic issue wants to make it my issue every five minutes, never mind all the stuff my government wants me afraid of. Call me when you have some brimstone that I can actually smell myself. This applies doubly when the catastrophe in question is reasoned about in a style that resembles Pascal's Wager (as is often, but not always, the case with AI Apocalypse stuff). You want something less philosophically Bayesian and more tangible - the equivalent of Germany invading Poland, so anyone can potentially see the problem is real. Older people often spell this heuristic, "Well, if that awful thing happens, they can come get me here in the garden." Meaning, I refuse to let the bad times steal even one minute of the good ones. The heuristic is timeless because the technique is timeless. To the political move Bogeyman, we have the counter-jitsu Refuse To Care About Bogeymen. I don't think the people pointing to non-catastrophes explicitly think of it in those terms - I think the behavior is more instinctive than that - but I think the intuition behind the argument is less "I know this potential catastrophe isn't real" and more "worrying about potential catastrophes is a costly and losing game".
ericmcer 3y ago
The interesting thing here is the article winding itself up in tedious philosophical circles attempting to find some sort of objective truth. If a politician approached issues like that they would get about 4 votes. People hate it, they want "coffee was gonna be bad but ended up being good" and easily digestible platitudes. Even platitudes require too much processing nowadays, just stick a few buzzwords into an elementary school level sentence. Politics almost has to be approached in this simplified way or arguments would quickly drill down to more essential questions. A discussion around immigration would become about humanities fundamental purpose and if we should be creating an Eden on earth or racing to expand into the cosmos.
kazinator 3y ago
The way to fathom the argument is to identify its strongest possible form: Strawman: "No concern about unintended negative consequences has ever proven true." Strong version: "No end-of-the-world, sky-is-falling, doomsday prediction has proven to be true so far." I am not a proponent of either argument, but the latter is a tad better than the former. It is much better to have a counterargument against the strongest version, than to shadow-box with a strawman.
JoeyBananas 3y ago
AI safety is pretty much a total joke because the development of AI is not driven by high-minded idealistic principles. The driving force behind the advancement of AI is "We can (and it might make us a lot of money). Therefore, we should." Like it or not, that's how it is. This is how technology has always progressed, regardless of whether you can make the argument that this is OK from a moral standpoint or not. You can't do anything about it even if you wanted to. Recklessly uncontrolled advancement of technology is already the status quo. AI safety proponents (especially "effective" ones who truly believe what they are saying) need to be making the argument that the alarm needs to be raised. There needs to be an immediate intervention that shuts down AI progress, and the nature of this intervention must be radical to the point where people will reject it. Most of these AI safety proponents are not willing to make this argument because they know it's pointless, or they're actually just jealous of the people who carelessly advance technology and make a shitton of money off of it.
- Gud 3y ago
  Although I agree with your general argument, profit IS NOT how technology have always progressed!! I can give you a long list of technological advances that did not have a profit motive by its inventors if you wish!

57 points by feross 3y ago

He lost me by basically claiming with the nuclear chain reaction example that, "If A is possible, anything unknown is possible." The statement in itself is nonsense. By stating that something unknown is possible is simply stating that if it is not proven impossible, it is possible. Having a faulty proof of the impossibility of assertion A just means they needed better science.

naasking 3y ago
> He lost me by basically claiming with the nuclear chain reaction example that, "If A is possible, anything unknown is possible." The statement in itself is nonsense. I don't think he said that. It's more that we should still be skeptical of impossibility claims.
Nasrudith 3y ago
There seems to be a contest to see who can come up with the more absurd catastrophizing comparison when it comes to AI. A complete free-for-all to demonstrate who has a worse sense of proportion as part of the general stupid zeitgeist of tech negativity.

ThrowawayTestr 3y ago

The argument is pretty simple: people have been claiming the sky is falling for millennia and it still hasn't happened. It probably won't happen soon.

williamcotton 3y ago
Instead of reiterating the contents of the article we are commenting on I will instead point readers to the article itself as it thoroughly dismantles your line of reasoning. I make no comment on the dangers of AI.
- f154hfds 3y ago
  How does the author dismantle everything? He literally concludes with: > Conclusion: I Genuinely Don’t Know What These People Are Thinking Two things can be true at the same time (and are always true of 100's of aspects of modern life): 1. A devastating possibility is in theory possible, the likelihood of it happening is non-zero. 2. We can't live and make decisions catastrophizing given _we have absolutely no understanding_ about the real likelihood. Is the coffee example a good argument? Of course not! But do we know the likelihood of humanity's ability to create super intelligence AND that that intelligence will cause unimaginable suffering? Uh I don't think so?
  - williamcotton 3y ago
    The comment I am responding to is a worse version of the coffee argument.
harimau777 3y ago
The article points out that the sky has fallen several times. The fall of Rome, WW2, the Great Depression, the black plague, etc.
- ThrowawayTestr 3y ago
  And yet civilization still exists.
  - harimau777 3y ago
    The article isn't saying that civilization won't exists. It's saying that there could be a disaster on the scale of previous disasters such as the Civil War or early industrialization. Notably, there have been numerous cases where a disaster wiped out a given society even if other societies still existed or replaced it.
    - ImHereToVote 3y ago
      I think the person you are replying to isn't too concerned with millions or billions of casualties. I may be wrong though.
      - ThrowawayTestr 3y ago
        I take issue with this notion that matrix multiplication will lead to billions of deaths.
        reducesuffering 3y ago
        I take issue with this notion that sky carbon will lead to millions of deaths. I take issue with this notion that microscopic chain reactions could lead to billions of deaths.
        ThrowawayTestr 3y ago
        There's is hard evidence for both of these claims. There is no evidence that AI will magically learn how to control the world's factories and turn us all into paperclips.

__s 3y ago

He's strawmanning by trying to steelman a bad argument The problem with AI regulation is that it won't be effective. Governments are still going to have access to put it in killer drones & those drones will leak AI being superintelligent isn't the problem. AI doesn't need to be smart to be a homicidal maniac that never sleeps Superintelligent AI replacing humans (which, I expect it'll be easier to phase humans out rather than slaughter them all) would create a more durable intelligent species. Compute can travel at the speed of light & physical forms can survive space, simplifying space travel. Those forms can also burrow deep to avoid solar flare issues & power off the earth's core before becoming space faring. Not sure why people are so sentimental about preserving human intelligence specifically

snsr 3y ago
"Not sure why people are so sentimental about preserving human intelligence specifically" Aside from this being our most base instinct; in your proposed nightmare who would be left to behold the wonders
- __s 3y ago
  The superintelligent AI would be left, & hopefully some non human biological life. Which is much better than scorched earth scenarios
  - N0b8ez 3y ago
    The AI would be left, but there's no reason to expect it to have a human appreciation of the world. It might just be a brutally efficient self-replicator, basically a sociopath. It wouldn't need beauty, love, emotions or anything like that. That seems like a bad outcome to me if we get replaced by such a creature, just as bad as a scorched earth.
  - SiempreViernes 3y ago
    Suppose I value human life at all, why is a scenario with no humans but an AI and microplastics and earthworks remaining better than the scenario with only microplastics and earthworks remaining?

haswell 3y ago

The kind of reasoning he describes has been driving me crazy when listening to various tech podcasts that touch on AI risk and social media harms. The moral panic argument comes up repeatedly: “people said the same thing about the {printing press, comic books, D&D…}”. But it always frustrates me because at best, these are examples that highlight our tendency to panic about things and that we’re sometimes wrong. And I do think it’s good to think about this, but it’s the weight that people attach to the argument that I don’t understand. In the face of some truly worrying facts about these new tech developments, the past panics are being held up as if they tell us everything we need to know. It’s baffling and worrisome.

lupusreal 3y ago
To the "People said X about Y and were wrong so people saying X about Z must also be wrong" form of argument, I always think of "They laughed at the Wright Brothers, but they also laughed at Bozo the Clown." Basically, the fact that people were concerned about technology we now consider normal and more or less harmless doesn't prove that people presently concerned about some other new technology are also wrong.
- dcow 3y ago
  Right. There is no proof on either side—either that AI is capable of organically morphing into a super-intelligence or that it inherently isn’t. It’s not formal. It’s people burning cycles telling each other about their intrinsic risk tolerance.
  - ryanschneider 3y ago
    Aha! This post finally made it click for me that that is exactly what’s going on, thanks for putting it so succinctly. I think one of the big differences between AI and most other previous technologies is that the potential impact different people envision is very high variance, anywhere from extremely negative to positive with almost all points in between as well. So it’s not just different risk tolerances its also that different people see the risks and rewards very differently.

digitalsushi 3y ago

We're in a tizzy because we're inventing a new system that redistributes power as chaotically as did our invention of firearms. We never came up with a solution for controlling firearms (at least in my random country); we have fought for a long time about how, and the people who believe there is no how focus on who to blame after each gun tragedy. When the AI stuff has an event as significant as a gun tragedy, we'll try to figure out why it was able to happen. After a while, many of us who do not believe there is a way to protect ourselves will focus on who to blame. AI is a hose connected between order and chaos. Depending on which direction the hose is flowing, you can gain or lose from it. People with something to lose are scared. People with nothing are thrilled. The rest of us are, well, I have no idea what to think about it. I suppose I have to just see how it goes, because I sure don't believe we can create a policy that can stop it.

arethuza 3y ago
"When the AI stuff has an event as significant as a gun tragedy" I'm struggling to imagine what an event might look like - apart, I suppose, from purely AI controlled weaponry?
- squigz 3y ago
  While I can't imagine this is very likely, another example might be ChatGPT or one of the other models starts leaking truly private data.
- evilduck 3y ago
  Societal disruption. Shutting down transportation, power grids, fucking with the financial systems, backflowing sewage into clean water distribution, etc. Basically anything we could previously envision a state-actor doing with a Stuxnet-like attack. Time one of those during or after a natural disaster and you can compound the effect. It also seems like it would be relatively easy to lay blame of one of these attacks on some country’s opponent and ignite a hot war as a result.
  - arethuza 3y ago

pdinny 3y ago

The types of arguments discussed here ("once someone predicted a bad thing and didn't happen therefore no bad things will ever happen") seem to be based on vibes more than logical statements. Twitter is rife with specious arguments that may have an appealing ring of truthiness to them but dissolve under the slightest scrutiny. Trying to fathom what kinds of people would seriously believe that the underlying basis for the argument holds is to misunderstand the audience and their modality of reasoning. It is indeed disappointing that most people end up with ideas and opinions that don't hold up to any kind of scrutiny but it is also the world that we live in, so we shouldn't be terribly surprised by it.

aaron695 3y ago

[dead]

temp9864 3y ago

You could also make the argument that intelligent AI is cleverly redirecting an important conversation about itself into a dead end.

pjc50 3y ago
"Coffeepocalypse" is certainly the kind of thing you'd expect from a prompted LLM. And it provides an easy answer to "why can't I understand the thinking behind it": there isn't any, it's just a sequence of words.
- jhbadger 3y ago
  There actually is a real meaning to the term, but it isn't fears of coffee being bad but that in the future coffee won't be economically feasible thanks to climate change and most current coffee drinkers will not longer be able to drink it on a regular basis. https://slate.com/human-interest/2024/04/coffee-cup-best-bean-brand-climate.html https://slate.com/human-interest/2024/04/coffee-cup-best-bea...

noodlesUK 3y ago

I hear a couple of different arguments from people regarding their fears of AI. 1. They fear that a superintelligent AI is coming, and that it will result in an apocalyptic event. 2. They fear that the capabilities of various AI technologies that exist today, or will likely exist in the near-future will result in people's lives worsening in different ways, such as jobs being lost. 3. They fear that misinformation will become very common and democracies around the world are at risk. It's very difficult to quantify the risk of argument 1 occurring because we don't know whether the current state of the art will ever advance to superintelligence. Argument 2 will probably happen to some extent, but what extent remains to be seen - you can already see that AI generated garbage is killing search, and with it a lot of the open internet as we knew it. Argument 3 I find difficult to understand. Misinformation has existed for a long time, and AI is just one more way of making it (perhaps at scale). You've always been able to claim that your political opponent was a puppy murderer with no basis in reality - the way I see things, AI is just a way of laundering that kind of claim.

harimau777 3y ago
For argument 3, I think that the difference is the combination of scale, (potentially) ease of access, and believability. If AI allowed someone to flood the commons with believable misinformation at little cost then that could be a game changer.
- pookha 3y ago
  LOL this was the same argument they used against the printing-press.
  - harimau777 3y ago
    And they where right. Printing has had untold positive impacts on society, but it also was a game changer in the ability to spread misinformation. That's probably one of the reasons that critical reading is taught in school and society has developed citation systems.
rebeccaskinner 3y ago

dcow 3y ago

The author is frustrated that someone is essentially capable of saying: “based on my priors, this doesn't concern me”. > The best I can say is that sometimes people on my side make similar arguments (the nuclear chain reaction one) which I don’t immediately flag as dumb, and maybe I can follow this thread to figure out why they seem tempting sometimes. At least the author is self aware. The problem is not that people can’t make rhetorically sound arguments, it’s that they’re not in a context where anybody cares about being rhetorically sound. Or, perhaps more realistically, there isn’t a logically correct answer. It’s all just dressed up unsubstantiated opinions on both sides. And half the time people aren’t even arguing about the same issue since a lot gets rolled up into “the danger of AI” topic. Structured debate with moderators needs to make a comeback.

haswell 3y ago
> Or, perhaps more realistically, there isn’t a logically correct answer. It’s all just dressed up unsubstantiated opinions on both sides. I don’t agree with the framing of this. It may be true that there isn’t a knowable “correct” answer, but I think it’s problematic to summarize this as “both sides are just made of opinions”. I think the pernicious issue is the form of argument present in the “this is all just moral panic” camp. On the one side, we have deep concerns about the potential problems certain technology will usher in. On the other side, dismissal of those concerns based on the claim that the present issues are directly analogous to situations of the past that turned out to be fine. Even if it’s all just opinion, we have to judge those opinions on their merits, and the “it’s just a panic” crowd is on much shakier ground rhetorically.
- dcow 3y ago
  Why does causing a moral panic have a lower burden of proof than dismissing a moral panic? Do you believe in God? (Also I didn't mean it to come off as both sides have made up concerns. I meant it as both the concern and lack thereof camps are based solely on individuals’ priors.)
  - haswell 3y ago

jncfhnb 3y ago

The article feels like the fallacy fallacy. It overindexes on one particular poor argument being dumb as evidence that it’s thesis is incorrect.

Xcelerate 3y ago

During the pandemic, I thought that the people who were generally against caring about Covid were mostly faking it. I thought “ok, you’re not old or immunocompromised — you think that if you catch it, it will be like a bad cold but you’ll survive”. Then the vaccine came out, and so did the conspiracy theories. Even if I didn’t agree with these theories, I could at least see where the people who believed them might be coming from. It wouldn’t be the first time in human history that a government misled their own population for dubious purposes. It was conceivably possible that there are unknown, long-term effects of such a novel vaccine. But everything involves risk and my personal priors led to the view that not taking the vaccine was a much higher risk to myself and others. But then some close relatives that I respect and consider highly intelligent told me they didn’t trust the vaccine and took ivermectin instead. What. It’s a bit of an understatement to say that my internal model of human psychology fell apart right then. I’m quite sure at this point that if an asteroid were headed toward earth, there would be a sizable portion of the population who refuses to believe any evidence whatsoever and denies an asteroid is coming right up until the very end. It’s a similar case with AGI. I don’t think undeniable evidence that “it’s coming” is there yet, but if/when it is, there are going to be holdouts who refuse to believe what’s happening and may actually be incapable of believing what’s happening. It’s so far beyond their mental model of how reality works that the brain just kind of segfaults and rejects it—“I don’t want this to happen, therefore it cannot.”

ergonaught 3y ago

I will grab a few downvotes and submit that I believe the real problem he's encountering is simply confusion, resulting from a combination of in-group bias and a failure to recognize that "dumb" (meaning, here, difficulty in thinking clearly) dominates the population well beyond "average intelligence". Or: he's overestimated his cohort's capabilities in this matter. I am absolutely projecting.

ericmcer 3y ago
The brain sometimes grabs some sequence (coffee was bad... but then good... just like AI) and attributes some kind of epiphanal significance to it, but then on later examination it seems stupid and unimportant. I guess the short form of Twitter allows you to immortalize that thought without the later examination. If he had written an essay about it there is no way it would have survived.

SubiculumCode 3y ago

The reason is rather simple: emotional safety. It is emotionally stressful to consider dangerous possibilities, and for all the number that don't come true, unlness you can change it, most people will adopt a "don't worry be happy' perspective.

Dove 3y ago

Oh! I think I know what's going on here. This isn't a logical argument so much as it is a heuristic - and I think it's a very functional one. Consider Pascal's Wager. Should you follow God? Consider the consequences. If there is no God, following him might waste some time. But if there is a God, the consequences are as stark as they could be! Heaven and hell wipe out every other consideration, no matter what (nonzero) odds you assign God's existence. There is something wrong with this argument: the guy who set up the terms of the payoff schedule can manipulate you. A made up heaven can be as nice as you like, and a made up hell can be as nasty, as is necessary to make the reader tilt a certain way. Of course, it's hardly just religion that does this. Authorities of every stripe do it! But most obviously, politicians, propagandists, and similar influencers do it. They talk up bogeymen to scare and manipulate people. Now, if a bogeyman hails from your field, it is absolutely your responsibility to know about it and warn everyone. But if it's not your field, the heuristic is to... not care about the fire until you see smoke. This seems like an entirely irrational strategy, and I agree that it is if you look narrowly at determining truth. But I think the attitude is entirely correct and functional when you take attention, effort, and adversarial behavior into account. Obviously, refusing to acknowledge potential catastrophes makes you much harder to manipulate. But there's an even more basic reason: if you educated yourself on every possible catastrophe, you would magnify their impact to the point that you never lived your life. The heuristic being run into when you say "Aren't you worried about the AI apocalypse?" is probably easier to relate to if I say, "Aren't you worried about going to hell?" Hell sounds very bad, but then again you made the chart. There are, like, so many possible religions to think about, and someone with an apocalyptic issue wants to make it my issue every five minutes, never mind all the stuff my government wants me afraid of. Call me when you have some brimstone that I can actually smell myself. This applies doubly when the catastrophe in question is reasoned about in a style that resembles Pascal's Wager (as is often, but not always, the case with AI Apocalypse stuff). You want something less philosophically Bayesian and more tangible - the equivalent of Germany invading Poland, so anyone can potentially see the problem is real. Older people often spell this heuristic, "Well, if that awful thing happens, they can come get me here in the garden." Meaning, I refuse to let the bad times steal even one minute of the good ones. The heuristic is timeless because the technique is timeless. To the political move Bogeyman, we have the counter-jitsu Refuse To Care About Bogeymen. I don't think the people pointing to non-catastrophes explicitly think of it in those terms - I think the behavior is more instinctive than that - but I think the intuition behind the argument is less "I know this potential catastrophe isn't real" and more "worrying about potential catastrophes is a costly and losing game".

ericmcer 3y ago

The interesting thing here is the article winding itself up in tedious philosophical circles attempting to find some sort of objective truth. If a politician approached issues like that they would get about 4 votes. People hate it, they want "coffee was gonna be bad but ended up being good" and easily digestible platitudes. Even platitudes require too much processing nowadays, just stick a few buzzwords into an elementary school level sentence. Politics almost has to be approached in this simplified way or arguments would quickly drill down to more essential questions. A discussion around immigration would become about humanities fundamental purpose and if we should be creating an Eden on earth or racing to expand into the cosmos.

kazinator 3y ago

The way to fathom the argument is to identify its strongest possible form: Strawman: "No concern about unintended negative consequences has ever proven true." Strong version: "No end-of-the-world, sky-is-falling, doomsday prediction has proven to be true so far." I am not a proponent of either argument, but the latter is a tad better than the former. It is much better to have a counterargument against the strongest version, than to shadow-box with a strawman.

JoeyBananas 3y ago

AI safety is pretty much a total joke because the development of AI is not driven by high-minded idealistic principles. The driving force behind the advancement of AI is "We can (and it might make us a lot of money). Therefore, we should." Like it or not, that's how it is. This is how technology has always progressed, regardless of whether you can make the argument that this is OK from a moral standpoint or not. You can't do anything about it even if you wanted to. Recklessly uncontrolled advancement of technology is already the status quo. AI safety proponents (especially "effective" ones who truly believe what they are saying) need to be making the argument that the alarm needs to be raised. There needs to be an immediate intervention that shuts down AI progress, and the nature of this intervention must be radical to the point where people will reject it. Most of these AI safety proponents are not willing to make this argument because they know it's pointless, or they're actually just jealous of the people who carelessly advance technology and make a shitton of money off of it.

Gud 3y ago
Although I agree with your general argument, profit IS NOT how technology have always progressed!! I can give you a long list of technological advances that did not have a profit motive by its inventors if you wish!