5 ms·

A lot of the responses to this seem like Bulverism, ie., trying to refute an argument by psychoanalyzing the people who argue it: "Suppose I think, after doing

A lot of the responses to this seem like Bulverism, ie., trying to refute an argument by psychoanalyzing the people who argue it: "Suppose I think, after doing my accounts, that I have a large balance at the bank. And suppose you want to find out whether this belief of mine is "wishful thinking." You can never come to any conclusion by examining my psychological condition. Your only chance of finding out is to sit down and work through the sum yourself. When you have checked my figures, then, and then only, will you know whether I have that balance or not. If you find my arithmetic correct, then no amount of vapouring about my psychological condition can be anything but a waste of time. If you find my arithmetic wrong, then it may be relevant to explain psychologically how I came to be so bad at my arithmetic, and the doctrine of the concealed wish will become relevant—but only after you have yourself done the sum and discovered me to be wrong on purely arithmetical grounds. It is the same with all thinking and all systems of thought. If you try to find out which are tainted by speculating about the wishes of the thinkers, you are merely making a fool of yourself. You must first find out on purely logical grounds which of them do, in fact, break down as arguments. Afterwards, if you like, go on and discover the psychological causes of the error. You must show that a man is wrong before you start explaining why he is wrong. The modern method is to assume without discussion that he is wrong and then distract his attention from this (the only real issue) by busily explaining how he became so silly." - CS Lewis https://en.wikipedia.org/wiki/Bulverism https://en.wikipedia.org/wiki/Bulverism

skepticATX 3y ago
But what argument is there to refute? It feels like Aquinas “proving” God’s existence by stating that it is self evident. They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. So again, what can be refuted?
- notahacker 3y ago
  Seems apt the term "Bulverism" comes from CS Lewis, since he was also positing that an unseen, unfalsifiable entity would grant eternal reward to people that listened to him and eternal damnation to those that didn't...
  - alasdair_ 3y ago
    The irony of critiquing Bulverism as a concept, not by attacking the idea itself, but instead by assuming it is wrong and attacking the character of the author, is staggeringly hilarious.
    - notahacker 3y ago
      I'm replying in agreement with someone who already pointed out the obvious flaw in labelling any questioning of the inspirations or motivations of AI researchers as "Bulverism": none of the stuff they're saying is actually a claim that can be falsified in the first place! I'm unconvinced by the position that the only valid means of casting doubt on a claim is through forensic examination of hard data that may be inaccessible to the interlocutor (like most people's bank accounts...), but whether that is or isn't a generally good approach is irrelevant here as we're talking about claims about courses of action to avoid hypothetical threats. I just noted it was a particularly useful rhetorical flourish when advocating acting on beliefs which aren't readily falsifiable, something CS Lewis was extremely proud of doing and certainly wouldn't have considered a character flaw! Ironically, your reply also failed to falsify anything I said and instead critiqued my assumed motivations for making the comment. It's Bulverism all the way down!
      - deltaninenine 3y ago
        Logical Induction has been successful in predicting future events.
        notahacker 3y ago
        Sometimes it makes good predictions, sometimes bad. But "advances in AI might lead to Armageddon" isn't the only conclusion induction can reach. Induction can also lead to people concluding certain arguments seem to a mashup of traditional millennialist "end times" preoccupations with the sort of sci-fi they grew up with, or that this looks a lot like a movement towards regulatory capture. Ultimately any (possibly even all) these inferences from past trends and recent actions can be correct, but none of them are falsifiable. So I don't think it's a good idea to insist that people should be falsifying the idea that AI is a risk before we start questioning whether the behaviour of some of the entities on the list says more about their motivations than their words.
- duvenaud 3y ago
  Here's one of my concrete worries: At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence. I agree we don't necessarily know the details of how to build such a system, but am pretty sure we will be able to eventually.
  - Infernal 3y ago
    “Humans are going to be outcompeted by AI” is the concrete bit as best I can tell. Historically humans are not outcompeted by new tools, but humans using old tools are outcompeted by humans using new tools. It’s not “all humans vs the new tool”, as the tool has no agency. If you meant “humans using old tools get outcompeted by humans using AI”, then I agree but I don’t see it any differently than previous efficiency improvements with new tooling. If you meant ”all humans get outcompeted by AI”, then I think you have a lot of work to do to demonstrate how AI is going to replace humans in “every important job”, and not simply replace some of the tools in the humans’ toolbox.
    - duvenaud 3y ago
      I see what you mean - for a while, the best chess was played by humans aided by chess engines. But that era has passed, and now having a human trying to aid the best chess engines just results in worse chess (or the same, if the human does nothing). But whether there a few humans in the loop doesn't change the likely outcomes, if their actions are constrained by competition. What abilities do humans have that AIs will never have?
      - jvanderbot 3y ago
        Well, in this case, we have the ability to invent chess (a game that will be popular for centuries), invent computers, and invent chess tournaments, and invent programs that can solve chess, and invent all the supporting agriculture, power, telco, silicon boards, etc that allow someone to run a program to beat a person at chess. Then we have bodies to accomplish everything on top of it. The "idea" isn't enough. We have to "do" it. If you take a chess playing robot as the peak of the pyramid, there are probably millions of people and trillions of dollars toiling away to support it. Imagine all the power lines, sewage, HVAC systems, etc that humans crawl around in to keep working. And really, are we "beaten" at chess, or are we now "unbeatable" at chess. If an alien warship came and said "we will destroy earth if you lose at chess", wouldn't we throw our algorithms at it? I say we're now unbeatable at chess.
- SpaceManNabs 3y ago
  > They can’t point to an existing system that poses existential risk, because it doesn’t exist. There are judges using automated decision systems to excuse away decisions that send people back to jail for recidivism purposes. These systems are just enforcing societal biases at scale. It is clear that we are ready to acquiesce control to AI systems without much care to any extra ethical considerations.
  - brookst 3y ago
    I think you just said that the problem is systemic in our judicial system, and that AI has nothing to do with it.
    - SpaceManNabs 3y ago
      AI is the tool that provides "objective to truth" that enables such behavior. It is definite unique in its depth, scale, and implications.
  - NathanFulton 3y ago
    Absolutely. These are the types of pragmatic, real problems we should be focusing on instead of the "risk of extinction from AI". (The statement at hand reads "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.")
    - holmesworcester 3y ago
      Einstein's letter to Roosevelt was written before the atomic bomb existed. There's a point where people see a path, and they gain confidence in their intuition from the fact that other members of their field also see a path. Einstein's letter said 'almost certain' and 'in the immediate future' but it makes sense to sound the alarm about AI earlier, both given what we know about the rate of progress of general purpose technologies and given that the AI risk, if real, is greater than the risk Einstein envisioned (total extermination as opposed to military defeat to a mass murderer.)
- deltaninenine 3y ago
  It's arrived at through induction. Induction is logic involving probability. Probabilistic logic and predictions of the future are valid logic that has demonstrably worked in other situations so if such logic has a level of validity then induction is a candidate for refutation. So we know a human of human intelligence can take over a humans job and endanger other humans. AI has been steadily increasing in intelligence. The latest leap with LLMs crossed certain boundaries of creativity and natural language. By induction the trendline points to machines approaching human intelligence. Also by induction if humans of human intelligence can endanger humanity then a machine of human intelligence should do the same. Now. All of this induction is something you and everyone already knows. We know that this level of progress increases the inductive probabilities of this speculation playing out. None of us needs to be explained any of this logic as we are all well aware of it. What's going on is that humans like to speculate on a future that's more convenient for them. Science shows human psychology is more optimistic then realistic. Hence why so many people are in denial.
- ericb 3y ago
  What? ChatGPT 4 can already pass the bar exam and is fluent in every language. It is super intelligent. Today. No human can do that, the system is here, and so is an architecture. As for the existential risk, assume nothing other than evil humans will use it to do evil human stuff. Most technology iteratively gets better, so there's no big leaps of imagination required to imagine that we're equipping bad humans with super-human, super-intelligent assistants.
  - ilaksh 3y ago
    Right. And it would be a complete break from the history of computing if human-level GPT doesn't get 100+ times faster in the next few years. Certainly within five years. All it takes is for someone to give an AI that thinks 100 times faster than humans an overly broad goal. Then the only way to counteract it is with another AI with overly broad goals. And you can't tell it to stop and wait for humans to check it's decisions, because while it is waiting for you to come back from your lunch break to try to figure out what it is asking, the competitor's AI did the equivalent of a week of work. So then even if at some level people are "in control" of the AIs, practically speaking they are spectators. And there is no way you will be able to prevent all people from creating fully autonomous lifelike AI with its own goals and instincts. Combine that with hyperspeed and you are truly at it's mercy.
    - mrtranscendence 3y ago
      Computational power does not grow at the rate of over 100x within the span of “a few years”. If that were the case we’d have vastly more powerful kit by now.
      - ilaksh 3y ago
        I didn't quite say that. The efficiency of this very specific application absolutely can and almost certainly will increase by more than one order of magnitude within four years. It's got a massive new investment and research focus, is a very specific application, and room for improvement in AI model, software, and hardware. Even if we have to "cheat" to get to 100 times performance in less than five years the effect will be the same. For example, there might be a way to accelerate something like the Tree of Thoughts in hardware. So if the hardware can't actually speed up by that much, the effectiveness of the system still has increased greatly.
- a257 3y ago
  > They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. Inductive reasoning is in favor of their argument being possible. From observing nature, we know that a variety of intelligent species can emerge from physical phenomenon alone. Historically, the dominance of one intelligent species has contributed to the extinction of others. Given this, it can be said that AI might cause our extinction.
- Symmetry 3y ago
  The idea is that if you build a system that poses an existential risk you want to be reasonably sure it's safe before you turn it on, not afterwards. It would have been irresponsible for the scientists at Los Alamost to do the math on whether an atomic explosion would create a sustained fusion reaction in the atmosphere until after their first test, for example. I don't think it's possible for a large language model, operating in a conventional feed forward way, to really pose a significant danger. But I do think it's hard to say exactly what advances could lead to a dangerous intelligence and with the current state of the art it looks to me at least like we might very well be only one breakthrough away from that. Hence the calls for prudence. The scientists creating the atomic bomb knew a lot more about what they were doing than we do. Their computations sometimes gave the wrong result, see Castle Bravo, but had a good framework for understanding everything that was happening. We're more like cavemen who've learned to reliably make fire but still don't understand it. Why can current versions of GPT reliably add large numbers together when previous versions couldn't? We're still a very long way away from being able to answer questions like that.
- lxnn 3y ago
  You can't take an empirical approach to existential risk as you don't get the opportunity to learn from your mistakes. You have to prospectively reason about it and plan for it.
mefarza123 3y ago
CS Lewis's quote highlights the importance of addressing the logical validity of an argument before attempting to explain the psychological reasons behind it. This approach is essential to avoid committing the fallacy of Bulverism, which involves dismissing an argument based on the presumed motives or biases of the person making the argument, rather than addressing the argument itself. In the context of AI and decision-making, it is crucial to evaluate the logical soundness of arguments and systems before delving into the psychological factors and biases that may have influenced their development. For instance, when assessing the effectiveness of an AI-assisted decision-making system, one should first examine the accuracy and reliability of the system's outputs and the logic behind its algorithms. Only after establishing the system's validity or lack thereof, can one explore the potential biases and psychological factors that may have influenced its design. Several papers from MirrorThink.ai emphasize the importance of addressing logical fallacies and biases in AI systems. For example, the paper "Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments" proposes a method for identifying logical fallacies in natural language arguments, which can be used to improve AI systems' argumentation capabilities. Similarly, the paper "Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making" explores the role of cognitive biases in AI-assisted decision-making and provides recommendations for addressing these biases. In conclusion, it is essential to prioritize the evaluation of logical soundness in arguments and AI systems before exploring the psychological factors and biases that may have influenced their development. This approach helps to avoid committing the fallacy of Bulverism and ensures that discussions and evaluations remain focused on the validity of the arguments and systems themselves.
- Veen 3y ago
  Are you laboring under the misapprehension that ChatGPT is a better writer than C.S. Lewis?

skepticATX 3y ago

But what argument is there to refute? It feels like Aquinas “proving” God’s existence by stating that it is self evident. They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. So again, what can be refuted?

notahacker 3y ago
Seems apt the term "Bulverism" comes from CS Lewis, since he was also positing that an unseen, unfalsifiable entity would grant eternal reward to people that listened to him and eternal damnation to those that didn't...
- alasdair_ 3y ago
  The irony of critiquing Bulverism as a concept, not by attacking the idea itself, but instead by assuming it is wrong and attacking the character of the author, is staggeringly hilarious.
  - notahacker 3y ago
    I'm replying in agreement with someone who already pointed out the obvious flaw in labelling any questioning of the inspirations or motivations of AI researchers as "Bulverism": none of the stuff they're saying is actually a claim that can be falsified in the first place! I'm unconvinced by the position that the only valid means of casting doubt on a claim is through forensic examination of hard data that may be inaccessible to the interlocutor (like most people's bank accounts...), but whether that is or isn't a generally good approach is irrelevant here as we're talking about claims about courses of action to avoid hypothetical threats. I just noted it was a particularly useful rhetorical flourish when advocating acting on beliefs which aren't readily falsifiable, something CS Lewis was extremely proud of doing and certainly wouldn't have considered a character flaw! Ironically, your reply also failed to falsify anything I said and instead critiqued my assumed motivations for making the comment. It's Bulverism all the way down!
    - deltaninenine 3y ago
      Logical Induction has been successful in predicting future events.
      - notahacker 3y ago
        Sometimes it makes good predictions, sometimes bad. But "advances in AI might lead to Armageddon" isn't the only conclusion induction can reach. Induction can also lead to people concluding certain arguments seem to a mashup of traditional millennialist "end times" preoccupations with the sort of sci-fi they grew up with, or that this looks a lot like a movement towards regulatory capture. Ultimately any (possibly even all) these inferences from past trends and recent actions can be correct, but none of them are falsifiable. So I don't think it's a good idea to insist that people should be falsifying the idea that AI is a risk before we start questioning whether the behaviour of some of the entities on the list says more about their motivations than their words.
duvenaud 3y ago
Here's one of my concrete worries: At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence. I agree we don't necessarily know the details of how to build such a system, but am pretty sure we will be able to eventually.
- Infernal 3y ago
  “Humans are going to be outcompeted by AI” is the concrete bit as best I can tell. Historically humans are not outcompeted by new tools, but humans using old tools are outcompeted by humans using new tools. It’s not “all humans vs the new tool”, as the tool has no agency. If you meant “humans using old tools get outcompeted by humans using AI”, then I agree but I don’t see it any differently than previous efficiency improvements with new tooling. If you meant ”all humans get outcompeted by AI”, then I think you have a lot of work to do to demonstrate how AI is going to replace humans in “every important job”, and not simply replace some of the tools in the humans’ toolbox.
  - duvenaud 3y ago
    I see what you mean - for a while, the best chess was played by humans aided by chess engines. But that era has passed, and now having a human trying to aid the best chess engines just results in worse chess (or the same, if the human does nothing). But whether there a few humans in the loop doesn't change the likely outcomes, if their actions are constrained by competition. What abilities do humans have that AIs will never have?
    - jvanderbot 3y ago
      Well, in this case, we have the ability to invent chess (a game that will be popular for centuries), invent computers, and invent chess tournaments, and invent programs that can solve chess, and invent all the supporting agriculture, power, telco, silicon boards, etc that allow someone to run a program to beat a person at chess. Then we have bodies to accomplish everything on top of it. The "idea" isn't enough. We have to "do" it. If you take a chess playing robot as the peak of the pyramid, there are probably millions of people and trillions of dollars toiling away to support it. Imagine all the power lines, sewage, HVAC systems, etc that humans crawl around in to keep working. And really, are we "beaten" at chess, or are we now "unbeatable" at chess. If an alien warship came and said "we will destroy earth if you lose at chess", wouldn't we throw our algorithms at it? I say we're now unbeatable at chess.
SpaceManNabs 3y ago
> They can’t point to an existing system that poses existential risk, because it doesn’t exist. There are judges using automated decision systems to excuse away decisions that send people back to jail for recidivism purposes. These systems are just enforcing societal biases at scale. It is clear that we are ready to acquiesce control to AI systems without much care to any extra ethical considerations.
- brookst 3y ago
  I think you just said that the problem is systemic in our judicial system, and that AI has nothing to do with it.
  - SpaceManNabs 3y ago
    AI is the tool that provides "objective to truth" that enables such behavior. It is definite unique in its depth, scale, and implications.
- NathanFulton 3y ago
  Absolutely. These are the types of pragmatic, real problems we should be focusing on instead of the "risk of extinction from AI". (The statement at hand reads "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.")
  - holmesworcester 3y ago
    Einstein's letter to Roosevelt was written before the atomic bomb existed. There's a point where people see a path, and they gain confidence in their intuition from the fact that other members of their field also see a path. Einstein's letter said 'almost certain' and 'in the immediate future' but it makes sense to sound the alarm about AI earlier, both given what we know about the rate of progress of general purpose technologies and given that the AI risk, if real, is greater than the risk Einstein envisioned (total extermination as opposed to military defeat to a mass murderer.)
deltaninenine 3y ago
It's arrived at through induction. Induction is logic involving probability. Probabilistic logic and predictions of the future are valid logic that has demonstrably worked in other situations so if such logic has a level of validity then induction is a candidate for refutation. So we know a human of human intelligence can take over a humans job and endanger other humans. AI has been steadily increasing in intelligence. The latest leap with LLMs crossed certain boundaries of creativity and natural language. By induction the trendline points to machines approaching human intelligence. Also by induction if humans of human intelligence can endanger humanity then a machine of human intelligence should do the same. Now. All of this induction is something you and everyone already knows. We know that this level of progress increases the inductive probabilities of this speculation playing out. None of us needs to be explained any of this logic as we are all well aware of it. What's going on is that humans like to speculate on a future that's more convenient for them. Science shows human psychology is more optimistic then realistic. Hence why so many people are in denial.
ericb 3y ago
What? ChatGPT 4 can already pass the bar exam and is fluent in every language. It is super intelligent. Today. No human can do that, the system is here, and so is an architecture. As for the existential risk, assume nothing other than evil humans will use it to do evil human stuff. Most technology iteratively gets better, so there's no big leaps of imagination required to imagine that we're equipping bad humans with super-human, super-intelligent assistants.
- ilaksh 3y ago
  Right. And it would be a complete break from the history of computing if human-level GPT doesn't get 100+ times faster in the next few years. Certainly within five years. All it takes is for someone to give an AI that thinks 100 times faster than humans an overly broad goal. Then the only way to counteract it is with another AI with overly broad goals. And you can't tell it to stop and wait for humans to check it's decisions, because while it is waiting for you to come back from your lunch break to try to figure out what it is asking, the competitor's AI did the equivalent of a week of work. So then even if at some level people are "in control" of the AIs, practically speaking they are spectators. And there is no way you will be able to prevent all people from creating fully autonomous lifelike AI with its own goals and instincts. Combine that with hyperspeed and you are truly at it's mercy.
  - mrtranscendence 3y ago
    Computational power does not grow at the rate of over 100x within the span of “a few years”. If that were the case we’d have vastly more powerful kit by now.
    - ilaksh 3y ago
      I didn't quite say that. The efficiency of this very specific application absolutely can and almost certainly will increase by more than one order of magnitude within four years. It's got a massive new investment and research focus, is a very specific application, and room for improvement in AI model, software, and hardware. Even if we have to "cheat" to get to 100 times performance in less than five years the effect will be the same. For example, there might be a way to accelerate something like the Tree of Thoughts in hardware. So if the hardware can't actually speed up by that much, the effectiveness of the system still has increased greatly.
a257 3y ago
> They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. Inductive reasoning is in favor of their argument being possible. From observing nature, we know that a variety of intelligent species can emerge from physical phenomenon alone. Historically, the dominance of one intelligent species has contributed to the extinction of others. Given this, it can be said that AI might cause our extinction.
Symmetry 3y ago
The idea is that if you build a system that poses an existential risk you want to be reasonably sure it's safe before you turn it on, not afterwards. It would have been irresponsible for the scientists at Los Alamost to do the math on whether an atomic explosion would create a sustained fusion reaction in the atmosphere until after their first test, for example. I don't think it's possible for a large language model, operating in a conventional feed forward way, to really pose a significant danger. But I do think it's hard to say exactly what advances could lead to a dangerous intelligence and with the current state of the art it looks to me at least like we might very well be only one breakthrough away from that. Hence the calls for prudence. The scientists creating the atomic bomb knew a lot more about what they were doing than we do. Their computations sometimes gave the wrong result, see Castle Bravo, but had a good framework for understanding everything that was happening. We're more like cavemen who've learned to reliably make fire but still don't understand it. Why can current versions of GPT reliably add large numbers together when previous versions couldn't? We're still a very long way away from being able to answer questions like that.
lxnn 3y ago
You can't take an empirical approach to existential risk as you don't get the opportunity to learn from your mistakes. You have to prospectively reason about it and plan for it.

mefarza123 3y ago

CS Lewis's quote highlights the importance of addressing the logical validity of an argument before attempting to explain the psychological reasons behind it. This approach is essential to avoid committing the fallacy of Bulverism, which involves dismissing an argument based on the presumed motives or biases of the person making the argument, rather than addressing the argument itself. In the context of AI and decision-making, it is crucial to evaluate the logical soundness of arguments and systems before delving into the psychological factors and biases that may have influenced their development. For instance, when assessing the effectiveness of an AI-assisted decision-making system, one should first examine the accuracy and reliability of the system's outputs and the logic behind its algorithms. Only after establishing the system's validity or lack thereof, can one explore the potential biases and psychological factors that may have influenced its design. Several papers from MirrorThink.ai emphasize the importance of addressing logical fallacies and biases in AI systems. For example, the paper "Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments" proposes a method for identifying logical fallacies in natural language arguments, which can be used to improve AI systems' argumentation capabilities. Similarly, the paper "Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making" explores the role of cognitive biases in AI-assisted decision-making and provides recommendations for addressing these biases. In conclusion, it is essential to prioritize the evaluation of logical soundness in arguments and AI systems before exploring the psychological factors and biases that may have influenced their development. This approach helps to avoid committing the fallacy of Bulverism and ensures that discussions and evaluations remain focused on the validity of the arguments and systems themselves.

Veen 3y ago
Are you laboring under the misapprehension that ChatGPT is a better writer than C.S. Lewis?