8 ms·

> There is no evidence that a runaway artificial intelligence is even plausible Really? There is a lot of theory behind why this is likely to happen and if you

• civilitty 3y ago
  Yes, really. Scifi fantasies don’t count as evidence and we’ve learned since the Renaissance and scientific revolution that all this Platonic theorizing is just an intellectual circle jerk.
  • circuit10 3y ago
    The fact that something has been covered in sci-fi doesn’t mean that it can’t happen. https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone “Speaker A: Infectious diseases are caused by tiny organisms that are not visible to unaided eyesight. Speaker B: Your statement is false. Speaker A: Why do you think that it is false? Speaker B: It sounds like nonsense. Speaker B denies Speaker A's claim without providing evidence to support their denial.” Also I gave a real world example that wasn’t related to sci-fi in any way
    • Al0neStar 3y ago
      The burden of proof is on the person making the initial claim and i highly doubt that the reasoning behind AI ruin is as extensive and compelling as the germ theory of disease. We assume human-level general AI is possible because we exist in nature but a super-human self-optimizing AI god is nowhere to be found.
      • yeck 3y ago
        Which claim does the burden of proof land on? That an artificial super intelligence can easily be controlled or that it cannot? And what is your rational for deciding?
        • Al0neStar 3y ago
          The claim that there's a possibility of a sudden intelligence explosion. Like i said above you can argue that an AGI can be realized because there's plenty of us running around on earth but claims about a hypothetical super AGI are unfounded and akin to Russell's Teapot.
          • pixl97 3y ago
            Ok, in a converse question, why would evolution create the human mind as the most intelligent possible system? So I ask, are systems capable of better than human intelligence possible? And if so to what factor? I would answer yes and unknown. If there are states of intelligent far greater than human then that gets really risky very fast.
      • hackinthebochs 3y ago
        Burden of proof on neutral matters depends on whoever makes the claim. But AI and the potential for doom isn't a neutral matter. The question is what should be one's default belief until proven otherwise. It is a matter of determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. But when it comes to AGI, extreme caution is the only sensible default position. The history of humanity is replete with examples of the slightly more technologically advanced group decimating their competition. The default position should be that uneven advantage is extremely dangerous to those disadvantaged. This idea that an intelligence significantly greater than our own is benign just doesn't pass the smell test.
• mmaunder 3y ago
  Humans evolved unsupervised. AI is highly supervised. The idea that an AI will enslave us all is as absurd as suggesting “computers” will enslave us all merely because they exist. Models are designed and operated by people for specific use cases. The real risks are people using this new tool for evil, not the tool itself. AI sentience is a seductive concept being used by self professed experts to draw attention to themselves and by mega corps to throw up competitive barriers to entry.
  • sorokod 3y ago
    A fair amount of effort is spent on autonomous models, e.g. driving. Who knows what the military are up to.
  • circuit10 3y ago
    It’s almost impossible to supervise something more intelligent than you because you can’t tell why it’s doing things. For now it’s easy to supervise them because AIs are way less intelligent than humans (though even now it’s hard to tell exactly why they’re doing things), but in the future it probably won’t be
    • z3c0 3y ago
      but in the future it probably won’t be I see this parroted so often, and I have to ask: Why? What is there outside of the world of SciFi that makes the AGI of the future so nebulous, when humans would have presumably advanced to a point to be able to create the intelligence to begin with. Emergent properties are often as unexpected as they are bizarre, but they are not unexplainable, especially when you understand the underpinning systems.
      • circuit10 3y ago
        We can’t even fully explain how our own brains work, never mind a system that’s completely alien to us and that would have to be more complex. We can’t even explain how current LLMs work internally. Maybe we’ll make some breakthrough if we put enough resources into it but if people keep denying the problem there will never be enough resources out into it
• dwallin 3y ago
  The theories all inevitably rely on assumptions that are essentially the equivalence of spherical cows in a frictionless universe. All evidence is that costs for intelligence likely scale superlinearly. Each increase in intelligence capability requires substantially more resources (Computing power, training data, electricity, hardware, time, etc). Being smart doesn’t just directly result in these becoming available with no limit. Any significant attempts to increase the availability of these to a level that mattered would almost certainly draw attention. In addition, even for current AI we don’t even fully understand what we are doing, even though they are operating at a lower generalized intelligence level than us. Since we don’t have a solid foundational model for truly understanding intelligence, progress relies heavily on experimentation to see what works. (Side note: my gut is that we will find there’s some sort of equivalent to the halting problem when it comes to understanding intelligence) It’s extremely likely that this remains true, even for artificial intelligence. In order for an AI to improve upon itself, it would likely also need to do significant experimentation, with diminishing returns and exponentially increasing costs for each level of improvement it achieves. In addition, a goal-oriented generalized AI would have the same problems that you worry about. In trying to build a superior intelligence to itself it risks building something that undermines its own goals. This increases the probability of either us, or a goal-aligned AI, noticing and being able to stop things from escalating. It also means that a super intelligent AI has disincentives to build better AIs.
  • arisAlexis 3y ago
    "In addition, even for current AI we don’t even fully understand what we are doing" That is the problem, don't you get it?
    • ls612 3y ago
      “In addition, even with the current state of the internet, we don’t have understanding everything we are doing with it” -some guy in the ‘90s probably
      • pixl97 3y ago
        Many people would look at engagement algorithms braching out of social media and causing riots and uprisings as one of those issues that would be difficult to predict in the 90s
• AbrahamParangi 3y ago
  Theory is not actually a form of evidence.
  • circuit10 3y ago
    Theory can be used to predict things with reasonable confidence. It could be wrong, but assuming it’s wrong is a big risk to take. Also I gave a real-world analogy that has actually happened
    • AnimalMuppet 3y ago
      An accurate theory can be used to predict things with reasonable confidence, within the limits of the theory. We don't have an accurate theory of intelligence. What we have now is at the "not even wrong" stage. Assuming it's wrong is about like assuming that alchemy is wrong.
      • pixl97 3y ago
        Then let's go with an observation. Humans got smarter than most large mammals, then most large mammals went extinct.
• throwaway9274 3y ago
  Is there really “a lot of theory” that says runaway AI is possible? In the sense of empirical fact-based peer reviewed machine learning literature? Because if so I must have missed it. It seems more accurate to say there is quite a bit of writing done by vocal influencers who frequent a couple online forums.
  • pixl97 3y ago
    In evolutionary biological systems 'capabilities break out' common enough in viral and bacterial systems with their fast evolution times. The same thing can occur when new predators are introduced to ecosystems that are not prepared for them. The question that I assume you have is 'can AI systems experience large evolutionary jumps in capabilities'. I would say yes myself, but you apper to disagree.
  • DirkH 3y ago
    There was also only "a lot of theory" and zero empirical evidence that nuclear bombs were possible not that long ago. Theoretical fact-based peer reviewed literature often (though not always) precedes empirical evidence. We shouldn't jump the gun and say runaway AI is inevitable just because there is fact-based theory on its possibility. Just like it would have been dumb to just look at E=mc^2 and conclude nuclear weapon development is 100% inevitable. But in the same breath we shouldn't say it's impossible just because we don't have empirical evidence either.
    • throwaway9274 3y ago
      Prior to the development of nuclear weapons, there was well-grounded peer reviewed theory in the physics literature. There were predictions that could be made by the theoretical frameworks, and experiments to confirm those predictions. The literature well described the composition of atoms, the decay of radioactive elements, the concept of fission, and the conversion of mass to energy. No such groundwork exists to substantiate the claim that runaway AI is possible. LessWrong, the so-called Alignment Forum, and similar Internet groups propose thought experiments. They couch these thought experiments in the guise of inevitable outcomes. The prophesies of the influencers originating there have no more relationship to the truth than the average enthusiast’s daydreams. The content there fills a role similar to science fiction in the atomic age: fanciful speculation on the properties of imaginary systems. Over time, the bad content is forgotten, and a handful of items survive and look prescient via selection bias.
• voidhorse 3y ago
  You do realize the "AI" we're talking about here is just LLMs, right? Language is a huge part of being human, so that's a good elements but: 1. There's no evidence that these LLMs have any sort of actual understanding of what they produce. 2. Being human is a lot more than just linguistic capability. We are embodied in a certain way, can interact with environments, have rich, multi-sensory experiences etc. tbh I think anyone that believes AGI can arise from an LLM is extremely naive and has basically no idea of the history of this track of research or how it works. To assert that LLMs will be capable of achieving consciousness is equivalent to asserting that human consciousness is reducible to statistical inference, which is palpably not true. (see. e.g. deductive logic). The only reason people even fall into the trap of thinking computers are remotely close to getting to AGI is because it became very popular to reason about the brain using analogies from computer science in the 50s, despite the fact that this was completely unjustified, and people glob on to the metaphors and then have the mypoia to actually think the human brain (which is exceedingly complex) is reducible to a freakin' von neuman architecture, lol. Biological systems and inorganic systems are fundamentally different.
  • flangola7 3y ago
    I don't think anyone is talking about consciousness here. Consciousness is tangential to safety concerns.