8 ms·

I remember reading Bostrom's work in 2014 and raving about it to others while no one really understood what I was so interested in. Well, now everyone is talkin

• keiferski 2y ago
  This seems like a poor metaphor, considering that we can understand what constituent things would make up a superintelligence, even if we don’t understand the whole. This discussion centers too much on the definitions of words like superintelligent and reminds me a lot of philosophical discussion about omnipotence. Both seem to rely more on defining concepts first and then assuming their existence as a consequence.
  • shinycode 2y ago
    We understand some of it but who can say we understand the majority of it ? We might be at 0.1% of understanding the reality without be able to state this. Just as a worm surely « understand » some of it to differentiate and process it’s surroundings.
  • satvikpendem 2y ago
    > considering that we can understand what constituent things would make up a superintelligence Can we? What constituent things would make up a superintelligence? Who's to say that our answer to that question is what is actually true in the case of a real superintelligence? > Both seem to rely more on defining concepts first and then assuming their existence as a consequence. Unlike religious philosophers like Anselm of Canterbury and Descartes and their ontological argument for the existence of a supreme being merely by imagining it, I don't believe anyone in the study of superintelligence is presupposing that they exist, or even can exist, they only presuppose how one might hypothetically exist.
    • keiferski 2y ago
      Presumably the AI is in charge of things that humans used to manage and therefore understand. Worms don’t understand anything about even a slice of human society, so I don’t think it’s a great metaphor.
      • satvikpendem 2y ago
        Why does an AI have to handle human affairs at all? It could exist outside of human goals, that would not make it not a superintelligence, just as we don't really care about worms.
        • keiferski 2y ago
          I guess it doesn’t, but at this point what exactly are we speculating about? Because it seems like imaginary sci-fi, dependent on the definition of superintelligence and not on any real world developments. It seems much more realistic to me that AI will be running systems that humans used to run, and therefore will understand at some level.
          • satvikpendem 2y ago
            Well that is what the book is about, it is a speculative look at what hypothetical superintelligences might look like, it is explicitly not about the real world at all. Remember that the author is a philosopher, not an engineer, philosophy is all about hypotheticals.
            • keiferski 2y ago
              Philosophy is not all about hypotheticals. Philosophy of technology especially is mostly about technologies that already exist and their impact on society. Not speculation.
              • satvikpendem 2y ago
                Some philosophy (of technology, among others) relates to that, not all. It is not necessarily all about concrete impacts either, it depends on the author and their interests.
  • shrimp_emoji 2y ago
    I always thought about it like being a permanent infant. The world is huge, full of colorful things you don't understand, and it'll be like that forever. But it's also a poor metaphor because adults have something that toddlers don't: fear. As a kid, you're an ignorant and curious blank slate; as an adult, you've established expectations and anxieties, so you'd probably be having a much worse time. :D
    • satvikpendem 2y ago
      Toddlers do have fear, see something like the bridge crossing experiment [0]. There are many other such examples, toddlers are not tabulae rasae. [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3628785/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3628785/
      • trashtester 2y ago
        Even infants are really good at sucking.
        • jhbadger 2y ago
          Hence the old saying "the only truly intuitive user interface is the nipple".
  • ben_w 2y ago
    Omnipotence is provably impossible: "Could Jesus microwave a burrito so hot that he himself could not eat it?" etc. Super-intelligence, not so much — there's plenty of examples of above-average humans in many areas, and no reason to think that putting the top expert in each field into one room is impossible, and no reason to think that this configuration cannot be implemented in software with a sufficiently powerful computer. And that's without the things that machines already beat us at, because super-human chess playing software is easily available, and computers that do arithmetic better than the entire human species even if we were all trained to the level of the current world record holder are cheap enough to be given away free glued to the front of a magazine, so there's no single person who has a particular advantage with those things. What chess does do, is give an example: if I was playing a game with Kasparov, I would have no idea which move he might make at any given moment, but despite that I'd still expect him to win. With an AI, I don't even necessarily know what "game" it's playing even if I'm the one who wrote the utility function it's trying to maximise.
    • wait_a_minute 2y ago
      Omnipotence just means being able to do whatever power can do. So it is incoherent to ask if God could make a square circle. The words don’t mean anything. God could make a burrito too hot for himself to eat. Then he could make himself capable of eating it. Rinse and repeat.
      • orbisvicis 2y ago
        Yeah I never really bought that language argument - the idea that the paradox arises solely from a schism between language and reality. I prefer to think eschatology solves the issue quite nicely. The omnipotent entity is the one for which such a situation never arises. Similarly if a universe evolves to give rise to God, who is to say that God was never present. Or all around us, really? Either way I'd say Matthew 4:1 solves that paradox in a way that satisfies billions of people. Though I am getting a kick out of imagining Mexican/American Jesus munching on burritos.
        • ben_w 2y ago
• exe34 2y ago
  I like to imagine bacteria as the compute substrate for an immaterial city of digital inhabitants. Fungus are even cooler, with the hypothetical wood wide web. maybe we already are the worms!
• john_minsk 2y ago
  Interesting. Could you share what are you interested in at the moment?
  • satvikpendem 2y ago
    In intellectual terms, I'm currently interested in the fusion of Asian and Western history, reading James Clavell's Asian Saga now, after watching Shogun recently. David Graeber's books are also on my list once I finish the Saga. I've read Bullshit Jobs and Debt by him but I've heard good things about The Dawn of Everything, particularly how European Enlightenment ideas might have actually been influenced by what they saw from Native Americans. In terms of projects I'm working on, I'm traveling currently and it's a pain to track how much money I've spent due to needing to convert foreign currencies, so I'm building a simple app for that.
• ganzuul 2y ago
  Humans use their multiply redundant brain power to align with absurd goals. We are simply hobbled by non-Star Trek culture.
• jhanschoo 2y ago
  We already have "superintelligences" in the world; Nature and other humans treated as a collective is far more powerful than any individual. We manage these risks by not trusting them completely, and restricting their dominance over us. I don't see that we can't adapt to superintelligent machines as long as we don't surrender all decision making to them; the risk comes from the same old, where a group possesses overwhelming power that is then used to regiment and oppress a less powerful group. In which case, possession of AI is far from unique.
  • _nalply 2y ago
    And why do you seem to think we are managing these risks successfully today?
  • streptomycin 2y ago
    Yes we already have "superintelligences" if we redefine the word to mean something different.
• adastra22 2y ago
  Imagine a clocked-down Turing computer, whose operating speed us is so slow that we can see its operations. Now imagine the programs which cannot, in principle be run on this computer. You can't because there aren't any. Now imagine we are that computer. Humans are capable of abstract thought and reasoning, which is sufficient to understand even the most complex software programs, to such an extent as they are understandable. The comparison to the worm is bogus.
  • thraxil 2y ago
    The clocked-down Turing computer you are proposing would have to have an infinite amount of tape to be truly "Turing complete" and capable of running any program we can imagine in principle. To actually match the worm analogy, imagine a clocked-down Turing machine with a tape length of maybe 100k cells. Now you can definitely imagine programs that can't run on it. The worm's limitation isn't that it's slow at thinking, it basically has only a handful of neurons compared to a person and there are problems/concepts that are just too large to fit in its brain and it would never be able to imagine even if it could live and think until the end of time.
    • adastra22 2y ago
      Human beings have effectively infinite tape in the form of pencils and paper, to say nothing of digital storage.
      • thraxil 2y ago
        Those things aren't infinite and you don't have infinite time either. If you're trying to use something like Turing completeness to argue a point, you need to respect that it's only relevant for theoretical, formal systems. You can't pull it out as a "gotcha" when it's a question of real-world capabilities. The basic argument here is specifically about the practical limitations. An actual worm, with a finite number of neurons and a finite lifetime, even if it was an exceptionally clever one, lived much longer than average, and spent every waking minute of its life pondering the mysteries of humans, would never be capable of comprehending more than the tiniest fraction. Even if you had worm scholars that developed some form of writing and could pass along information to subsequent generations, it's still not going to be possible because of the actual, physical limitations that have to be dealt with. A theoretical intelligence of a similar ratio to human intelligence would be similarly incomprehensible to a really clever human that spent their whole life studying the output of scholars who spent all their lives trying to understand it.
• _wire_ 2y ago
  Bostrom's accomplishment is to succeed at bamboozling many who felt they were beyond being bamboozled. Bostrom is a charlatan playing rubes with an intellectual version of three-card monty: //This paper argues that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor‐simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed.// What is a post-human stage of development? Is human extinction a post-human stage of development? How did this argument get from post-human to ancestor simulation? It's stipulated in a lemma. What is a simulation according to Bostrum? Undefined. What then is an "ancestor" simulation?! The weaseling on display adds gross insult to intellectual injury: How unlikely is "extremely unlikely"? What is a "significant number" of ancestor" simulations? What is "evolutionary history"? How did he get from ancestor simulation to computer simulation? Who else can stake an academic reputation on tenets delivered in the form of afterthoughts? A theory must contain a promise of making matters simpler. If there's any kind of theory associated with Bostrom's argument, it's that many people can't distinguish a theory from an arbitrary collection of intriguing statements. Bostrom is functioning as an academic instance of Weisenbaum's Eliza: it just circularly echoes its own kooky conjectures in the form of lemmas and an outwardly spiraling discussion of itself. Bostrom is playing a silly joke on readers. He gives his game away with his ridiculous lemmmas, showing the ace during the shuffle: "It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor‐simulations is false"—period. So, what was the question which was to be begged? Read it and weep. You've been played. You've got to hand it to a magician / mentalist who gets his audience to believe they are as "worms wiggling in the ground with no conception of god-like beings that inhabit the world". The most pitiless aspect of this fantasy is that it includes a proposition for what man is unaware of in the form of the overt statement of that which he is unaware: "god-like beings" Could the question of faith be any further begged? It's deranged. Here we encounter Zizek's fourth quadrant of Rumsfeld's knowns: the "unknown knowns." Ideology. That which you know but avoid or refuse to become aware that you know to keep your faith alive. There's a rule about the actual unknown, which is that it must be literally about the which is not known! More generally, as to what is not known, we can only poke at the edge of a lacunae, maybe inferring something about the contours or gradient to mystery, maybe following the edge to refining our adaptation to ourselves. But our work doesn't belong with the unknowable. It belongs with what we can know about ourselves. The unknown will take care of itself.