3 ms·

This seems like a poor metaphor, considering that we can understand what constituent things would make up a superintelligence, even if we don’t understand the w

This seems like a poor metaphor, considering that we can understand what constituent things would make up a superintelligence, even if we don’t understand the whole. This discussion centers too much on the definitions of words like superintelligent and reminds me a lot of philosophical discussion about omnipotence. Both seem to rely more on defining concepts first and then assuming their existence as a consequence.

shinycode 2y ago
We understand some of it but who can say we understand the majority of it ? We might be at 0.1% of understanding the reality without be able to state this. Just as a worm surely « understand » some of it to differentiate and process it’s surroundings.
satvikpendem 2y ago
> considering that we can understand what constituent things would make up a superintelligence Can we? What constituent things would make up a superintelligence? Who's to say that our answer to that question is what is actually true in the case of a real superintelligence? > Both seem to rely more on defining concepts first and then assuming their existence as a consequence. Unlike religious philosophers like Anselm of Canterbury and Descartes and their ontological argument for the existence of a supreme being merely by imagining it, I don't believe anyone in the study of superintelligence is presupposing that they exist, or even can exist, they only presuppose how one might hypothetically exist.
- keiferski 2y ago
  Presumably the AI is in charge of things that humans used to manage and therefore understand. Worms don’t understand anything about even a slice of human society, so I don’t think it’s a great metaphor.
  - satvikpendem 2y ago
    Why does an AI have to handle human affairs at all? It could exist outside of human goals, that would not make it not a superintelligence, just as we don't really care about worms.
    - keiferski 2y ago
      I guess it doesn’t, but at this point what exactly are we speculating about? Because it seems like imaginary sci-fi, dependent on the definition of superintelligence and not on any real world developments. It seems much more realistic to me that AI will be running systems that humans used to run, and therefore will understand at some level.
      - satvikpendem 2y ago
        Well that is what the book is about, it is a speculative look at what hypothetical superintelligences might look like, it is explicitly not about the real world at all. Remember that the author is a philosopher, not an engineer, philosophy is all about hypotheticals.
        keiferski 2y ago
        Philosophy is not all about hypotheticals. Philosophy of technology especially is mostly about technologies that already exist and their impact on society. Not speculation.
        satvikpendem 2y ago
        Some philosophy (of technology, among others) relates to that, not all. It is not necessarily all about concrete impacts either, it depends on the author and their interests.
shrimp_emoji 2y ago
I always thought about it like being a permanent infant. The world is huge, full of colorful things you don't understand, and it'll be like that forever. But it's also a poor metaphor because adults have something that toddlers don't: fear. As a kid, you're an ignorant and curious blank slate; as an adult, you've established expectations and anxieties, so you'd probably be having a much worse time. :D
- satvikpendem 2y ago
  Toddlers do have fear, see something like the bridge crossing experiment [0]. There are many other such examples, toddlers are not tabulae rasae. [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3628785/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3628785/
  - trashtester 2y ago
    Even infants are really good at sucking.
    - jhbadger 2y ago
      Hence the old saying "the only truly intuitive user interface is the nipple".
ben_w 2y ago
Omnipotence is provably impossible: "Could Jesus microwave a burrito so hot that he himself could not eat it?" etc. Super-intelligence, not so much — there's plenty of examples of above-average humans in many areas, and no reason to think that putting the top expert in each field into one room is impossible, and no reason to think that this configuration cannot be implemented in software with a sufficiently powerful computer. And that's without the things that machines already beat us at, because super-human chess playing software is easily available, and computers that do arithmetic better than the entire human species even if we were all trained to the level of the current world record holder are cheap enough to be given away free glued to the front of a magazine, so there's no single person who has a particular advantage with those things. What chess does do, is give an example: if I was playing a game with Kasparov, I would have no idea which move he might make at any given moment, but despite that I'd still expect him to win. With an AI, I don't even necessarily know what "game" it's playing even if I'm the one who wrote the utility function it's trying to maximise.
- wait_a_minute 2y ago
  Omnipotence just means being able to do whatever power can do. So it is incoherent to ask if God could make a square circle. The words don’t mean anything. God could make a burrito too hot for himself to eat. Then he could make himself capable of eating it. Rinse and repeat.
  - orbisvicis 2y ago
    Yeah I never really bought that language argument - the idea that the paradox arises solely from a schism between language and reality. I prefer to think eschatology solves the issue quite nicely. The omnipotent entity is the one for which such a situation never arises. Similarly if a universe evolves to give rise to God, who is to say that God was never present. Or all around us, really? Either way I'd say Matthew 4:1 solves that paradox in a way that satisfies billions of people. Though I am getting a kick out of imagining Mexican/American Jesus munching on burritos.
    - ben_w 2y ago

shinycode 2y ago

We understand some of it but who can say we understand the majority of it ? We might be at 0.1% of understanding the reality without be able to state this. Just as a worm surely « understand » some of it to differentiate and process it’s surroundings.

satvikpendem 2y ago

> considering that we can understand what constituent things would make up a superintelligence Can we? What constituent things would make up a superintelligence? Who's to say that our answer to that question is what is actually true in the case of a real superintelligence? > Both seem to rely more on defining concepts first and then assuming their existence as a consequence. Unlike religious philosophers like Anselm of Canterbury and Descartes and their ontological argument for the existence of a supreme being merely by imagining it, I don't believe anyone in the study of superintelligence is presupposing that they exist, or even can exist, they only presuppose how one might hypothetically exist.

keiferski 2y ago
Presumably the AI is in charge of things that humans used to manage and therefore understand. Worms don’t understand anything about even a slice of human society, so I don’t think it’s a great metaphor.
- satvikpendem 2y ago
  Why does an AI have to handle human affairs at all? It could exist outside of human goals, that would not make it not a superintelligence, just as we don't really care about worms.
  - keiferski 2y ago
    I guess it doesn’t, but at this point what exactly are we speculating about? Because it seems like imaginary sci-fi, dependent on the definition of superintelligence and not on any real world developments. It seems much more realistic to me that AI will be running systems that humans used to run, and therefore will understand at some level.
    - satvikpendem 2y ago
      Well that is what the book is about, it is a speculative look at what hypothetical superintelligences might look like, it is explicitly not about the real world at all. Remember that the author is a philosopher, not an engineer, philosophy is all about hypotheticals.
      - keiferski 2y ago
        Philosophy is not all about hypotheticals. Philosophy of technology especially is mostly about technologies that already exist and their impact on society. Not speculation.
        satvikpendem 2y ago
        Some philosophy (of technology, among others) relates to that, not all. It is not necessarily all about concrete impacts either, it depends on the author and their interests.

shrimp_emoji 2y ago

I always thought about it like being a permanent infant. The world is huge, full of colorful things you don't understand, and it'll be like that forever. But it's also a poor metaphor because adults have something that toddlers don't: fear. As a kid, you're an ignorant and curious blank slate; as an adult, you've established expectations and anxieties, so you'd probably be having a much worse time. :D

satvikpendem 2y ago
Toddlers do have fear, see something like the bridge crossing experiment [0]. There are many other such examples, toddlers are not tabulae rasae. [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3628785/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3628785/
- trashtester 2y ago
  Even infants are really good at sucking.
  - jhbadger 2y ago
    Hence the old saying "the only truly intuitive user interface is the nipple".

ben_w 2y ago

Omnipotence is provably impossible: "Could Jesus microwave a burrito so hot that he himself could not eat it?" etc. Super-intelligence, not so much — there's plenty of examples of above-average humans in many areas, and no reason to think that putting the top expert in each field into one room is impossible, and no reason to think that this configuration cannot be implemented in software with a sufficiently powerful computer. And that's without the things that machines already beat us at, because super-human chess playing software is easily available, and computers that do arithmetic better than the entire human species even if we were all trained to the level of the current world record holder are cheap enough to be given away free glued to the front of a magazine, so there's no single person who has a particular advantage with those things. What chess does do, is give an example: if I was playing a game with Kasparov, I would have no idea which move he might make at any given moment, but despite that I'd still expect him to win. With an AI, I don't even necessarily know what "game" it's playing even if I'm the one who wrote the utility function it's trying to maximise.

wait_a_minute 2y ago
Omnipotence just means being able to do whatever power can do. So it is incoherent to ask if God could make a square circle. The words don’t mean anything. God could make a burrito too hot for himself to eat. Then he could make himself capable of eating it. Rinse and repeat.
- orbisvicis 2y ago
  Yeah I never really bought that language argument - the idea that the paradox arises solely from a schism between language and reality. I prefer to think eschatology solves the issue quite nicely. The omnipotent entity is the one for which such a situation never arises. Similarly if a universe evolves to give rise to God, who is to say that God was never present. Or all around us, really? Either way I'd say Matthew 4:1 solves that paradox in a way that satisfies billions of people. Though I am getting a kick out of imagining Mexican/American Jesus munching on burritos.
  - ben_w 2y ago