4 ms·

Your comment deserves a much longer response than I can give it, but you seem to be saying that it's impossible to know anything with any certainty, and that is

Your comment deserves a much longer response than I can give it, but you seem to be saying that it's impossible to know anything with any certainty, and that is absolutely true. I'm sure all the authors of this work would agree with you. Coming to conclusions/guesses requires starting with some assumptions about about how the universe works. It's important not to lose sight of those assumptions and very important to have a discussion about how likely each assumption is to be true. So to the extent that anyone is saying "this is definitely how the universe is", I wholeheartedly agree with you. However, it feels like you're being a little too epistemically nihilistic here. This kind of speculative work always starts without confirmation or any particular reason to believe it beyond how well it matches the curve it was fit to. To quibble with a particular, this work doesn't attempt to predict the actions of super-intelligences, it's not making the claim that most civilizations will be "grabby", it's making predictions about what will happen under the assumption that some subpopulation of them decides to become grabby.

titzer 5y ago
I am not being nihilistic, I am being realistic. We really can't predict the actions of superintelligence, full stop. Here's what I mean. What makes the lights on your router blink? Usually, it's packets going in and out. At a coarse level, if your computer is doing something on the internet, the lights are gonna be blinking. But now increase the resolution, from seconds, to milliseconds, to microseconds, to nanoseconds. At a nanosecond level, whether the LED is on or off depends on whether a packet is going over the wire right this nanosecond. Whether a packet is earlier or later by a nanosecond is influenced by a bazillion variables: the router's internal state, its own operating system's scheduling, whether the devices connected to it are sending packets at it, dragging in their state, their scheduling, their programs, packet queues, etc. Gigabytes of state, changing nano-second to nano-second. Oh, and it's connected to the internet. When does the internet decide to send a packet to this router? If those are faster or slower by a nanosecond, it could be due the state along the entire path, all of which are servicing packets from all over the place, decided upon by super-complex distributed systems--datacenters full of them. So even the blinking of a router LED is connected to a supremely complex causality tangle. To understand the blinking nano-second to nano-second, you basically have to replicate the details of the entire internet. So, if I were to ask you instead to predict what will happen tomorrow at 3:33am--will your router be blinking, let's say, fast or slow? Who knows. A DDoS might crop up because someone was mad at Amazon, or maybe your OS downloads a software update, or maybe you are up watching Netflix, or maybe the whole thing is down because nuclear war. It's progressed so far from a simple bell curve that probabilistic reasoning, I would argue, is useless. And that's just without the internet being "intelligent"! Now assume the internet knows this prediction game is afoot, and it gets to bet too. The whole thing goes off the rails. No, we can't predict what aliens will do. This paper is a mathematical exercise that might apply to bacteria or video game characters, not interstellar civilizations.
- qwytw 5y ago
  How can you predict that a "superintelligence" does or could even exist then? To be fair, based on your description, assuming is is not completely random, you can still build a somewhat accurate model that would be able to somewhat accurately estimate when and how often the light will be blinking without having a full understanding on what exactly causes it.

titzer 5y ago

I am not being nihilistic, I am being realistic. We really can't predict the actions of superintelligence, full stop. Here's what I mean. What makes the lights on your router blink? Usually, it's packets going in and out. At a coarse level, if your computer is doing something on the internet, the lights are gonna be blinking. But now increase the resolution, from seconds, to milliseconds, to microseconds, to nanoseconds. At a nanosecond level, whether the LED is on or off depends on whether a packet is going over the wire right this nanosecond. Whether a packet is earlier or later by a nanosecond is influenced by a bazillion variables: the router's internal state, its own operating system's scheduling, whether the devices connected to it are sending packets at it, dragging in their state, their scheduling, their programs, packet queues, etc. Gigabytes of state, changing nano-second to nano-second. Oh, and it's connected to the internet. When does the internet decide to send a packet to this router? If those are faster or slower by a nanosecond, it could be due the state along the entire path, all of which are servicing packets from all over the place, decided upon by super-complex distributed systems--datacenters full of them. So even the blinking of a router LED is connected to a supremely complex causality tangle. To understand the blinking nano-second to nano-second, you basically have to replicate the details of the entire internet. So, if I were to ask you instead to predict what will happen tomorrow at 3:33am--will your router be blinking, let's say, fast or slow? Who knows. A DDoS might crop up because someone was mad at Amazon, or maybe your OS downloads a software update, or maybe you are up watching Netflix, or maybe the whole thing is down because nuclear war. It's progressed so far from a simple bell curve that probabilistic reasoning, I would argue, is useless. And that's just without the internet being "intelligent"! Now assume the internet knows this prediction game is afoot, and it gets to bet too. The whole thing goes off the rails. No, we can't predict what aliens will do. This paper is a mathematical exercise that might apply to bacteria or video game characters, not interstellar civilizations.

qwytw 5y ago
How can you predict that a "superintelligence" does or could even exist then? To be fair, based on your description, assuming is is not completely random, you can still build a somewhat accurate model that would be able to somewhat accurately estimate when and how often the light will be blinking without having a full understanding on what exactly causes it.