3 ms·

I find these kinds of analyses to be hopelessly oversimplified. First of all, we are talking about predicting the actions of superintelligent aliens. We aren't

by titzer 5y ago

I find these kinds of analyses to be hopelessly oversimplified. First of all, we are talking about predicting the actions of superintelligent aliens. We aren't talking about slime molds or a bacterial infection, but beings with more intelligence than us, like orders of magnitude more complicated reasoning processes. They are effectively unpredictable, because if they weren't, wouldn't we be just as smart as they are? That level of intelligence comes with new levels of reflectiveness that are frankly not accessible to our weak brains. Like, maybe after reaching the next level of enlightenment, they decide that expansion throughout the universe like a worm is not actually what they want or that is what is good for the universe? Maybe they decide to rather study the universe and not interfere, the way biologists study, say, apes, without wanting to disrupt their natural behavior with their presence? Or maybe superintelligent beings find a way out of our universe, and pass into another dimension (cyberspace? black holes? disappearing through quantum foam? heaven?). Or maybe something else insane happens, like being superintelligent inevitably makes you completely terrified of everything, because you realize how powerful other aliens are and your calculations unequivocally show it's safer to stay at home and fart around doing the super-intelligent analog of baking chocolate-chip cookies? Not being superintelligent, we are like ants trying to contemplate human civilization. We have no context or experience, and we simply don't have the mental capacity to even imagine what we don't know. Second, these probabilistic models are so fundamentally stupid. It's impossible for us to reliably estimate the probability of any event that's happened only once (without reasoning about the process that gave rise to that event), as we cannot estimate posterior probabilities without lots of samples. And humans on Earth have a massive chain of single-occurrence events that all seem to be absurdly improbable. Besides that, and perhaps even because of that, it's not even a foregone conclusion that our universe even selects from the middle of the distribution (i.e. is not biased). We could perfectly estimate the shape of a probability curve and yet live in a universe that stubbornly selects from the end of the distribution, all the time. All of our math could be totally right, but crazy shit just keeps happening. Third, the anthropic principle is a total bitch. It means that we will never be able to estimate probabilities of events that were required to create us, because no matter how improbable they were, if they didn't happen, we're not even here to argue about it. If there are infinite universes (a multiverse, see Tegmark), then forget this whole mathematical sophistry, because then there is no need to live in some "average" universe; we can easily live in some stupidly improbably snowflake universe that has the 90 trillion improbable things that make apes possible; infinity doesn't care.

brian_cloutier 5y ago
Your comment deserves a much longer response than I can give it, but you seem to be saying that it's impossible to know anything with any certainty, and that is absolutely true. I'm sure all the authors of this work would agree with you. Coming to conclusions/guesses requires starting with some assumptions about about how the universe works. It's important not to lose sight of those assumptions and very important to have a discussion about how likely each assumption is to be true. So to the extent that anyone is saying "this is definitely how the universe is", I wholeheartedly agree with you. However, it feels like you're being a little too epistemically nihilistic here. This kind of speculative work always starts without confirmation or any particular reason to believe it beyond how well it matches the curve it was fit to. To quibble with a particular, this work doesn't attempt to predict the actions of super-intelligences, it's not making the claim that most civilizations will be "grabby", it's making predictions about what will happen under the assumption that some subpopulation of them decides to become grabby.
- titzer 5y ago
  I am not being nihilistic, I am being realistic. We really can't predict the actions of superintelligence, full stop. Here's what I mean. What makes the lights on your router blink? Usually, it's packets going in and out. At a coarse level, if your computer is doing something on the internet, the lights are gonna be blinking. But now increase the resolution, from seconds, to milliseconds, to microseconds, to nanoseconds. At a nanosecond level, whether the LED is on or off depends on whether a packet is going over the wire right this nanosecond. Whether a packet is earlier or later by a nanosecond is influenced by a bazillion variables: the router's internal state, its own operating system's scheduling, whether the devices connected to it are sending packets at it, dragging in their state, their scheduling, their programs, packet queues, etc. Gigabytes of state, changing nano-second to nano-second. Oh, and it's connected to the internet. When does the internet decide to send a packet to this router? If those are faster or slower by a nanosecond, it could be due the state along the entire path, all of which are servicing packets from all over the place, decided upon by super-complex distributed systems--datacenters full of them. So even the blinking of a router LED is connected to a supremely complex causality tangle. To understand the blinking nano-second to nano-second, you basically have to replicate the details of the entire internet. So, if I were to ask you instead to predict what will happen tomorrow at 3:33am--will your router be blinking, let's say, fast or slow? Who knows. A DDoS might crop up because someone was mad at Amazon, or maybe your OS downloads a software update, or maybe you are up watching Netflix, or maybe the whole thing is down because nuclear war. It's progressed so far from a simple bell curve that probabilistic reasoning, I would argue, is useless. And that's just without the internet being "intelligent"! Now assume the internet knows this prediction game is afoot, and it gets to bet too. The whole thing goes off the rails. No, we can't predict what aliens will do. This paper is a mathematical exercise that might apply to bacteria or video game characters, not interstellar civilizations.

by titzer 5y ago

brian_cloutier 5y ago

Your comment deserves a much longer response than I can give it, but you seem to be saying that it's impossible to know anything with any certainty, and that is absolutely true. I'm sure all the authors of this work would agree with you. Coming to conclusions/guesses requires starting with some assumptions about about how the universe works. It's important not to lose sight of those assumptions and very important to have a discussion about how likely each assumption is to be true. So to the extent that anyone is saying "this is definitely how the universe is", I wholeheartedly agree with you. However, it feels like you're being a little too epistemically nihilistic here. This kind of speculative work always starts without confirmation or any particular reason to believe it beyond how well it matches the curve it was fit to. To quibble with a particular, this work doesn't attempt to predict the actions of super-intelligences, it's not making the claim that most civilizations will be "grabby", it's making predictions about what will happen under the assumption that some subpopulation of them decides to become grabby.

titzer 5y ago
I am not being nihilistic, I am being realistic. We really can't predict the actions of superintelligence, full stop. Here's what I mean. What makes the lights on your router blink? Usually, it's packets going in and out. At a coarse level, if your computer is doing something on the internet, the lights are gonna be blinking. But now increase the resolution, from seconds, to milliseconds, to microseconds, to nanoseconds. At a nanosecond level, whether the LED is on or off depends on whether a packet is going over the wire right this nanosecond. Whether a packet is earlier or later by a nanosecond is influenced by a bazillion variables: the router's internal state, its own operating system's scheduling, whether the devices connected to it are sending packets at it, dragging in their state, their scheduling, their programs, packet queues, etc. Gigabytes of state, changing nano-second to nano-second. Oh, and it's connected to the internet. When does the internet decide to send a packet to this router? If those are faster or slower by a nanosecond, it could be due the state along the entire path, all of which are servicing packets from all over the place, decided upon by super-complex distributed systems--datacenters full of them. So even the blinking of a router LED is connected to a supremely complex causality tangle. To understand the blinking nano-second to nano-second, you basically have to replicate the details of the entire internet. So, if I were to ask you instead to predict what will happen tomorrow at 3:33am--will your router be blinking, let's say, fast or slow? Who knows. A DDoS might crop up because someone was mad at Amazon, or maybe your OS downloads a software update, or maybe you are up watching Netflix, or maybe the whole thing is down because nuclear war. It's progressed so far from a simple bell curve that probabilistic reasoning, I would argue, is useless. And that's just without the internet being "intelligent"! Now assume the internet knows this prediction game is afoot, and it gets to bet too. The whole thing goes off the rails. No, we can't predict what aliens will do. This paper is a mathematical exercise that might apply to bacteria or video game characters, not interstellar civilizations.