5 ms·

One could further raise an association with Nick Bostrom's vulnerable world hypothesis: > One way of looking at human creativity is as a process of pulling bal

One could further raise an association with Nick Bostrom's vulnerable world hypothesis: > One way of looking at human creativity is as a process of pulling balls out of a giant urn. The balls represent possible ideas, discoveries, technological inventions. Over the course of history, we have extracted a great many balls – mostly white (beneficial) but also various shades of gray (moderately harmful ones and mixed blessings). [..] What we haven’t extracted, so far, is a black ball: a technology that invariably or by default destroys the civilization that invents it. The reason is not that we have been particularly careful or wise in our technology policy. We have just been lucky. https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf

walleeee 5y ago
This is a useful thought experiment but it ignores the fact that the balls are more like a tangled mass of burrs, all variously interwoven Try to pull one out and you get a web of affordances and side effects and unforeseen interactions stuck to it If we were to be wiped out by our technology I see no guarantee some extradimensional coroner would be able to point definitively to just one Say it's climate heating, rapid-onset Venus mode. Was it the internal combustion engine? The coal plant? The automobile? The wheel? Language? Money? Industrialism? The limited liability corporation? How do you pick one? No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms
- gfodor 5y ago
  Some balls are more webby than others. Your counterpoint is valid but it is additive: the discrete balls exist too in cases where there are significant stepwise breakthroughs.
  - walleeee 5y ago
    Fair point, yes, the urn's contents are mereologically weird, neither wholly discrete nor continuous, liable to coalesce into orbs of manual or digital or social or intellectual utility which evolve in our minds and hands Also some balls (e.g. nuclear weapons, stray asteroids) make it a lot easier to point to something immediate and mostly unambiguous
- h0l0cube 5y ago
  > No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms I agree the analogy isn't perfect, but one can say that despite technologies not having a discreet origin, we can reason about them in retrospect. So maybe modern artificial neural networks were initially made possible by improvements in microscopy and experiments in cell culture to deduce the mechanisms of biological neurons, but we can reason about ANNs as a technology that is changing what is possible by a machine. I imagine Nick would also be inclined to be fatalistic about technological progress, in that these "black balls" will inevitably be found. His main stance is that we should be enacting the global ethical frameworks around these technologies much quicker.
  - walleeee 5y ago
    > despite technologies not having a discreet origin, we can reason about them in retrospect Yes, agreed, good point I also agree about Nick's probable fatalism, and I think his work is brilliant and valuable, however one thing irks me. He's made a successful career out of hypothetical "black ball" speculation while saying little about the potential for blackness of the tech we already have. He frames existential risk as if we are on the cusp of dramatic danger but still relatively safe. In truth we are already over the cliff & falling rapidly Take his paperclip maximizer. They already exist! We need not wait for some future AI to realize a badly misaligned objective function. We're already determined to transmute the remains of our homeworld's long dead into plastic and atmospheric carbon
- egypturnash 5y ago
  IIRC, when atomic bombs were first tested, there was some moderately compelling theorizing that they would set the entire planet's atmosphere on fire. That's a black ball.
  - Nadya 5y ago
    >there was some moderately compelling theorizing that they would set the entire planet's atmosphere on fire. A very popular story but it was also primarily 1 of the 8 scientists working on the Manhattan project and only because he hadn't had all the math showing how incredibly unlikely it would be shared with him. It was already a settled "No, not going to happen." before the first test was conducted. The story ends up being a popular soundbite but with no real teeth on it. [0] [0] https://blogs.scientificamerican.com/cross-check/bethe-teller-trinity-and-the-end-of-earth/ https://blogs.scientificamerican.com/cross-check/bethe-telle...
    - ALittleLight 5y ago
      I think the idea is, if the math had worked out the opposite way, then nuclear tech would be a black ball. If there was a bomb we could make and if it went off it would ignite the atmosphere and kill everyone - that would be a black ball.
narrator 5y ago
Nuclear weapons could have been that black ball.
- dougmwne 5y ago
  And still may yet be. The nukes haven't gone anywhere or become less world-ending. We are just having a moment where there's no serious risk of nuclear powers declaring all out war. But I don't recall inventing a new kind of diplomacy or geopolitics that makes war unthinkable.
- jtsiskin 5y ago
  This is what almost all of the essay describes
nostrademons 5y ago
One should also mention the anthropic principle in connection with this: the reason we haven't drawn a black ball is because we're here to observe that we still exist. Also of interest is the Fermi paradox: if (relatively high) estimates for the prevalence of extraterrestrial intelligence are correct, why haven't we found anyone? Taken together, it seems very likely that there is in fact a black ball in that urn, and we just happen to be one of the extraordinarily lucky planets that has not yet found it. Oddly, this is kinda freeing in a nihilistic way. If you know that you and your species are going to die eventually (which you do, the eventual fate of the universe is heat death), you can focus on living for the present instead of brooding on the inevitable death of mankind, and just be grateful that you and your species happen to be alive at this particular time on this particular planet where life is actually a thing.
- vagrantJin 5y ago
  > why haven't we found anyone? Space is big. Earth is old. When all is told, we just don't know.

walleeee 5y ago

This is a useful thought experiment but it ignores the fact that the balls are more like a tangled mass of burrs, all variously interwoven Try to pull one out and you get a web of affordances and side effects and unforeseen interactions stuck to it If we were to be wiped out by our technology I see no guarantee some extradimensional coroner would be able to point definitively to just one Say it's climate heating, rapid-onset Venus mode. Was it the internal combustion engine? The coal plant? The automobile? The wheel? Language? Money? Industrialism? The limited liability corporation? How do you pick one? No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms

gfodor 5y ago
Some balls are more webby than others. Your counterpoint is valid but it is additive: the discrete balls exist too in cases where there are significant stepwise breakthroughs.
- walleeee 5y ago
  Fair point, yes, the urn's contents are mereologically weird, neither wholly discrete nor continuous, liable to coalesce into orbs of manual or digital or social or intellectual utility which evolve in our minds and hands Also some balls (e.g. nuclear weapons, stray asteroids) make it a lot easier to point to something immediate and mostly unambiguous
h0l0cube 5y ago
> No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms I agree the analogy isn't perfect, but one can say that despite technologies not having a discreet origin, we can reason about them in retrospect. So maybe modern artificial neural networks were initially made possible by improvements in microscopy and experiments in cell culture to deduce the mechanisms of biological neurons, but we can reason about ANNs as a technology that is changing what is possible by a machine. I imagine Nick would also be inclined to be fatalistic about technological progress, in that these "black balls" will inevitably be found. His main stance is that we should be enacting the global ethical frameworks around these technologies much quicker.
- walleeee 5y ago
  > despite technologies not having a discreet origin, we can reason about them in retrospect Yes, agreed, good point I also agree about Nick's probable fatalism, and I think his work is brilliant and valuable, however one thing irks me. He's made a successful career out of hypothetical "black ball" speculation while saying little about the potential for blackness of the tech we already have. He frames existential risk as if we are on the cusp of dramatic danger but still relatively safe. In truth we are already over the cliff & falling rapidly Take his paperclip maximizer. They already exist! We need not wait for some future AI to realize a badly misaligned objective function. We're already determined to transmute the remains of our homeworld's long dead into plastic and atmospheric carbon
egypturnash 5y ago
IIRC, when atomic bombs were first tested, there was some moderately compelling theorizing that they would set the entire planet's atmosphere on fire. That's a black ball.
- Nadya 5y ago
  >there was some moderately compelling theorizing that they would set the entire planet's atmosphere on fire. A very popular story but it was also primarily 1 of the 8 scientists working on the Manhattan project and only because he hadn't had all the math showing how incredibly unlikely it would be shared with him. It was already a settled "No, not going to happen." before the first test was conducted. The story ends up being a popular soundbite but with no real teeth on it. [0] [0] https://blogs.scientificamerican.com/cross-check/bethe-teller-trinity-and-the-end-of-earth/ https://blogs.scientificamerican.com/cross-check/bethe-telle...
  - ALittleLight 5y ago
    I think the idea is, if the math had worked out the opposite way, then nuclear tech would be a black ball. If there was a bomb we could make and if it went off it would ignite the atmosphere and kill everyone - that would be a black ball.

narrator 5y ago

Nuclear weapons could have been that black ball.

dougmwne 5y ago
And still may yet be. The nukes haven't gone anywhere or become less world-ending. We are just having a moment where there's no serious risk of nuclear powers declaring all out war. But I don't recall inventing a new kind of diplomacy or geopolitics that makes war unthinkable.
jtsiskin 5y ago
This is what almost all of the essay describes

nostrademons 5y ago

One should also mention the anthropic principle in connection with this: the reason we haven't drawn a black ball is because we're here to observe that we still exist. Also of interest is the Fermi paradox: if (relatively high) estimates for the prevalence of extraterrestrial intelligence are correct, why haven't we found anyone? Taken together, it seems very likely that there is in fact a black ball in that urn, and we just happen to be one of the extraordinarily lucky planets that has not yet found it. Oddly, this is kinda freeing in a nihilistic way. If you know that you and your species are going to die eventually (which you do, the eventual fate of the universe is heat death), you can focus on living for the present instead of brooding on the inevitable death of mankind, and just be grateful that you and your species happen to be alive at this particular time on this particular planet where life is actually a thing.

vagrantJin 5y ago
> why haven't we found anyone? Space is big. Earth is old. When all is told, we just don't know.