5 ms·

The Well-Poisoning Machine

71 points by _Microft 4y ago

1attice 4y ago
Pitch-perfect. To the mis-en-scene, I could only add: the erotism that is behind technological accelerationism. What I mean is that there is a sort of visceral, cthonic attraction to the idea of creating 'new life', even as the ecosystems that support actual living systems grow frail. Just as Oppenheimer made the bomb in part because he couldn't resist the attraction of solving the intellectual problems inherent in its design, we're poisoning the well not just to do it first, but because we are hot for well-poisoning. It's such an interesting intellectual problem! Poisons are beautiful.
- 1attice 4y ago
  After typing this, I realized that there's a great movie that captures this exactly: _Dr. Strangelove: Or I how I Learned to Stop Worrying and Love the Bomb_. The eponymous character's boner-like n*zi salute, irrepressible when discussing nukes, reminds me of the irrepressible boner-like response that the most brilliant minds I know have to GPT-4. Me included. The other cultural touchstone the current vibe reminds me of is Saruman telling Gandalf about the inevitability of Sauron's reign. That same glitter in the eye: "We have work to do."
  - destroy-2A 4y ago
    Mr. President, we cannot allow a mineshaft gap!
    - 1attice 4y ago
      Could someone who is actually good at such things please deepfake me Sam Altman's face atop the dude who bullrides the falling nuke in the film's final scene?
      - hgsgm 4y ago
        Ask Sam's AI to do it for you.
  - hgsgm 4y ago
    Now think of the current generation of AI programmers who were born after the Cold War ended, who don't understand Dr Strangelove.
- nonbirithm 4y ago
  This is a problem that I hope attracts much more scrutiny in the near future, and beyond the domain of what's called "AI" right now. It doesn't feel talked about much at all. It feels like we can analyze the inner workings of new technology endlessly, because shiny new tech is cool and racks our brains, but hardly ever stop to consider why we keep pressing on to make even more advances. "Because we can" isn't a satisfactory answer, especially not when the stakes are so high. Why do we keep saying that? Can we ever be totally satisfied with what we already have available to use? Does this imply there will never be a stopping point? Would we go beyond inventing AGI if such a thing were possible? I think the idea of resistance to change to preserve our chances of survival is worthy of study. I hope there will be serious anthropological studies investigating the nature of that kind of resistance and the opposite, driving insatiability of unbounded, overextended curiosity that marches forward no matter the cost - even if the chances of it affecting the final outcome are slim. It doesn't seem like there's much of an alternative besides repeating "pandora's box, cat out of the bag, genie out of the bottle" as a post-hoc justification for every set of unintended consequences for the rest of time. The fact that we have imaginations and can foresee the potential consequences of actions has helped our species for so long up to this point. But if safety regulations needing to be written in blood to be effective is really the way our species is most likely to operate, then that has terrifying implications. I think 10 years from now a television crew will sit down the original researchers and engineers behind this tech branded AI and ask: Why didn't you stop?
layer8 4y ago
The machine, in any case, is well poised to change the world.
mnemotronic 4y ago
Ok. I'm dense. I can't understand what this is about. Is this a threat?
- jacquesm 4y ago
  I read it as a way to express the frustration about the speed with which AI tools are unleashed without proper consideration for the consequences 'just because we can'. We can't allow a well poisoning machine gap to come into existence.
  - evouga 4y ago
    Seems like a flagrant strawman to characterize AI as a “well-poisoning machine.”
    - jacquesm 4y ago
      Let's wait and see. I can see plenty of ways in which that could come true.
    - 1attice 4y ago
      It's not a strawman, it's an analogy. In order to be a man of straw, it would have to claim that AI "just is" a well-poisoning machine (in the literal sense of a machine that inserts poison into a hole in the ground containing H2O). Now, that alone does not mean that it goes through; the analogy works as an intuition pump, and those can be flakey (see https://en.wikipedia.org/wiki/Intuition_pump https://en.wikipedia.org/wiki/Intuition_pump) But nor does it mean that a fallacy has been committed
    - tpoacher 4y ago
      Not AI as a whole. Particular activities to do with AI which would generally be deemed not only dangerous, but also not having any clear non-malicious purpose. So if there was a strawman argument here at all, it's literally the one you just made!
- 1attice 4y ago
  This is about Sam Altman types rushing to win the ML race. We can reasonably expect civ ('the village') to be disrupted ('poisoned'), but instead of that counting as a reason to not do it, it counts --- somehow --- as a reason to do it harder, so we do it first.
textninja 4y ago
Good rhetorical imagery, but ultimately an emotional argument with a subtext loaded with assumptions: 1) that there is an unpoisoned well to begin with 2) that the chemicals are poison or can only be used for poison 3) that increased access to chemicals will result in more poisoned wells 4) that the people who invented the chemicals are benevolent stewards and will not just use them to further their own interests 5) that it is better for aforementioned benevolent stewards to hoard the chemicals than permit access to those who would concoct poisons, antidotes, and preventative measures, not to mention industry innovations that aren’t related to poison at all 6) that the interests of the people on the ethics committees are aligned with the interests of humanity 7) that ethics committees employed by a chemical manufacturing firms is the only thing standing between ethical and responsible management of chemicals and complete chaos and environmental destruction 8) that the fired ethicists were good at their jobs in the first place
- 1attice 4y ago
  This argument does not feel emotional to me, it feels like an intuition pump (https://en.wikipedia.org/wiki/Intuition_pump https://en.wikipedia.org/wiki/Intuition_pump). Moral arguments almost always work by analogy, and this is one of them. 1) This assumption does not affect the example. If the well is already somewhat poisoned, making it more poisoned is still Bad. 2) The chemicals are definitely poisonous, but also have non-poisonous uses. 'The does makes the poison,' as Paracelsus put it. You still don't want them in your well water, typically. 3) Increased presence of poisonous chemicals in the well is literally what is presupposed by the example -- it's not an assumption, it's a premise. 4) This argument is anti-chemical-in-wellwater, it in no way touches on 'chemical stewards'. It's weird that you introduce this here -- effectively, you're modifying the analogy, and then pointing to the modified analogy and claiming it doesn't work. 5) See 4 6) See 4 7) This is a pretty reasonable assumption, given that there is no 'Ministry of AI'. 8) The fired ethicists may or may not have been good at their jobs, but no ethicists is probably worse than fallible ethicists. You want to deflate the wallop this analogy packs, but I don't think you succeed. EDIT @ 19:10PDT: added ', it feels like an intuition pump. ...'
  - textninja 4y ago
draw_down 4y ago
[dead]

71 points by _Microft 4y ago

Pitch-perfect. To the mis-en-scene, I could only add: the erotism that is behind technological accelerationism. What I mean is that there is a sort of visceral, cthonic attraction to the idea of creating 'new life', even as the ecosystems that support actual living systems grow frail. Just as Oppenheimer made the bomb in part because he couldn't resist the attraction of solving the intellectual problems inherent in its design, we're poisoning the well not just to do it first, but because we are hot for well-poisoning. It's such an interesting intellectual problem! Poisons are beautiful.

1attice 4y ago
After typing this, I realized that there's a great movie that captures this exactly: _Dr. Strangelove: Or I how I Learned to Stop Worrying and Love the Bomb_. The eponymous character's boner-like n*zi salute, irrepressible when discussing nukes, reminds me of the irrepressible boner-like response that the most brilliant minds I know have to GPT-4. Me included. The other cultural touchstone the current vibe reminds me of is Saruman telling Gandalf about the inevitability of Sauron's reign. That same glitter in the eye: "We have work to do."
- destroy-2A 4y ago
  Mr. President, we cannot allow a mineshaft gap!
  - 1attice 4y ago
    Could someone who is actually good at such things please deepfake me Sam Altman's face atop the dude who bullrides the falling nuke in the film's final scene?
    - hgsgm 4y ago
      Ask Sam's AI to do it for you.
- hgsgm 4y ago
  Now think of the current generation of AI programmers who were born after the Cold War ended, who don't understand Dr Strangelove.
nonbirithm 4y ago
This is a problem that I hope attracts much more scrutiny in the near future, and beyond the domain of what's called "AI" right now. It doesn't feel talked about much at all. It feels like we can analyze the inner workings of new technology endlessly, because shiny new tech is cool and racks our brains, but hardly ever stop to consider why we keep pressing on to make even more advances. "Because we can" isn't a satisfactory answer, especially not when the stakes are so high. Why do we keep saying that? Can we ever be totally satisfied with what we already have available to use? Does this imply there will never be a stopping point? Would we go beyond inventing AGI if such a thing were possible? I think the idea of resistance to change to preserve our chances of survival is worthy of study. I hope there will be serious anthropological studies investigating the nature of that kind of resistance and the opposite, driving insatiability of unbounded, overextended curiosity that marches forward no matter the cost - even if the chances of it affecting the final outcome are slim. It doesn't seem like there's much of an alternative besides repeating "pandora's box, cat out of the bag, genie out of the bottle" as a post-hoc justification for every set of unintended consequences for the rest of time. The fact that we have imaginations and can foresee the potential consequences of actions has helped our species for so long up to this point. But if safety regulations needing to be written in blood to be effective is really the way our species is most likely to operate, then that has terrifying implications. I think 10 years from now a television crew will sit down the original researchers and engineers behind this tech branded AI and ask: Why didn't you stop?

layer8 4y ago

The machine, in any case, is well poised to change the world.

mnemotronic 4y ago

Ok. I'm dense. I can't understand what this is about. Is this a threat?

jacquesm 4y ago
I read it as a way to express the frustration about the speed with which AI tools are unleashed without proper consideration for the consequences 'just because we can'. We can't allow a well poisoning machine gap to come into existence.
- evouga 4y ago
  Seems like a flagrant strawman to characterize AI as a “well-poisoning machine.”
  - jacquesm 4y ago
    Let's wait and see. I can see plenty of ways in which that could come true.
  - 1attice 4y ago
    It's not a strawman, it's an analogy. In order to be a man of straw, it would have to claim that AI "just is" a well-poisoning machine (in the literal sense of a machine that inserts poison into a hole in the ground containing H2O). Now, that alone does not mean that it goes through; the analogy works as an intuition pump, and those can be flakey (see https://en.wikipedia.org/wiki/Intuition_pump https://en.wikipedia.org/wiki/Intuition_pump) But nor does it mean that a fallacy has been committed
  - tpoacher 4y ago
    Not AI as a whole. Particular activities to do with AI which would generally be deemed not only dangerous, but also not having any clear non-malicious purpose. So if there was a strawman argument here at all, it's literally the one you just made!
1attice 4y ago
This is about Sam Altman types rushing to win the ML race. We can reasonably expect civ ('the village') to be disrupted ('poisoned'), but instead of that counting as a reason to not do it, it counts --- somehow --- as a reason to do it harder, so we do it first.

textninja 4y ago

Good rhetorical imagery, but ultimately an emotional argument with a subtext loaded with assumptions: 1) that there is an unpoisoned well to begin with 2) that the chemicals are poison or can only be used for poison 3) that increased access to chemicals will result in more poisoned wells 4) that the people who invented the chemicals are benevolent stewards and will not just use them to further their own interests 5) that it is better for aforementioned benevolent stewards to hoard the chemicals than permit access to those who would concoct poisons, antidotes, and preventative measures, not to mention industry innovations that aren’t related to poison at all 6) that the interests of the people on the ethics committees are aligned with the interests of humanity 7) that ethics committees employed by a chemical manufacturing firms is the only thing standing between ethical and responsible management of chemicals and complete chaos and environmental destruction 8) that the fired ethicists were good at their jobs in the first place

1attice 4y ago
This argument does not feel emotional to me, it feels like an intuition pump (https://en.wikipedia.org/wiki/Intuition_pump https://en.wikipedia.org/wiki/Intuition_pump). Moral arguments almost always work by analogy, and this is one of them. 1) This assumption does not affect the example. If the well is already somewhat poisoned, making it more poisoned is still Bad. 2) The chemicals are definitely poisonous, but also have non-poisonous uses. 'The does makes the poison,' as Paracelsus put it. You still don't want them in your well water, typically. 3) Increased presence of poisonous chemicals in the well is literally what is presupposed by the example -- it's not an assumption, it's a premise. 4) This argument is anti-chemical-in-wellwater, it in no way touches on 'chemical stewards'. It's weird that you introduce this here -- effectively, you're modifying the analogy, and then pointing to the modified analogy and claiming it doesn't work. 5) See 4 6) See 4 7) This is a pretty reasonable assumption, given that there is no 'Ministry of AI'. 8) The fired ethicists may or may not have been good at their jobs, but no ethicists is probably worse than fallible ethicists. You want to deflate the wallop this analogy packs, but I don't think you succeed. EDIT @ 19:10PDT: added ', it feels like an intuition pump. ...'
- textninja 4y ago

draw_down 4y ago

[dead]