3 ms·

I know you don't particularly subscribe to the "AGI is extremely dangerous" line of thinking, but let's say for the sake of argument we're a decade in the futur

by rl3 11y ago

I know you don't particularly subscribe to the "AGI is extremely dangerous" line of thinking, but let's say for the sake of argument we're a decade in the future and it's starting to become very evident that it in fact is dangerous—like in a recursively self-improving nightmare type way—and it could be realized via a few thousand AWS instances with the application of some very specialized knowledge. What do you imagine the U.S. Government's reaction might be?

fsam 11y ago
I know that this is completely off topic, but for the sake of argument, let's imagine that we are thirty years into the future, we have invented efficient teleportation and have colonized mars. What do you think the government should do to encourage family ties to remain strong when loved ones are living on another planet?
- rl3 11y ago
  While I appreciate you mocking me, I can't help but disagree with the implication that my post was completely off-topic. AndrewKemendo has conducted research into AGI on behalf of the military. My hypothetical was intended as a near-term scenario in which the technology proved far more dangerous than originally thought. Asking him how he thinks USG would react to such a scenario doesn't strike me as unreasonable given his background.
  - fsam 11y ago
    Did you really expect to get a response other than "It's basically impossible to have a plan of response for something that nobody even knows how to build"? I'm just tired of hearing unproductive questions like this to which any response other than "we don't know" is literally science fiction. Andrew's response would have applied equally to teleportation. Why don't we talk instead about how methods such as deep learning actually do work, and what problems that have been successfully applied to?
    - AndrewKemendo 11y ago
      Why don't we talk instead about how methods such as deep learning actually do work, and what problems that have been successfully applied to? Well we do, but it's clear to the community that we won't get to AGI with deep learning classifiers and systems alone. So the questions we are asking are "what would a system look like that results in X kind of behavior." I don't disagree with your teleportation analogy either, but I think you weight it too heavily with impossibility. In fact there are serious people working on teleportation - at this point it's quantum state teleportation [1] but it's a start. [1] http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.70.1895 http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.70....
AndrewKemendo 11y ago
I know you don't particularly subscribe to the "AGI is extremely dangerous" line of thinking I think the question in my mind is "dangerous to who and when?" Is AGI an existential threat? Probably. But on what time horizon? Through what mechanism? And can we evolve and collaborate with AGI instead of de-facto competing with it? None of the AGI warning people (Bostrom, Yudkowski, Barrat et al.) have come up with a plausible chain of events that leads to human irrelevance or extinction. They always make a few assumptions and then claim "and then exponential growth happens" and boom everyone's a paperclip. The USG doesn't have a position at this point and it's ill prepared to react. In reality if there is some kind of world ending gray goo scenario that an AGI creates - nothing the US or any other Government can do will matter. I think that's about as likely as Roko's Basilisk being true though (eg. 0% chance).
- rl3 11y ago
  >The USG doesn't have a position at this point and it's ill prepared to react. That's what I thought. Kind of bizzare considering USG usually tends to stay far ahead of the curve when dealing with technology that has potentially profound national security implications. Yet at the same time, programs such as DARPA SyNAPSE exist (though only at moderate funding levels). >In reality if there is some kind of world ending gray goo scenario that an AGI creates - nothing the US or any other Government can do will matter. For sure. My hypothetical was intended more as taking place prior to any such disaster—in a world where AGI is viewed in the same light as WMDs, but where nothing catastrophic has yet happened. In that scenario, USG could potentially have great influence (for better or worse) over any outcome. For my money, counter-proliferation measures would be ultimately worthless in that situation, except as a stop-gap to buy time for a larger project to solve AGI correctly.
  - AndrewKemendo 11y ago
    Kind of bizzare considering USG usually tends to stay far ahead of the curve when dealing with technology that has potentially profound national security implications. It's basically impossible to have a plan of response for something that nobody even knows how to build. That said, we have a lot crisis action plans that might apply depending on whatever actions are happening. It would likely fall into the realm of contingency plans that have "Complex electromagnetic environments" as core assumptions.

by rl3 11y ago

fsam 11y ago

I know that this is completely off topic, but for the sake of argument, let's imagine that we are thirty years into the future, we have invented efficient teleportation and have colonized mars. What do you think the government should do to encourage family ties to remain strong when loved ones are living on another planet?

rl3 11y ago
While I appreciate you mocking me, I can't help but disagree with the implication that my post was completely off-topic. AndrewKemendo has conducted research into AGI on behalf of the military. My hypothetical was intended as a near-term scenario in which the technology proved far more dangerous than originally thought. Asking him how he thinks USG would react to such a scenario doesn't strike me as unreasonable given his background.
- fsam 11y ago
  Did you really expect to get a response other than "It's basically impossible to have a plan of response for something that nobody even knows how to build"? I'm just tired of hearing unproductive questions like this to which any response other than "we don't know" is literally science fiction. Andrew's response would have applied equally to teleportation. Why don't we talk instead about how methods such as deep learning actually do work, and what problems that have been successfully applied to?
  - AndrewKemendo 11y ago
    Why don't we talk instead about how methods such as deep learning actually do work, and what problems that have been successfully applied to? Well we do, but it's clear to the community that we won't get to AGI with deep learning classifiers and systems alone. So the questions we are asking are "what would a system look like that results in X kind of behavior." I don't disagree with your teleportation analogy either, but I think you weight it too heavily with impossibility. In fact there are serious people working on teleportation - at this point it's quantum state teleportation [1] but it's a start. [1] http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.70.1895 http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.70....

AndrewKemendo 11y ago

I know you don't particularly subscribe to the "AGI is extremely dangerous" line of thinking I think the question in my mind is "dangerous to who and when?" Is AGI an existential threat? Probably. But on what time horizon? Through what mechanism? And can we evolve and collaborate with AGI instead of de-facto competing with it? None of the AGI warning people (Bostrom, Yudkowski, Barrat et al.) have come up with a plausible chain of events that leads to human irrelevance or extinction. They always make a few assumptions and then claim "and then exponential growth happens" and boom everyone's a paperclip. The USG doesn't have a position at this point and it's ill prepared to react. In reality if there is some kind of world ending gray goo scenario that an AGI creates - nothing the US or any other Government can do will matter. I think that's about as likely as Roko's Basilisk being true though (eg. 0% chance).

rl3 11y ago
>The USG doesn't have a position at this point and it's ill prepared to react. That's what I thought. Kind of bizzare considering USG usually tends to stay far ahead of the curve when dealing with technology that has potentially profound national security implications. Yet at the same time, programs such as DARPA SyNAPSE exist (though only at moderate funding levels). >In reality if there is some kind of world ending gray goo scenario that an AGI creates - nothing the US or any other Government can do will matter. For sure. My hypothetical was intended more as taking place prior to any such disaster—in a world where AGI is viewed in the same light as WMDs, but where nothing catastrophic has yet happened. In that scenario, USG could potentially have great influence (for better or worse) over any outcome. For my money, counter-proliferation measures would be ultimately worthless in that situation, except as a stop-gap to buy time for a larger project to solve AGI correctly.
- AndrewKemendo 11y ago
  Kind of bizzare considering USG usually tends to stay far ahead of the curve when dealing with technology that has potentially profound national security implications. It's basically impossible to have a plan of response for something that nobody even knows how to build. That said, we have a lot crisis action plans that might apply depending on whatever actions are happening. It would likely fall into the realm of contingency plans that have "Complex electromagnetic environments" as core assumptions.