4 ms·

> If there's a possibility of an asteroid coming that will wipe out life on earth, waiting until our telescopes can see it to start taking action may not give u

> If there's a possibility of an asteroid coming that will wipe out life on earth, waiting until our telescopes can see it to start taking action may not give us enough time to react. How would we even know what to do until we have a concrete target?

Vecr 3y ago
They can test deflection methods on other asteroids, build large spacecraft launchers and have them ready to go, do surveys to make sure the N-body predictions are as accurate as possible, etc. Some of that's being done (NEOWISE, DART) but if you want to deflect a planet killer you need a lot more.
NumberWangMan 3y ago
Vecr gives good analogies, but for AGI, I think investing a lot into alignment research, trying to set up incentive structures to actively avoid "arms races", establishing global oversight and agreements, having a "fire alarm" or process in place that a company can trigger if they develop an AI that they think is close to becoming dangerous, or even individuals. Strong whistleblower protections may help. Tracking GPUs so we know where compute is being gathered together. Starting talks with other countries like China, right now, to build a common understanding of how to deal with a potential problem. I think it's true that it may be very difficult to solve alignment theoretically, and solving it when we have real-world systems to look at may make it easier (but still probably not easy). But we need to be able to buy companies time so they can slow down as we approach what seems to be the "edge" of dangerous capability / consciousness / goal directedness, without feeling like they will lose out. And to be very strict about the conditions the models are trained under and what testing they undergo before release. A lot of this is stuff that people are already suggesting, but not everyone is taking the risk seriously.

Vecr 3y ago

They can test deflection methods on other asteroids, build large spacecraft launchers and have them ready to go, do surveys to make sure the N-body predictions are as accurate as possible, etc. Some of that's being done (NEOWISE, DART) but if you want to deflect a planet killer you need a lot more.

NumberWangMan 3y ago

Vecr gives good analogies, but for AGI, I think investing a lot into alignment research, trying to set up incentive structures to actively avoid "arms races", establishing global oversight and agreements, having a "fire alarm" or process in place that a company can trigger if they develop an AI that they think is close to becoming dangerous, or even individuals. Strong whistleblower protections may help. Tracking GPUs so we know where compute is being gathered together. Starting talks with other countries like China, right now, to build a common understanding of how to deal with a potential problem. I think it's true that it may be very difficult to solve alignment theoretically, and solving it when we have real-world systems to look at may make it easier (but still probably not easy). But we need to be able to buy companies time so they can slow down as we approach what seems to be the "edge" of dangerous capability / consciousness / goal directedness, without feeling like they will lose out. And to be very strict about the conditions the models are trained under and what testing they undergo before release. A lot of this is stuff that people are already suggesting, but not everyone is taking the risk seriously.