I think you sum up their position's fallacy pretty nicely here: >"if we don't figure out AGI safety now, by the time AGI happens it may be too late" The keywo

I think you sum up their position's fallacy pretty nicely here: >"if we don't figure out AGI safety now, by the time AGI happens it may be too late" The keyword for me is "happens". It's as if technology ever happens, or emerges serendipitously. It's like the Kurzweilian exponential law which make it seem as if there is no agency in technology, a natural law. And our role in it is make sure when the aliens or the gods arrive we are prepared for them.