Maybe. I think the idea is that it's preferable to have wasted a lot of effort on coming up with irrelevant abstractions addressing a nonexistent problem, than

Maybe. I think the idea is that it's preferable to have wasted a lot of effort on coming up with irrelevant abstractions addressing a nonexistent problem, than to be surprised by something weird and poorly-understood presenting novel and not-clearly-bounded risk and not have effective tools at hand to address it. I won't claim to know what the right amount of effort directed toward this is though, nor do I really think AI safety research has a >epsilon chance of accomplishing its goal if it turns out to be relevant at all. ¯\_(ツ)_/¯