3 ms·

I don't think it's odd. I agree that optimising for the wrong thing (or, probably, for any particular thing) can be harmful, especially when it's done on a very

I don't think it's odd. I agree that optimising for the wrong thing (or, probably, for any particular thing) can be harmful, especially when it's done on a very large scale. AI-as-we-know-it doesn't introduce a new class of risk, it just makes some of the optimisations faster. I don't see any point in thinking about AGI risk until we get some experimental evidence that AGI is possible

convexfunction 9y ago
Maybe. I think the idea is that it's preferable to have wasted a lot of effort on coming up with irrelevant abstractions addressing a nonexistent problem, than to be surprised by something weird and poorly-understood presenting novel and not-clearly-bounded risk and not have effective tools at hand to address it. I won't claim to know what the right amount of effort directed toward this is though, nor do I really think AI safety research has a >epsilon chance of accomplishing its goal if it turns out to be relevant at all. ¯\_(ツ)_/¯