That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many ar

That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.