Some benefits can be measured, some can't. Mosquito nets can be measured. Interventions to reduce AI risk can never be measured, because we'll only ever get one

Some benefits can be measured, some can't. Mosquito nets can be measured. Interventions to reduce AI risk can never be measured, because we'll only ever get one data point (many years from now) and evaluating an intervention needs at least two data points. I don't think anyone working on, say, AI risk is claiming that their work is 1.37x more valuable than mosquito nets. Rather, they claim that there are plausible assumptions under which it's highly valuable. If I also find their assumptions plausible, I'm now persuaded they're working on something worthwhile. Great! It distinguishes them from lots of people working on stuff that definitely isn't. Utilitarianism doesn't need all the numbers to be precise. You really only need to get the signs right to improve your decision making. Often, you can get different signs for overall utility from two different assumptions that are both plausible. That's an important signal for which assumptions to worry about. The assumptions that only change the magnitude by a digit or two aren't that important.