On a more specific level, it's possible to test formalized versions of cost-benefit analysis as a planning methodology. In some setting (company? education? res

On a more specific level, it's possible to test formalized versions of cost-benefit analysis as a planning methodology. In some setting (company? education? research?) have the experimental groups make decisions according to strict guidelines for how to quantify costs and benefits and weigh the two, and the control groups make decisions based on traditional "holistic management" where subject-matter experts have wide latitude to make recommendations, after perusing data, but not according to a strictly quantified methodology. See how their performance compares over some suitable period. What I think this would mainly be testing is not really the philosophical question of whether we ought to weigh costs and benefits, but whether the erroneously-captured and un-captured information in quantification is a bigger or smaller problem, from the perspective of decision-making, than the potential bias and unrepeatability of a subject-matter expert's case-by-case judgment. Some of my skepticism of quantitative cost-benefit analysis is that it's basically GIGO: many quantifications are so inaccurate as to actually make quantitative cost-benefit analysis worse than "expert judgment" at decision-making, which is one reason we can't just replace all decision-makers with computers applying mechanical decision-theoretic rules yet, except in specific areas (like some kinds of medical diagnosis) where we actually do have very good data. But it's not impossible to test in a particular setting whether that's true. And here we are discussing on Y Combinator's website, who are in a perfect position to run such an experiment! They could choose some companies to fund via the traditional method, "Paul Graham & friends' expert judgment on how to allocate money, based on their history and experience", and another portion of the pool on a more quantitative basis. Then see which pool is more successful.