This article also interesting "Nobel Prize winner Daniel Kahneman is out with a new book on why we all make such bad judgments" https://www.cnbc.com/2021/05/1

This article also interesting "Nobel Prize winner Daniel Kahneman is out with a new book on why we all make such bad judgments" https://www.cnbc.com/2021/05/14/nobel-prize-winner-daniel-kahneman-is-out-with-a-new-book-on-why-we-all-make-such-bad-judgments.html https://www.cnbc.com/2021/05/14/nobel-prize-winner-daniel-ka... _"Noise audits"_ "For example, if members of a radiology team provide widely different analysis of the same X-ray of the same patient, that is a noise problem. Kahneman describes a protocol for a noise audit that involves studying how a group of experts in a firm (he suggests a minimum of 12 participants) react to two or three realistic case studies on an individual basis. Each member would have to summarize the case and the judgment either numerically (in dollars, percentiles, or probabilities) or on some scale with at least five degrees (such as “very strong,” “strong,” “average,” “poor,” or “very poor”). They would not be allowed to communicate with each other. The executives who assemble the case study are surveyed beforehand to assess how much they expect their experts to agree on any given case, and what level of disagreement would be acceptable. If the results diverge significantly from expectations, then there is a noise problem."