3 ms·
The plot in the article is an example of a "reliability diagram" frequently used in weather forecast verification. See, e.g., http://www.bom.gov.au/wmo/lrfvs/re
by emkemp 12y ago
The plot in the article is an example of a "reliability diagram" frequently used in weather forecast verification. See, e.g., http://www.bom.gov.au/wmo/lrfvs/reliability.shtml http://www.bom.gov.au/wmo/lrfvs/reliability.shtml.
Reliability is considered separate from accuracy in meteorology -- the former evaluates success conditioned on what was forecasted, while the latter is an unconditional evaluation of success or failure.
Other facets of forecast "goodness" exist and are often considered in meteorology. A seminal paper on the subject was penned by Allan Murphy, who identified three types of "goodness" (consistency, quality, and value) and ten subsets of quality (including reliability and accuracy). See http://www.glerl.noaa.gov/seagrant/ClimateChangeWhiteboard/Resources/Uncertainty/Mac1/murphy93PR.pdf http://www.glerl.noaa.gov/seagrant/ClimateChangeWhiteboard/R.... [PDF warning]
A popular companion to the reliability diagram is the Relative Operating Characteristics (ROC) curve. Here different forecast probability thresholds are tested to calculate likelihood of success if the event occurred, and likelihood of error if the event did not occur. This evaluates what Murphy calls discrimination (forecast quality conditioned what was observed) which complements reliability. See, e.g., http://www.bom.gov.au/wmo/lrfvs/roc.shtml http://www.bom.gov.au/wmo/lrfvs/roc.shtml.
Curiously, accuracy tends to take a back seat in forecast verification to other aspects of quality, particularly in rare event situations. This trend first began in the mid 1880's with the "Finley Affair", a series of published articles debating how to evaluate tornado forecasts issued by the US Army Signal Corps. Murphy published a fascinating literature review on the subject and showed that many of the skill scores and debates born during the Finley Affair are still active today. See http://www.nssl.noaa.gov/users/brooks/public_html/feda/papers/murphy96.pdf http://www.nssl.noaa.gov/users/brooks/public_html/feda/paper.... [PDF warning]