4 ms·

Anyone read the source and find the 98% accuracy stat? The source link has a UTM source of ChatGPT. Could it be a hallucination?

astrea 1y ago
I believe it’s seeing the 98% mortality rate in 2020 vs 2019 for the same month.
yesfitz 1y ago
Maybe more of a misinterpretation by AI and/or author than full on hallucination. "Life insurers can predict when you'll die with about 98% accuracy." "98%" appears in the citation[1], but as the ratio of actual deaths to expected deaths. (i.e. 98% of the deaths they expected actually occurred.) Some months that figure was ~104%, so it's not a measure of accuracy. 1: https://www.soa.org/4aa060/globalassets/assets/files/resources/research-report/2024/rpec-mort-improvement-update.pdf https://www.soa.org/4aa060/globalassets/assets/files/resourc...
nrenegar 1y ago
It's extremely incorrect. 98% accuracy on when "you'll" die would imply an R^2 of 0.98 for individual-level lifespan predictions. We are nowhere close to that.
- neaden 1y ago
  It could mean a lot of things. It could mean "We think you'll die at 87+-5 years" and counting it as a success if you die anywhere in that age range for instance which I could believe they can predict the vast majority of the time. I doubt they are referring to the R Squared.
subarctic 1y ago
I saw that too and thought the utm_source=chatgpt was pretty funny, since I only noticed it when I was pasting the link into chatgpt myself to get it to tell me where the "98% accurate" claim was coming from. Anyway, it's a pretty vague statement. What it sounds like it's saying is for the average person they can predict when they will die, and you have to decide if they mean the day, the year or the decade. But chatgpt gave me an interpretation that seems to make more sense: > The 98 % figure is about aggregate forecasting, not clairvoyance. It means that when an insurer predicts, say, 10 000 deaths across its book in 2024, the actual count typically falls within roughly ±200. For any single customer, the prediction is still just a probability curve, not a calendar appointment with the Grim Reaper.

astrea 1y ago

I believe it’s seeing the 98% mortality rate in 2020 vs 2019 for the same month.

yesfitz 1y ago

Maybe more of a misinterpretation by AI and/or author than full on hallucination. "Life insurers can predict when you'll die with about 98% accuracy." "98%" appears in the citation[1], but as the ratio of actual deaths to expected deaths. (i.e. 98% of the deaths they expected actually occurred.) Some months that figure was ~104%, so it's not a measure of accuracy. 1: https://www.soa.org/4aa060/globalassets/assets/files/resources/research-report/2024/rpec-mort-improvement-update.pdf https://www.soa.org/4aa060/globalassets/assets/files/resourc...

nrenegar 1y ago

It's extremely incorrect. 98% accuracy on when "you'll" die would imply an R^2 of 0.98 for individual-level lifespan predictions. We are nowhere close to that.

neaden 1y ago
It could mean a lot of things. It could mean "We think you'll die at 87+-5 years" and counting it as a success if you die anywhere in that age range for instance which I could believe they can predict the vast majority of the time. I doubt they are referring to the R Squared.

subarctic 1y ago

I saw that too and thought the utm_source=chatgpt was pretty funny, since I only noticed it when I was pasting the link into chatgpt myself to get it to tell me where the "98% accurate" claim was coming from. Anyway, it's a pretty vague statement. What it sounds like it's saying is for the average person they can predict when they will die, and you have to decide if they mean the day, the year or the decade. But chatgpt gave me an interpretation that seems to make more sense: > The 98 % figure is about aggregate forecasting, not clairvoyance. It means that when an insurer predicts, say, 10 000 deaths across its book in 2024, the actual count typically falls within roughly ±200. For any single customer, the prediction is still just a probability curve, not a calendar appointment with the Grim Reaper.