4 ms·
It isn't just human bias that makes this problematic. Their definition of success is that a prediction of 70% should be right about 70/100 times. Give a model
by _ea1k 2y ago
It isn't just human bias that makes this problematic. Their definition of success is that a prediction of 70% should be right about 70/100 times.
Give a model a big reward for "success" under this criteria and it will have a tendency to converge on the long term average. As long as the long term average stays the same, no one will be able to say the estimate was wrong.
If you've ever seen the YT videos from aspiring data scientists learning sequence prediction, you've inevitably seen a variation from people trying to predict stock prices. "Look how low the loss is!" - Sure, but it gets the direction wrong, and just consistently predicts a close value to the previous day's close. But the model is technically correct, just useless.
A 50-50 model for an election might not be giving up and might not be wrong, but it is often useless.
- gowld 2y ago50-50 means "I have 0 no useful information". Some facts are more useful than others.
- rurp 2y agoForecasts will always have some level of uncertainty and if a race is very close a prediction around 50/50 will be the most honest result. A prediction like that does have value, just like a prediction that candidate A is a large favorite, it's informing you about the future state of the world. I'm not sure what you're expecting forecasters to do when they have a 50/50 result. Put a thumb on the scale in a random direction so that you will think they are providing useful information, even though they'll be less accurate?