3 ms·
I think this might be conflating confidence with accuracy. I tried leaving the slider the the middle (nominally the least confident position) and it gave a scor
by reltnek 6mo ago
I think this might be conflating confidence with accuracy. I tried leaving the slider the the middle (nominally the least confident position) and it gave a score of 0.25 and diagnosed it as 'overconfident'.
- convexly 6mo agoThat is definitely a bug, thank you for pointing that out. Should have been neutral! I'll push a fix for this.
- macleginn 6mo agoThe Brier score is pathological when the guess is 0.5: regardless of the outcome, it will be equal to 0.25, so if you define "better than random" as having a score < 0.25, actually acting randomly makes you "overconfident".