I think this is specifically coming to terms with an insight that's taught to statisticians about a bias-variance tradeoff. From my understanding, in a statist

I think this is specifically coming to terms with an insight that's taught to statisticians about a bias-variance tradeoff. From my understanding, in a statistical setting, low variability in bias leads to high variability in variance whereas low variability in variance leads to high variability in bias. The example I saw was with K-means, where K = N leads to high variance (the predicted cluster is highly variable) but low bias (take an input point, you get that exact input point back), vs. K=1 low variance (there's only one cluster) but bad bias (input point is far away from the cluster center/representative point). I'm not sure I've characterized it well but there's a Twitter post from Alicia Curth that explains it [0] as well as a paper that goes into it [1]. [0] https://x.com/AliciaCurth/status/1841817856142348529 https://x.com/AliciaCurth/status/1841817856142348529 [1] https://arxiv.org/abs/2409.18842 https://arxiv.org/abs/2409.18842