4 ms·
Thanks for sharing! Very interesting. > The correlation is actually not bad (0.53), but our model is very consistently over-estimating the score at the low end
by youoy 2y ago
Thanks for sharing! Very interesting.
> The correlation is actually not bad (0.53), but our model is very consistently over-estimating the score at the low end, and underestimating it at the high end. This is surprising; some variation on any given data point is expected, but such a consistent mis-estimation trend isn’t what we’d expect.
This is a consequence on the model objective. If you don't know what is really happening, a good way of reducing the overall error is to do that. If you instead try to exactly predict the very highs and very lows, you can see that you will get very high errors on those, resulting in a bigger overall error.
Appart from that, I want to comment on AI alignment here. For me the objective of "most up votes" is not fully correlated with where I get the most value on HN. Most of the time, the most up voted I would have found them anyway on other platforms. It's the middle range what I really like. So be careful implementing this algorithm at scale, it could turn the website into another platform with shitty AI recommendations.
- kcorbitt 2y ago> For me the objective of "most up votes" is not fully correlated with where I get the most value on HN. Most of the time, the most up voted I would have found them anyway on other platforms. Yes, this is a fantastic point. I'm curious if there's some other measurable proxy metric for "things I get the most value out of on HN"? Upvotes seems like the most natural but optimizing for it too strongly would definitely take HN down a dark path.
- losteric 2y agoPerhaps selecting for posts with the highest quality reply engagement? If many different people were drawn to lengthy discussions, that suggests the content sparks thoughts that others then feel compelled to engage with. Or select for the emotional content of replies, awe/empathy/anger, depending on what one wants from HN?
- coolcoder613 2y agoPerhaps number of comments, or number of non-flamewar comments, or proportion of flamewar comments together with number of comments?