3 ms·
As I've commented on HN over the past month or so and shared with dang via email, I've been running some analyses on the HN front page based on a full history[1
by dredmorbius 3y ago
As I've commented on HN over the past month or so and shared with dang via email, I've been running some analyses on the HN front page based on a full history[1]. One aspect I've looked at is the "flamewar detector", or as dang's described it elsewhere, and "overheated discussion" metric[3] --- not necessarily bad, but a sign that things might be getting tendentious. I've taken to calling it "spiciness".
In particular, it seems to me that this might penalise stories in which there is a genuine level of disagreement or contentiousness, or which are socially, politically, and/or culturally contested.[4]
As I understand, it kicks in where a post has >= 40 comments and comments > votes.[2]
Given that I have ~188k front page stories with vote and comment counts, it's pretty straightforward to calculate the comments:votes ratio, and look at patterns that emerge.
Looking at means, we can get, first, the overall ratio and its percentile distribution, and the overall story position (not surprisingly, 14, which is close to 15, the midpoint of the 1--30 range):
Overall ratio
n: 115908, sum: 73618, min: 0.25, max: 21.00, mean: 0.635141, median: 0.52, sd: 0.419485
%-ile:
5: 0.28, 10: 0.3, 15: 0.33,
20: 0.35, 25: 0.38, 30: 0.4,
35: 0.43, 40: 0.46, 45: 0.49,
55: 0.56, 60: 0.6, 65: 0.64,
70: 0.69, 75: 0.75, 80: 0.83,
85: 0.92, 90: 1.07, 95: 1.34
Note that both the mean and median ratios are well below the detector threshold of 1.0. By standard deviation (percentage of values included with a normal distribution in parentheses):
1σ: 1.055 (68%)
2σ: 1.474 (95%)
3σ: 1.894 (99.73%)
Note that the normal distribution values seem to agree with the calculated percentiles. And that HN's flamewar detector kicks off for stories roughly 1σ > mean spiciness.
I pulled up the list of the 40 most "ratioed" (or "spiciest") HN threads of all time:
rank ratio date-pos year pos DoW user vote com site
1 21.00 2007-4-18.18 2007 18 Wed juwo 5 105 juwo.com
2 10.83 2008-2-4.26 2008 26 Mon eusman 6 65 n/a
3 9.33 2007-4-26.26 2007 26 Thu juwo 3 28 juwo.com
4 9.14 2007-7-31.13 2007 13 Tue rokhayakebe 7 64 n/a
5 9.00 2007-7-22.21 2007 21 Sun gibsonf1 3 27 streamfocus.com
6 7.50 2007-4-20.25 2007 25 Fri jkush 6 45 n/a
7 7.50 2007-6-1.10 2007 10 Fri keiretsu 6 45 n/a
8 7.33 2007-3-26.20 2007 20 Mon BitGeek 3 22 n/a
9 7.29 2008-1-27.13 2008 13 Sun wumi 14 102 n/a
10 7.27 2008-7-22.29 2008 29 Tue BenS 15 109 n/a
11 7.22 2008-9-25.17 2008 17 Thu DanielBMarkham 23 166 n/a
12 6.79 2008-11-14.19 2008 19 Fri bbuderi 24 163 xconomy.com
13 6.77 2008-3-28.17 2008 17 Fri btw0 13 88 n/a
14 6.57 2007-10-9.20 2007 20 Tue _h4xr 7 46 f2s.com
15 6.50 2007-10-13.15 2007 15 Sat robmnl 6 39 n/a
16 6.50 2007-11-3.16 2007 16 Sat breck 4 26 qcrunch.com
17 6.50 2008-1-18.13 2008 13 Fri german 14 91 n/a
18 6.33 2007-12-10.24 2007 24 Mon kf 6 38 independent.ie
19 6.29 2007-9-5.10 2007 10 Wed rokhayakebe 14 88 n/a
20 6.25 2007-7-1.13 2007 13 Sun danw 4 25 n/a
21 6.20 2007-5-24.15 2007 15 Thu bootload 5 31 n/a
22 6.20 2007-6-11.9 2007 9 Mon danw 5 31 n/a
23 6.07 2007-2-21.1 2007 1 Wed pg 262 1590 n/a
24 6.00 2007-11-13.30 2007 30 Tue trekker7 5 30 n/a
25 5.99 2022-5-26.8 2022 8 Thu LoveGracePeace 280 1678 nih.gov
26 5.91 2008-5-5.30 2008 30 Mon rrival 11 65 n/a
27 5.85 2008-4-1.15 2008 15 Tue robmnl 27 158 n/a
28 5.82 2007-8-19.11 2007 11 Sun kf 11 64 n/a
29 5.80 2007-8-21.4 2007 4 Tue zenobo 20 116 n/a
30 5.78 2007-4-6.1 2007 1 Fri kkim 77 445 paulgraham.com
31 5.77 2008-4-18.10 2008 10 Fri Flemlord 26 150 n/a
32 5.75 2007-11-18.18 2007 18 Sun rokhayakebe 8 46 n/a
33 5.70 2008-2-12.22 2008 22 Tue theremora 10 57 n/a
34 5.64 2007-10-17.9 2007 9 Wed altay 14 79 n/a
35 5.55 2008-1-17.18 2008 18 Thu sspencer 11 61 n/a
36 5.50 2007-10-23.25 2007 25 Tue DanielBMarkham 6 33 whattofix.com
37 5.50 2007-4-18.25 2007 25 Wed yaacovtp 4 22 n/a
38 5.50 2007-4-7.17 2007 17 Sat me_jobs_r_u_woz 4 22 n/a
39 5.42 2008-3-4.29 2008 29 Tue kulkarnic 12 65 n/a
40 5.42 2008-5-24.7 2008 7 Sat _bbks 38 206 n/a
"n/a" site is a post without a URL, typically an "ask" or "tell" thread.
Eyeballing that list, one pattern that is suggested is that high-ratio posts seem to have relatively high story position values (that is, they appear low on the HN front page), notable exceptions being #23 (pg's 2007-2-21 request), and #30 (kf's 2007-8-21 post, a pg essay "Microsoft is Dead"). I'd done further analysis of story position by spiciness and found that in general for stories which make the front page, higher spiciness actually correlates slightly with a higher page position (lower StoryPos value). I suspect this is due to specific moderator interaction, that is, those stories which survive do so because they're deemed appropriate to HN.
Another is that high-ratio posts are notably clustered in 2007-2008. Frequency analysis suggests that flamewar detection was implemented around January, 2009 as spicy front-page posts thin markedly after that month. The one item appearing after that date ... ticks numerous hot buttons (2022-5-26.8).
Because of how I'm collecting and analysing data I don't have HN or article URLs directly available, though it's possible to determine the article from the front-page archive. "2008-5-24.7" corresponds to the 7th-position story on the 2008-5-24 front page archive: <https://news.ycombinator.com/front?day=2008-05-24 https://news.ycombinator.com/front?day=2008-05-24>
I also took a look at the highest and lowest ratioed sites, with >= 20 front-page appearances, appearing in a follow-up comment.
Overall, the detector seems reasonably justified, though not infallible. It's a much better indicator than I'd thought it would be.
________________________________
Notes:
1. 20 April 2007 -- 21 June 2023 presently. I re-up the archive periodically, but haven't fully automated this. Results change little incrementally.
2. Via ColinWright 2 years ago here: <https://news.ycombinator.com/item?id=25871251 https://news.ycombinator.com/item?id=25871251>
3. <https://news.ycombinator.com/item?id=16020089 https://news.ycombinator.com/item?id=16020089>
4. As it happens, HN tends to downrate each of these topics as dang's discussed over the years. Links ... not immediately available.
- dredmorbius 3y agoTop 20 spiciest sites (20+ front-page appearances): site n points comms ratio apnews.com 36 14674 17512 1.193 sfchronicle.com 25 5771 6174 1.070 variety.com 24 5479 4992 0.911 mattmaroon.com 73 3332 3023 0.907 axios.com 92 38075 34150 0.897 bizjournals.com 20 2183 1959 0.897 cnbc.com 174 59983 53056 0.885 apple.com 241 99945 88396 0.884 reason.com 70 13143 11614 0.884 nypost.com 28 5851 5088 0.870 markevanstech.com 22 290 251 0.866 macrumors.com 62 18700 16162 0.864 nikkei.com 56 17568 15174 0.864 economist.com 829 119205 102702 0.862 thewalrus.ca 30 6194 5199 0.839 techradar.com 30 7227 6053 0.838 backreaction.blogspot.com 33 7209 5968 0.828 strongtowns.org 27 8279 6857 0.828 mondaynote.com 45 7581 6268 0.827 coindesk.com 22 10236 8355 0.816 Bottom 20 least spicy sites (20+ front-page appearances): site n points comms ratio particletree.com 37 997 227 0.228 brendangregg.com 40 11135 2512 0.226 intruders.tv 28 324 73 0.225 aphyr.com 34 8514 1910 0.224 andrewchen.typepad.com 51 757 168 0.222 michaelnielsen.org 31 3335 723 0.217 igvita.com 38 3626 767 0.212 startuplessonslearned.blo 24 1101 232 0.211 citusdata.com 51 8361 1717 0.205 ferd.ca 21 5883 1132 0.192 ocks.org 27 6036 1120 0.186 tensorflow.org 22 5612 1020 0.182 aosabook.org 21 3899 669 0.172 ocw.mit.edu 41 8793 1500 0.171 david.weebly.com 20 1364 226 0.166 jslogan.com 24 97 16 0.165 burningdoor.com 23 149 23 0.154 linusakesson.net 26 4531 684 0.151 github.com/0xax 22 2168 121 0.056