4 ms·

The HN Effect in Numbers

42 points by hoop 16y ago

_delirium 16y ago
It'd be interesting if there were a way to measure proportion of visitors from different sources who engaged, by some definition of engagement. Ideally I want to know "read the article", but that's hard to measure. Number of new feed subscriptions might be a decent proxy for "read and really liked the article", though. One reason I want to know that is that for my own sites, I've generally found a large proportion of link-blog traffic to be very shallow, to the point where maybe one in 100 hits is a "real" hit. This is particularly the case at link-blogs without any summary blurbs. Slashdotters who don't want to read the article, for example, often just read the blurb and don't bother to click through, so a bigger proportion of those who do click through are at least somewhat interested in reading. A much larger percentage of people on sites like Reddit click through, but I don't think a much larger percentage actually read the articles; many are just clicking through to figure out what the article's about, since there's no blurb to explain it. And some don't even do that much, since a common Reddit use pattern is "middle-click on every article to open in new tab, then work through the tabs", often culminating in just closing the remaining still-unread tabs when you're done reddit-surfing, which results in a ton of spurious hits that involved no human eyeballs. Anecdotally, the proportion of shallow-hits/not-even-real-hits goes up somewhat proportionally to traffic. Front-page of reddit delivers maybe 100x as much traffic as front-page of a more niche sub-reddit, but much less than 100x real readership.
- hoop 16y ago
  Interesting point! I'm guilty of that Reddit use pattern and, in fact, wish HN had a similar toolbar so I could remember what link it was that I opened in the first place :) Feedburner (now owned by Google) tracks stats on # of subscribers and is kind enough to let you see the history. In my case, I started out with about 10 subscribers and reached 44 on launch day. Today, I'm looking at 24 subscribers.
- riffer 16y ago
  I can share a little data. Yesterday we got 7157 uniques for this: http://news.ycombinator.com/item?id=1627619 http://news.ycombinator.com/item?id=1627619 It has 76 votes, so that's pretty close to the 10 HN votes -> 1k uniques rule of thumb Of those 7157, 602 clicked on at least one of two links at the bottom of the blog post to do more on the site, but I'd be surprised if more than a handful return in the future (other than thru another blog post).
  - hoop 16y ago
    I hadn't heard the 10HN votes -> 1k uniques rule of thumb before but that seems spot on. The article I wrote seems to have come pretty close as well with ~9k uniques @ ~100 votes http://news.ycombinator.com/item?id=1600693 http://news.ycombinator.com/item?id=1600693
  - _delirium 16y ago
    My own data for a recent /r/programming submission: Around 3500 uniques, from net +35 score (50 upvotes, 15 downvotes), of which around 200 clicked through to something else on the site (even if only loading the domain's root page), and around 100 clicked through to at least one other article. Seems like ratios in the same ballpark. Unfortunately, it's still hard to tell how much of that is substantial engagement. I can imagine a decent number of people skimmed the first article, clicked to see what else was on the site, skimmed another article, lost interest and left, especially since reading reddit puts you in a sort of "idly click on things" mood, at least in my experience. I did get one email about the article with some thought-provoking comments, so that puts a lower bound of one engaged reader. ;-)
    - Encosia 16y ago
      > Unfortunately, it's still hard to tell how much of that is substantial engagement. A pinging stats provider, like Clicky, gives you great insight into single-pageview engagement. It's true that there's room for inaccuracy in the pinging measure. Some viewers with third-party (or all) JavaScript disabled won't be counted, viewers that leave the page open in a tab before they look at it may be over-counted, and readers who legitimately spend more than ten minutes reading a longer page are capped at the ten minute mark. However, averaging that across thousands of views is a very worthwhile measure of engagement. I had a relevant post about that here recently (which was castigated for semantics more than it was actually discussed, unfortunately): http://news.ycombinator.com/item?id=1294969 http://news.ycombinator.com/item?id=1294969 The Clicky pinging data was invaluable then too. HN visitors to that post show an abysmal 95% bounce rate and an average 14s time-on-site in Google Analytics. Meanwhile, Clicky shows those same HN visitors spending an average of 3m29s time-on-site and truly bouncing from the page only 12.9% of the time. Even if you assume Clicky's numbers are wildly optimistic, it's nice to have that alternate insight.
cperciva 16y ago
Good to see that HN Daily shows up in the referers, even if it is just 1% of the direct HN numbers -- I really had no idea how many people would end up using it, but it seems that there's a few people at least.
- jacquesm 16y ago
  It varies quite a bit actually, on some postings as much as 5%, others around 1%. edit: It looks like the higher the original is voted the fewer people will visit through HN daily, which suggests that people use it more to see if they've missed something than to look at just HN daily. edit2: > I really had no idea how many people would end up using it, but it seems that there's a few people at least. That's a curious statement, can't you tell from the logs?
  - cperciva 16y ago
    Thanks for the numbers. Yes, this did-I-miss-anything functionality is how I was hoping it would be used -- good to hear that it's working!
    - jacquesm 16y ago
      I don't think 1 to 5% is bad at all considering how long you've been live with that and that it is not backlinked from HN in any other way than the announcement. Probably referrer logs are one of your main sources of new visitors.
  - cperciva 16y ago
    That's a curious statement, can't you tell from the logs? In raw numbers, yes. But I don't know how much traffic HN gets, so the raw numbers don't tell me much.
holman 16y ago
The numbers from when I've been on HN is similar to yours. Traffic itself isn't super important to me, though; mostly I'm just happy knowing the people coming in from Hacker News might end up following me on Twitter, following me on GitHub, or just generally retweet and reblog my stuff. It's a great crew. Hacker News may not generate as many clicks as Digg would, but visitors from here are way more interesting to me.
hoop 16y ago
I've been trying to see if I can fairly-accurately match the number of visitors I had with some type of mathematical expression. As I pointed out in the article, the decay looks exponential. I've gotten closer with f(x) = 2658e^(-0.94)+30, but as x approaches infinity (or even 5) it starts to full apart. Any one good with the maths who wants to take a look? I decided with this one to throw out the first datapoint (the day my site launched) since it was likely still growing during this period of (overall) decay. How significant is this? The actual numbers: [2688, 1065, 452, 206, 138, 105] My function's numbers: [2688, 1068, 436, 188, 91, 54] For easy playing: http://www.wolframalpha.com/input/?i=table[+2658e^%28-0.94x%29+%2B+30%2C+{x%2C0%2C5}+ http://www.wolframalpha.com/input/?i=table[+2658e^%28-0.94x%...]
deltapoint 16y ago
If you're post is only on the new section and never makes the front page it will still get a bit of traffic. http://abiekatz.com/2010/08/20/introducing-a-new-theory-the-recommendularity/ http://abiekatz.com/2010/08/20/introducing-a-new-theory-the-... It got 5 karma points and 29 page views from Hacker News. So making the front page is very important for getting traffic from HN. I am sure this is the case for any social news site.

42 points by hoop 16y ago

It'd be interesting if there were a way to measure proportion of visitors from different sources who engaged, by some definition of engagement. Ideally I want to know "read the article", but that's hard to measure. Number of new feed subscriptions might be a decent proxy for "read and really liked the article", though. One reason I want to know that is that for my own sites, I've generally found a large proportion of link-blog traffic to be very shallow, to the point where maybe one in 100 hits is a "real" hit. This is particularly the case at link-blogs without any summary blurbs. Slashdotters who don't want to read the article, for example, often just read the blurb and don't bother to click through, so a bigger proportion of those who do click through are at least somewhat interested in reading. A much larger percentage of people on sites like Reddit click through, but I don't think a much larger percentage actually read the articles; many are just clicking through to figure out what the article's about, since there's no blurb to explain it. And some don't even do that much, since a common Reddit use pattern is "middle-click on every article to open in new tab, then work through the tabs", often culminating in just closing the remaining still-unread tabs when you're done reddit-surfing, which results in a ton of spurious hits that involved no human eyeballs. Anecdotally, the proportion of shallow-hits/not-even-real-hits goes up somewhat proportionally to traffic. Front-page of reddit delivers maybe 100x as much traffic as front-page of a more niche sub-reddit, but much less than 100x real readership.

hoop 16y ago
Interesting point! I'm guilty of that Reddit use pattern and, in fact, wish HN had a similar toolbar so I could remember what link it was that I opened in the first place :) Feedburner (now owned by Google) tracks stats on # of subscribers and is kind enough to let you see the history. In my case, I started out with about 10 subscribers and reached 44 on launch day. Today, I'm looking at 24 subscribers.
riffer 16y ago
I can share a little data. Yesterday we got 7157 uniques for this: http://news.ycombinator.com/item?id=1627619 http://news.ycombinator.com/item?id=1627619 It has 76 votes, so that's pretty close to the 10 HN votes -> 1k uniques rule of thumb Of those 7157, 602 clicked on at least one of two links at the bottom of the blog post to do more on the site, but I'd be surprised if more than a handful return in the future (other than thru another blog post).
- hoop 16y ago
  I hadn't heard the 10HN votes -> 1k uniques rule of thumb before but that seems spot on. The article I wrote seems to have come pretty close as well with ~9k uniques @ ~100 votes http://news.ycombinator.com/item?id=1600693 http://news.ycombinator.com/item?id=1600693
- _delirium 16y ago
  My own data for a recent /r/programming submission: Around 3500 uniques, from net +35 score (50 upvotes, 15 downvotes), of which around 200 clicked through to something else on the site (even if only loading the domain's root page), and around 100 clicked through to at least one other article. Seems like ratios in the same ballpark. Unfortunately, it's still hard to tell how much of that is substantial engagement. I can imagine a decent number of people skimmed the first article, clicked to see what else was on the site, skimmed another article, lost interest and left, especially since reading reddit puts you in a sort of "idly click on things" mood, at least in my experience. I did get one email about the article with some thought-provoking comments, so that puts a lower bound of one engaged reader. ;-)
  - Encosia 16y ago
    > Unfortunately, it's still hard to tell how much of that is substantial engagement. A pinging stats provider, like Clicky, gives you great insight into single-pageview engagement. It's true that there's room for inaccuracy in the pinging measure. Some viewers with third-party (or all) JavaScript disabled won't be counted, viewers that leave the page open in a tab before they look at it may be over-counted, and readers who legitimately spend more than ten minutes reading a longer page are capped at the ten minute mark. However, averaging that across thousands of views is a very worthwhile measure of engagement. I had a relevant post about that here recently (which was castigated for semantics more than it was actually discussed, unfortunately): http://news.ycombinator.com/item?id=1294969 http://news.ycombinator.com/item?id=1294969 The Clicky pinging data was invaluable then too. HN visitors to that post show an abysmal 95% bounce rate and an average 14s time-on-site in Google Analytics. Meanwhile, Clicky shows those same HN visitors spending an average of 3m29s time-on-site and truly bouncing from the page only 12.9% of the time. Even if you assume Clicky's numbers are wildly optimistic, it's nice to have that alternate insight.

cperciva 16y ago

Good to see that HN Daily shows up in the referers, even if it is just 1% of the direct HN numbers -- I really had no idea how many people would end up using it, but it seems that there's a few people at least.

jacquesm 16y ago
It varies quite a bit actually, on some postings as much as 5%, others around 1%. edit: It looks like the higher the original is voted the fewer people will visit through HN daily, which suggests that people use it more to see if they've missed something than to look at just HN daily. edit2: > I really had no idea how many people would end up using it, but it seems that there's a few people at least. That's a curious statement, can't you tell from the logs?
- cperciva 16y ago
  Thanks for the numbers. Yes, this did-I-miss-anything functionality is how I was hoping it would be used -- good to hear that it's working!
  - jacquesm 16y ago
    I don't think 1 to 5% is bad at all considering how long you've been live with that and that it is not backlinked from HN in any other way than the announcement. Probably referrer logs are one of your main sources of new visitors.
- cperciva 16y ago
  That's a curious statement, can't you tell from the logs? In raw numbers, yes. But I don't know how much traffic HN gets, so the raw numbers don't tell me much.

holman 16y ago

The numbers from when I've been on HN is similar to yours. Traffic itself isn't super important to me, though; mostly I'm just happy knowing the people coming in from Hacker News might end up following me on Twitter, following me on GitHub, or just generally retweet and reblog my stuff. It's a great crew. Hacker News may not generate as many clicks as Digg would, but visitors from here are way more interesting to me.

hoop 16y ago

I've been trying to see if I can fairly-accurately match the number of visitors I had with some type of mathematical expression. As I pointed out in the article, the decay looks exponential. I've gotten closer with f(x) = 2658e^(-0.94)+30, but as x approaches infinity (or even 5) it starts to full apart. Any one good with the maths who wants to take a look? I decided with this one to throw out the first datapoint (the day my site launched) since it was likely still growing during this period of (overall) decay. How significant is this? The actual numbers: [2688, 1065, 452, 206, 138, 105] My function's numbers: [2688, 1068, 436, 188, 91, 54] For easy playing: http://www.wolframalpha.com/input/?i=table[+2658e^%28-0.94x%29+%2B+30%2C+{x%2C0%2C5}+ http://www.wolframalpha.com/input/?i=table[+2658e^%28-0.94x%...]

deltapoint 16y ago

If you're post is only on the new section and never makes the front page it will still get a bit of traffic. http://abiekatz.com/2010/08/20/introducing-a-new-theory-the-recommendularity/ http://abiekatz.com/2010/08/20/introducing-a-new-theory-the-... It got 5 karma points and 29 page views from Hacker News. So making the front page is very important for getting traffic from HN. I am sure this is the case for any social news site.