5 ms·

Show HN: Analysis of how long do items survive on the "New" page

brandnewlow 16y ago
Semi-related question: So how many submissions is HN getting each day these days?
- DupDetector 16y ago
  You can work that out. Start from http://news.ycombinator.com/newest http://news.ycombinator.com/newest go to the bottom of the page and click on "More" until one of the entries says "1 day ago", and there you are. As I write this there have been 282 items in the past 24 hours. That's on a Sunday, which as the charts show, is a slow day. Also just checked - the gap from "3 days ago" to "4 days ago" is 604 items. That's 604 items in a 24 hour period during the week.
  - robryan 16y ago
    Pretty much means unless something is instantly interesting to people it will fade into obscurity quickly. It's a tough balance, the main thing that gets missed a fair bit is ask hn posts where only a small amount of the community would be able to chime in on, if those people aren't around at the time they will never see it.
  - spicyj 16y ago
    Wrong account, RiderOfGiraffes!
blahedo 16y ago
Nifty, but the graphs need to be labelled better; it's not always clear what exactly the colours mean or what is being graphed.
- deleted 16y ago
  [deleted]
- RiderOfGiraffes 16y ago
  Agreed, I wondered about that. I was going to spend time doing that, but when I ran it past two friends they both said that by the time the understood the text, they didn't need the labels, and not having the labels made them spend time trying to understand the text. I guess it's a "Read like Math" issue. You need to read carefully to understand, or skim and get the gist. And I've moved on to the next stage now. I may come back to it if more people feel it's important.
scorchin 16y ago
Another great write-up! As a side note, I missed your juggling talk at the recent HN London meetup. Is there a video available for it?
sesqu 16y ago
The chart for the weekend specifically has just a few red points, which has shifted the range. Removing them would make the range better, but that's not really worth the time. What do you mean by this? Are you normalizing before coloring? If so, don't. You said the colors were for <100, 100-200, 200< and I'm holding you to it. As for the interpolation, I think you're sampling at least hourly, so why didn't you just store all times at each scrape and match the >60min ones with those?
- RiderOfGiraffes 16y ago
  It is normalised, but the range over which it's normalising is pretty near constant, so you don't need to worry. What I wanted to say was that without the isloated red points we might get to see more structure in the data, becuase we could expand the range. I may yet do that, and make careful notes in the text and annotations to explain it. Then again, I probably won't get time. Regarding the extrapolation, I hadn't thought of what you suggest, and it's a good idea. It would be non-trivial to implement quickly - I'll see. Thanks.
Anon84 16y ago
Is the data available? I saw the note at the end of the post, but there doesn't seem to be a link associated with it.
pmichaud 16y ago
Nice work, very informative. This is an issue that I feel has the potential to kill HN in the long run. It is steadily growing closer and closer to being impossible to hit the front page without gaming the system. The thing is that these submissions are coming in because there are more people, so in theory there should be a way to balance the increased submission rate with an increased voting rate as well. But how?
- epochwolf 16y ago
  require 200 karma for submission?
  - petercooper 16y ago
    Even requiring 3 or 4 karma would wipe out nearly all the deads (which I have turned on, sometimes good front page stuff gets deaded..) and 50 karma would get the page down to about half.
  - kgermino 16y ago
    The biggest problem with setting a minimum, particularly one that high is that it is extremely difficult to get a lot of karma from comments alone, especially for new users without name recognition.
    - cglee 16y ago
      That's the point: raise the bar by forcing the new users to hang around and get a feel for the culture before submitting. On top of that, acquiring 200 karma points on HN definitely doesn't fall into the "extremely difficult" category.
      - gommm 16y ago
        Yes but I think a lower amount like 50 would probably be enough or at least a good test before trying an higher threshold..
    - gaustin 16y ago

Semi-related question: So how many submissions is HN getting each day these days?

DupDetector 16y ago
You can work that out. Start from http://news.ycombinator.com/newest http://news.ycombinator.com/newest go to the bottom of the page and click on "More" until one of the entries says "1 day ago", and there you are. As I write this there have been 282 items in the past 24 hours. That's on a Sunday, which as the charts show, is a slow day. Also just checked - the gap from "3 days ago" to "4 days ago" is 604 items. That's 604 items in a 24 hour period during the week.
- robryan 16y ago
  Pretty much means unless something is instantly interesting to people it will fade into obscurity quickly. It's a tough balance, the main thing that gets missed a fair bit is ask hn posts where only a small amount of the community would be able to chime in on, if those people aren't around at the time they will never see it.
- spicyj 16y ago
  Wrong account, RiderOfGiraffes!

Nifty, but the graphs need to be labelled better; it's not always clear what exactly the colours mean or what is being graphed.

deleted 16y ago
[deleted]
RiderOfGiraffes 16y ago
Agreed, I wondered about that. I was going to spend time doing that, but when I ran it past two friends they both said that by the time the understood the text, they didn't need the labels, and not having the labels made them spend time trying to understand the text. I guess it's a "Read like Math" issue. You need to read carefully to understand, or skim and get the gist. And I've moved on to the next stage now. I may come back to it if more people feel it's important.

scorchin 16y ago

Another great write-up! As a side note, I missed your juggling talk at the recent HN London meetup. Is there a video available for it?

sesqu 16y ago

The chart for the weekend specifically has just a few red points, which has shifted the range. Removing them would make the range better, but that's not really worth the time. What do you mean by this? Are you normalizing before coloring? If so, don't. You said the colors were for <100, 100-200, 200< and I'm holding you to it. As for the interpolation, I think you're sampling at least hourly, so why didn't you just store all times at each scrape and match the >60min ones with those?

RiderOfGiraffes 16y ago
It is normalised, but the range over which it's normalising is pretty near constant, so you don't need to worry. What I wanted to say was that without the isloated red points we might get to see more structure in the data, becuase we could expand the range. I may yet do that, and make careful notes in the text and annotations to explain it. Then again, I probably won't get time. Regarding the extrapolation, I hadn't thought of what you suggest, and it's a good idea. It would be non-trivial to implement quickly - I'll see. Thanks.

Anon84 16y ago

Is the data available? I saw the note at the end of the post, but there doesn't seem to be a link associated with it.

pmichaud 16y ago

Nice work, very informative. This is an issue that I feel has the potential to kill HN in the long run. It is steadily growing closer and closer to being impossible to hit the front page without gaming the system. The thing is that these submissions are coming in because there are more people, so in theory there should be a way to balance the increased submission rate with an increased voting rate as well. But how?

epochwolf 16y ago
require 200 karma for submission?
- petercooper 16y ago
  Even requiring 3 or 4 karma would wipe out nearly all the deads (which I have turned on, sometimes good front page stuff gets deaded..) and 50 karma would get the page down to about half.
- kgermino 16y ago
  The biggest problem with setting a minimum, particularly one that high is that it is extremely difficult to get a lot of karma from comments alone, especially for new users without name recognition.
  - cglee 16y ago
    That's the point: raise the bar by forcing the new users to hang around and get a feel for the culture before submitting. On top of that, acquiring 200 karma points on HN definitely doesn't fall into the "extremely difficult" category.
    - gommm 16y ago
      Yes but I think a lower amount like 50 would probably be enough or at least a good test before trying an higher threshold..
  - gaustin 16y ago