7 ms·

Show HN: Hacker News 2018, Year in Review

harrisreynolds 8y ago
Hey guys! We've been working on an idea of data stories here at Chartly and created this Hacker News story based on data from the community in 2018. It is not perfect, but was fun to create and I am happy with how it turned out. Would love to hear any feedback you have!
- THE_PUN_STOPS 8y ago
  This was a delightful way to show off your product. Kudos.
  - harrisreynolds 8y ago
    Thank you! I appreciate it!
- stevenicr 8y ago
  using firefox at 1044 x 687, the purple box next to the github and nytimes box... has "67 comments" visible.. kind of looks like something is supposed to be above that, shows something when I highlight the line. inspecting the element, it's whit text on white bg.. "Poll: Would you prefer" I'm sure this is not a common breakpoint and browser, so I wouldn't worry too much about it. Everything else is pretty cool - I did not know there were so many of so many things.
minimaxir 8y ago
> It would be cool if we could use the public Big Query data set in Google cloud, but it is missing some data that we'd like to report on. Namely users as well as enhancing the posts with fields like (domain name of a story). The BigQuery dataset has both of the fields necessary to do the analysis, so I'm confused. (you do need to use the NET.HOST function to extract a HN-esque domain name from the url; more info in this notebook: https://www.kaggle.com/minimaxir/hacker-news-submission-score-predictor https://www.kaggle.com/minimaxir/hacker-news-submission-scor...)
- harrisreynolds 8y ago
  Thanks for the link! Will take a look. Would love to dump our Redshift instance!
  - sbr464 8y ago
    I'm curious, what's the current size of your instance (disk space/# records) for this dataset? I'd be interested in a dump of the raw data, to save having to access via the API. I'd be willing to pay for the bandwidth/temp charges if any.
  - scapecast 8y ago
    What is it that makes you want to dump Redshift? Sounds like there's a specific frustration?
    - harrisreynolds 8y ago
      I actually like Redshift fine. But we are paying about $250/month for it just to host this data and if it was already in a publicly available database it would be nice to drop that. But using Redshift for this has been great for us in terms of testing our tool.
bovermyer 8y ago
This is fascinating. I love data stories.
jamestimmins 8y ago
This looks great! The most interesting behavior reflected here IMO is the folks who submit lots and lots of links (presumably) in the hunt for points. The allure of Fake Internet Points is fascinating. Btw minor typo: "discussiom of Karma"
jenscow 8y ago
Interesting how the top 4 scoring posts are about deaths...
dev1n 8y ago
I'm wondering how the user KMinshew was the top job poster for the year when the most recent comment of theirs was from 2014.
lifeisstillgood 8y ago
What seriously there are 1/2 million users on here? I honestly thought it was in the 10's of thousands tops. I think it says something about how it is possible to converse with a few people in a crowded room but not notice the crowd.
- shapiro92 8y ago
  there was a recent study for reddit showing that actually most users dont do anything and the whole content is generated from not even the 10%
  - Zitrax 8y ago
    Would be interesting to know how many of those 500 000 that were active during 2018 (at least one login).
    - harrisreynolds 8y ago
      I ran a query using Chartly and it came back with ~138K users that had at least one post (story or comment) during 2018. So maybe around 28% or so were active.
echelon 8y ago
> 53+ thousand new users This might explain why my perception of average comment quality has dropped. It's purely anecdotal, but it feels as if the volume of joke posts and "me too" responses has increased well beyond what it was five years ago. Comments have always been the most valuable part of HN for me. It's frankly starting to feel a little bit like Reddit around here, and I'm concerned about the signal:noise ratio dropping any further.
- diamondo25 8y ago
  Yes, this is an issue of anything popular. Not that it is a bad thing, as it keeps the community alive, but it feels "not the same" and might drive users away, like what happened on Facebook.
- dvno42 8y ago
  Perhaps a few tweaks can be made to the platform to help ensure HN's "core values" (that sounds silly) are passed on to new users. For the most part there's always been a good herd mentality to negate the one off low quality posts and a bit of public shaming "We don't talk like that around here...". Maybe a "Posting Guidelines" page and more weight added to the down-vote button to those that are privileged to have it could help.
- cordonbleu 8y ago
  there are sockpuppets rings abounding, likely there are black/grey hat SEO spammers trying to elevate thier live[valid,working] link count to game the g00gle search results, and then there are people that want to comment, civilly and substantiavely but have been somehow cut off from doing so, thus create a throwaway login for that purpose just to make thier voice heard. and then there are trolls and the indignant that just like to derail a normal conversation because its "funny". I have to deal with these on my chat board often but not at large scale. There are of course, normal users that are trying to have a constructive discourse, i believe there is ample signature to flag the differences to a discriminating mind, or someone with admin/mod status and access to the management tools.
- wyxuan 8y ago
  There are some positive aspects, as it allows for more diversified opinions on various issues. I find that it is great that there are chicken farmers, bridge builders, etc on this platform. But I agree. There have been less and less articles on directly tech relating things, and more on general world news. Some of the things are important and worth posting, but HN has become a little bit of a general news aggregator.
smsm42 8y ago
It's a bit depressing that out of top 5 posts, two are about deaths, one about prominent figure retiring and two are about companies being swallowed by other companies.
- scarface74 8y ago
  Of course death is sad, but we are all going to die one day. The people who died and were recognized made a meaningful impact on the world.
  - smsm42 8y ago
    I don't argue with that, but the overall negative vibe of top 5 is obvious to me. I mean we could have at least a couple of "this new great thing happened" there? Of course, corporate mergers may be good news too for some people, so maybe that's not full negative.
csbartus 8y ago
Lately I’m reading HN as a replacement for tv and newspapers, for world news. I’m not saying HN is not fun anymore it’s just the value is elsewhere. A guess is that slowly I’ll replace HN entirely. It’s too distractive and no essence is captured. Lobste.rs in contrast is a place where I go excited.
- purple_ducks 8y ago
  Completely off topic but I clicked through your profile (just to see if you were lobste.rs employee - (cynical I know)) Anyway, on your website, you have a 4 picture gallery with the selectors vertically at the side but it was a bit jarring as I'm used to those vertical selectors usually used to display for vertical scrolling. Even ignoring the above, the photos transition horizontally which is even more jarring (when the vertical selectors exist). <unwarranted-2-cents/>
alwayseptember 8y ago
I, for one, welcome our new users. But seriously, a couple of points worth discussing: 1. Moderation feels much more automated, with default rules cutting broad robotic swaths into (or more often out of) discussion. 2. If HN is this popular, it's leaning closer and closer to social networking platform, with at least one key point: you cannot delete things, but they can. 3. Both of these facts feed into throwaway accounts. Thus, "users" are not all truly users. The numbers don't tell the whole story.

Hey guys! We've been working on an idea of data stories here at Chartly and created this Hacker News story based on data from the community in 2018. It is not perfect, but was fun to create and I am happy with how it turned out. Would love to hear any feedback you have!

THE_PUN_STOPS 8y ago
This was a delightful way to show off your product. Kudos.
- harrisreynolds 8y ago
  Thank you! I appreciate it!
stevenicr 8y ago
using firefox at 1044 x 687, the purple box next to the github and nytimes box... has "67 comments" visible.. kind of looks like something is supposed to be above that, shows something when I highlight the line. inspecting the element, it's whit text on white bg.. "Poll: Would you prefer" I'm sure this is not a common breakpoint and browser, so I wouldn't worry too much about it. Everything else is pretty cool - I did not know there were so many of so many things.

minimaxir 8y ago

> It would be cool if we could use the public Big Query data set in Google cloud, but it is missing some data that we'd like to report on. Namely users as well as enhancing the posts with fields like (domain name of a story). The BigQuery dataset has both of the fields necessary to do the analysis, so I'm confused. (you do need to use the NET.HOST function to extract a HN-esque domain name from the url; more info in this notebook: https://www.kaggle.com/minimaxir/hacker-news-submission-score-predictor https://www.kaggle.com/minimaxir/hacker-news-submission-scor...)

harrisreynolds 8y ago
Thanks for the link! Will take a look. Would love to dump our Redshift instance!
- sbr464 8y ago
  I'm curious, what's the current size of your instance (disk space/# records) for this dataset? I'd be interested in a dump of the raw data, to save having to access via the API. I'd be willing to pay for the bandwidth/temp charges if any.
- scapecast 8y ago
  What is it that makes you want to dump Redshift? Sounds like there's a specific frustration?
  - harrisreynolds 8y ago
    I actually like Redshift fine. But we are paying about $250/month for it just to host this data and if it was already in a publicly available database it would be nice to drop that. But using Redshift for this has been great for us in terms of testing our tool.

bovermyer 8y ago

This is fascinating. I love data stories.

jamestimmins 8y ago

This looks great! The most interesting behavior reflected here IMO is the folks who submit lots and lots of links (presumably) in the hunt for points. The allure of Fake Internet Points is fascinating. Btw minor typo: "discussiom of Karma"

jenscow 8y ago

Interesting how the top 4 scoring posts are about deaths...

dev1n 8y ago

I'm wondering how the user KMinshew was the top job poster for the year when the most recent comment of theirs was from 2014.

lifeisstillgood 8y ago

What seriously there are 1/2 million users on here? I honestly thought it was in the 10's of thousands tops. I think it says something about how it is possible to converse with a few people in a crowded room but not notice the crowd.

shapiro92 8y ago
there was a recent study for reddit showing that actually most users dont do anything and the whole content is generated from not even the 10%
- Zitrax 8y ago
  Would be interesting to know how many of those 500 000 that were active during 2018 (at least one login).
  - harrisreynolds 8y ago
    I ran a query using Chartly and it came back with ~138K users that had at least one post (story or comment) during 2018. So maybe around 28% or so were active.

echelon 8y ago

> 53+ thousand new users This might explain why my perception of average comment quality has dropped. It's purely anecdotal, but it feels as if the volume of joke posts and "me too" responses has increased well beyond what it was five years ago. Comments have always been the most valuable part of HN for me. It's frankly starting to feel a little bit like Reddit around here, and I'm concerned about the signal:noise ratio dropping any further.

diamondo25 8y ago
Yes, this is an issue of anything popular. Not that it is a bad thing, as it keeps the community alive, but it feels "not the same" and might drive users away, like what happened on Facebook.
dvno42 8y ago
Perhaps a few tweaks can be made to the platform to help ensure HN's "core values" (that sounds silly) are passed on to new users. For the most part there's always been a good herd mentality to negate the one off low quality posts and a bit of public shaming "We don't talk like that around here...". Maybe a "Posting Guidelines" page and more weight added to the down-vote button to those that are privileged to have it could help.
cordonbleu 8y ago
there are sockpuppets rings abounding, likely there are black/grey hat SEO spammers trying to elevate thier live[valid,working] link count to game the g00gle search results, and then there are people that want to comment, civilly and substantiavely but have been somehow cut off from doing so, thus create a throwaway login for that purpose just to make thier voice heard. and then there are trolls and the indignant that just like to derail a normal conversation because its "funny". I have to deal with these on my chat board often but not at large scale. There are of course, normal users that are trying to have a constructive discourse, i believe there is ample signature to flag the differences to a discriminating mind, or someone with admin/mod status and access to the management tools.
wyxuan 8y ago
There are some positive aspects, as it allows for more diversified opinions on various issues. I find that it is great that there are chicken farmers, bridge builders, etc on this platform. But I agree. There have been less and less articles on directly tech relating things, and more on general world news. Some of the things are important and worth posting, but HN has become a little bit of a general news aggregator.

smsm42 8y ago

It's a bit depressing that out of top 5 posts, two are about deaths, one about prominent figure retiring and two are about companies being swallowed by other companies.

scarface74 8y ago
Of course death is sad, but we are all going to die one day. The people who died and were recognized made a meaningful impact on the world.
- smsm42 8y ago
  I don't argue with that, but the overall negative vibe of top 5 is obvious to me. I mean we could have at least a couple of "this new great thing happened" there? Of course, corporate mergers may be good news too for some people, so maybe that's not full negative.

csbartus 8y ago

Lately I’m reading HN as a replacement for tv and newspapers, for world news. I’m not saying HN is not fun anymore it’s just the value is elsewhere. A guess is that slowly I’ll replace HN entirely. It’s too distractive and no essence is captured. Lobste.rs in contrast is a place where I go excited.

purple_ducks 8y ago
Completely off topic but I clicked through your profile (just to see if you were lobste.rs employee - (cynical I know)) Anyway, on your website, you have a 4 picture gallery with the selectors vertically at the side but it was a bit jarring as I'm used to those vertical selectors usually used to display for vertical scrolling. Even ignoring the above, the photos transition horizontally which is even more jarring (when the vertical selectors exist). <unwarranted-2-cents/>

alwayseptember 8y ago

I, for one, welcome our new users. But seriously, a couple of points worth discussing: 1. Moderation feels much more automated, with default rules cutting broad robotic swaths into (or more often out of) discussion. 2. If HN is this popular, it's leaning closer and closer to social networking platform, with at least one key point: you cannot delete things, but they can. 3. Both of these facts feed into throwaway accounts. Thus, "users" are not all truly users. The numbers don't tell the whole story.