18 ms·

Is Hacker News a good predictor of future tech trends?

344 points by drpancake 5y ago

aaron695 5y ago
Is there a single comment here that understands the mathematics behind these graphs and as such can understand how to read them? This is not like comparing Google Trends. That's before we can get into positive vs negative posts and alleged confirmation/survivor bias and how they matter to what these graphs represent.
oakpond 5y ago
Regarding your conclusion: > My conclusion: Hacker News is typically ahead of the mainstream, often by a few years, but you would need to be paying very close attention to catch the early mentions of a new tech trend. Most of the linked posts and comments have very few upvotes and probably wouldn't even make it to the front page. HN also tracks front-page history: news.ycombinator.com/front. To go back to a particular date all you need to do is give the date, i.e. news.ycombinator.com/front?day=2020-08-24. Seems to go back to 2006-10-09.
chansiky 5y ago
Is no one going to mention Rust? If HN is good at predicting trends, then we're about to see a lot more Rust in a few years.
New_California 5y ago
Bitcoin was completely and totally missed by HN and more broadly by Silicon Valley. Only recently the SV/HN community is catching up and waking up to cryptocurrencies - and still hesitantly.
- johnnycerberus 5y ago
  It wasn't missed, I still remember checking the sourceforge source circa 2009. I was not really that interested in it and I am still not. https://news.ycombinator.com/item?id=599852 https://news.ycombinator.com/item?id=599852
  - fastball 5y ago
    So... you're admitting that you are a bad predictor of future trends?
    - johnnycerberus 5y ago
      We cannot predict future trends in technology, if we could, then Bitcoin would have today 0 value because it is boosted only by FOMO (fear of missing out) feelings, not because it is useful. In 2009, I was starting with Scala, which had functional features way beyond what was achievable in other languages at that time. Scala was a game changer, it rode later on the Big Data (Spark, Kafka) wave (circa 2015). Many academic things that Scala proved to be production-ready were backported after to Java, JS, etc. I have built systems with Scala that were actually useful for people and organisations, while Bitcoin as a technology... I am just glad that I have predicted worse in this case.
      - moreshadow 5y ago
        Bitcoin's value partially derives from permissionless transactions on an immutable ledger, if you'd actually been following it from 2009 you'd know that (but you don't). > Bitcoin would have today 0 value because it is boosted only by FOMO (fear of missing out) An incredibly tired take (but it's cute that you spelled out "FOMO" and abbreviated it). Compared to 2009, 2021 is the future, where the value of BTC is not 0. We don't have to speculate incorrectly as you do when you say "Bitcoin would have today 0 value," we know what value it is. You completely failed to predict the effect of BTC and cryptocurrency on the global financial market, which supports your idea that "we cannot predict future trends." We can make educated guesses though.
        johnnycerberus 5y ago
        > Bitcoin's value derives from permissionless transactions on an immutable ledger The imbecilic way of saying supply and demand?
        5y ago
- drpancake 5y ago
  I suspect the folks over at Coinbase would beg to differ.
  - New_California 5y ago
    Coinbase is successful b/c it went against the HN crowd wisdom. On the very HN thread Brian Armstrong announced his Coinbase idea the participants widely expressed skepticism.
    - throwawayFries 5y ago
      But he probably first heard about Bitcoin from something he read on HN.
FridayoLeary 5y ago
I like to see analysis like this that sheds light on the collective 'mind' of HN. Are there more of these out there?
leugim 5y ago
I'd say no. Lots of (successful) ideas have been dismissed by this community. And that is okey.
- davemp 5y ago
  Can't forget the famous dropbox dismissal: https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224
  - dcolkitt 5y ago
    I think that example really exemplifies HN's biggest blindspot. A lot of successful companies are built around taking an existing technology, then packaging it in a way that drastically reduces the frictions for the user. The typical HN user doesn't understand those pain point, because they're technically sophisticated enough that they're mostly invisible. So, you take something like Dropbox. Yeah, it pretty much is just mounted FTP. And the average HN is looking at that and thinking "I could do the same thing with a bash script, what's the big deal".
    - luffapi 5y ago
      It’s not a blindspot for many people. It’s open source and open protocols vs closed source spyware. SCP is far better than Dropbox if you value privacy for instance. A lot of resistance to things like Dropbox come from the experience of being screwed over by closed source software. If you look at the state of Dropbox today, those initial comments look pretty spot on.
    - sumtechguy 5y ago
      It is a blind spot of knowledge. Sure you can make a script that does it. But most people can not do that. Heck just yesterday something in my hacked up system did something odd. It spit out an error that I will now have to research (probably find nothing) and then have to dig out the code on. Then spend several days probably reverse engineering some tech API that I have never seen before, in a style that will be different than what I am used to. But if I had bought a product that did the same thing. More than likely 50 other people would have the same issue and a few dozen work arounds. If those failed I could have a shot at just taking it to whoever sold it to me and saying 'fix it'. But as I hacked together something on my own. I own it, warts and all. I enjoy doing that but most people have zero idea where to even begin in fixing something like that. Buying something that 'just works' solves so many issues (and creates others). Probably the biggest one that sticks out in my mind was when the iPad came out and all the tech boards like this were down on it. But it was what most people wanted to use a computer for. Most people did not want a computer. They wanted a media device. For many years the only way to get that was to get a computer. Now you can get that media device without most of the headaches of a computer.
kkdaemas 5y ago
In case of NFTs these were called "Colored Coins" much earlier.
theonlybutlet 5y ago
Would be interesting to see the prefix "meta"
slingnow 5y ago
Using total number of mentions as a way to predict future tech trends seems dubious at best. I understand coming up with a better metric would be difficult, but this one seems useless. For example, with this metric, how can we tell the difference between people constantly dismissing Bitcoin or people constantly praising it? Just because you mention it doesn't mean you're mentioning it in a positive light.
- hnbad 5y ago
  HN commenters famously dismissed Dropbox when it was first posted. I would love to see a variant of this based on sentiment analysis and whether the conclusion would turn into "yes, but not the way you think".
  - xx511134bz 5y ago
    A lot of people fall into the trap of negativity = IQ. Your brain can spit out a reason that DropBox will succeed or fail, but the failure case sounds smarter.
    - luffapi 5y ago
      Alternative take, people didn’t like Dropbox because it was the repackaging and close sourcing of existing solutions. It was more polished and financially successful but that doesn’t mean it was good for tech. It’s corporate software, yes it became valuable but it definitely represents an anti-user trend that actually would best have been resisted. You should never give closed source software root!
      - ratww 5y ago
        > Alternative take, people didn’t like Dropbox because it was the repackaging and close sourcing of existing solutions It wasn't. Even the famous comment just pointed it was possible to replicate with existing tools. And apart from one person the thread was actually pretty welcoming. That comment was just a grandfather of the very popular "Why should I use this instead of [completely different solution]?", that is also so very often also used against open source projects themselves.
black_13 5y ago
A lot of self involved bay area types
hnbad 5y ago
"My conclusion: Hacker News is typically ahead of the mainstream, often by a few years, but you would need to be paying very close attention to catch the early mentions of a new tech trend. Most of the linked posts and comments have very few upvotes and probably wouldn't even make it to the front page." That's a long way of saying "no".
- AnimalMuppet 5y ago
  Yeah. "HN mentioned the future, but no one noticed even on HN" != "HN predicts the future".
creaghpatr 5y ago
Love the visualizations, I would also want to compare against false positives though. From my time here, I feel like there's been a lot of flops that never broke into the public conscious.
MattGaiser 5y ago
1. For things like remote work, it only became a trend because of the pandemic. You would have needed to predict the pandemic to predict that management would permit remote work. It was not really a trend, but rather an emergency measure that morphed into a desirable benefit. 2. This only looks at successful things. We must examine unsuccessful things to see if there is any real predictive ability.
- em500 5y ago
  Regarding point 2, an old joke comes to mind: "economists have successfully predicted nine of the last five recessions".
  - MattGaiser 5y ago
    I am constantly seeing articles about how a market crash is coming and have been seeing them since 2014.
    - dragonwriter 5y ago
      > I am constantly seeing articles about how a market crash is coming and have been seeing them since 2014. I’ve been seeing them nonstop since the early 80s, and I suspect I would have seen them earlier except that my media consumption before elementary school was fairly constrained.
    - qnsi 5y ago
      I think part of this is media enjoying amplifying clickable stories. A lot of people (myself included) just can't help but click about news like "market crash is comming" "housing prices are about to crash" etc. Current media business models sadly are driven by sensational news. See also: all those revolutionary stories like cancer cure, that happen to only work in mice for now and will probably never work in humans, but we still click on those
      - cyberge99 5y ago
        I agree. I think The Great Recession of 2007 created an ongoing paranoid that marketing folks capitalize on.
shotta 5y ago
This only looked at submissions it seems. I gather a lot more useful information lurking the comments. I think discussions in the comments here are far ahead of the mainstream.
- jedimastert 5y ago
  > I gather a lot more useful information lurking the comments I don't know if it's a good thing or not, but I've definitely gotten into the habit of opening the comments of a post before the link itself.
  - lotsofpulp 5y ago
    Considering the prevalence of clickbait, it is an effective way to prevent wasting time if someone has already identified it.
    - jedimastert 5y ago
      It definitely catches a lot of clickbait, but there's a lot of false positives if the post goes against the hivemind
  - mellavora 5y ago
    As an old slashdot user, I've almost never done anything else :)
  - jasondigitized 5y ago
    +1 on this. All the good stuff is in the comments.
  - gnulinux 5y ago
    It's a bad habit since it encourages commenting before reading the submission. But I do the same thing as well, and it's not clear to me how someone could do otherwise without spending extraordinary time on HN. I find HN a very valuable source, so scanning some amount of comments is more important to me than reading all the submissions.
dougSF70 5y ago
Basically, no one knows what is going to catch in until it has. As one poster mentioned, false positives could be plentiful. Also, for companies or brands such as Tesla, their PR machine will be working harder than HN to spread the word, so not a good data point.
deleted 5y ago
[deleted]
yodsanklai 5y ago
HN is certainly a source to know what is trendy. I'd be interested to see a list of trends based on HN top posts and comment.
tommek4077 5y ago
Yes, if JavaScript is a future trend ;)
bpodgursky 5y ago
LOL Hacker News is where you to to learn that self-hosted rack-mount servers in your own basement running a LAMP stack provisioned with self-generated bash scripts is the future.
afavour 5y ago
Feels like a great case study in confirmation bias: you only looked at the topics that ended up being successful. How many tech trends did Hacker News hype up only to have them go nowhere? Without that info we don’t really know what HN’s success rate is.
- mempko 5y ago
  This! Since people have limited time, if HN leads more more often then not on some tech goose chase, then it doesn't matter if some tech trend is mentioned early on HN and has interest. But ultimately what makes something successful is investment in that thing in both time and money. HN is very VC heavy tech crowd and of course it should lead in trends where VC tech crowd creates out of pure effort and money. Telsa and 3D printing is an interesting example. If it isn't strictly a digital good, HN might not be ahead given the audience.
- bena 5y ago
  Not only confirmation bias, but also they're essentially polling a very specific subset of the population. This is a segment of the population that would be replete with model train enthusiasts, RC plane hobbyists, and HAM radio operators back in the day. So, yeah, things of interest in the tech space would be of even more interest to people on HN. But yeah, they also picked the "winners", they really have to filter for all topics discussed by HN and then find out how many of those became "future tech" and how many fell by the wayside. Because if it's 1 in 100, it's not a really good predictor. And if something becomes mainstream, then of course it'll be talked about. That's kind of how it works.
- hirako2000 5y ago
  Exactly. Or at last a comparative analysis with similar news feeds. Would reddit tech posts not yield similar results? If not then perhaps one has an edge in term of early discovery future trends. The variety of tech subjects submitted by hacker newsers is very large, and mostly about the most innovative and financially rewarding domain: tech. So sure, hacker news does predict every upcoming trend since someone will be posting something about each and every plausibly promising thing.
- 5y ago
mFixman 5y ago
> For each topic I counted how many times it had been mentioned in a post title, then calculated a 0-100 index, where 100 is the month with the most mentions. This is not really useful data then. A time series with a big spike will "lose" against an identical one without the spike. Maybe the amount of days when a certain word was in the front page would be a better comparison metric on the HN side?
timbaboon 5y ago
Hacker News causes success of future tech trends :P
jedimastert 5y ago
Egdelord opinion time: I think HN has way too much of a hivemind to really think outside the box enough to catch trends.
- luffapi 5y ago
  I think it’s more that many of the “leaders” of this community are investors and entrepreneurs who have vested interests and use this site and community for promotion or strategic purposes. This site’s contents and commentary are mostly ads, thus not good at prediction.
- goodpoint 5y ago
  Anybody talking about the hivemind gets downvoted by the hivemind.
  - dredmorbius 5y ago
    The hivemind practices humility.
username91 5y ago
I'm not aware of HN doing any predicting at large. If you mean 'will the average person later act like a given segment of HN people do now?' - then it depends on how good you are at selecting your segment, which makes it no better or worse than using any other sample of people for prediction. Less abstractly, productisation is what takes things from obscurity (e.g. HN) to mainstream audiences, so future trends might grow out of stuff that you see here, but the people who can figure out whether certain tech can become a product (would-be predictors) -hopefully- also have the means to turn it into a product themselves.
rafale 5y ago
They missed crypto and NFTs. For some reason HN doesn't appreciate the potential of these tech. Even when some prominent Silicon Valley investors do. I wouldn't have expected this from a technical and freedom minded audience.
- Ensorceled 5y ago
  Both crypto and NFTs are strugging to find productive real world applications; predicting those are like predicting which toy will be the "one to get" this Christmas or the Beanie Baby fad.
  - whitepaint 5y ago
    > are struggling to find productive real world applications You can lend and borrow on compound.finance, aave. You can play no-loss lotteries like PoolTogether. You can bet on your beliefs and hedge your investments on markets platforms like Polymarket. You can gamble. More on: https://ethereum.org/en/dapps/ https://ethereum.org/en/dapps/ Remember, anyone can use all of this without any arbitrary restrictions imposed by governments or companies.
    - Ensorceled 5y ago
      Financial shenanigans and gambling aren't really productive.
      - whitepaint 5y ago
        > Financial shenanigans I am curious, what's your job? Are you involved in finance in any way?
        Ensorceled 5y ago
        No. I'm also curious, do people in Finance think various forms of gambling (and hedging of speculations) are productive for society.
heywherelogingo 5y ago
Skimming the front page there's nothing on future tech.
ArtWomb 5y ago
If something looks useful, I'll sign up day one. Even if it doesn't pan out. That's how you get early adopter status. Found out about: Figma, Airtable, Stripe, Coinbase, Robinhood, Alpaca. All through HN posts ;)
alexnapiernomad 5y ago
I AGRII JAMES. BTC AND NFT ARE THE FUTURE! BUT Y NO DOGE?!?!
littleweep 5y ago
If so, we might have (actually) seen the year of the Linux desktop.
deleted 5y ago
[deleted]
throwthere 5y ago
This is a good start. It does not do much to answer the question of "predicting" trends. You would need to quantify false positives somehow. That is, tech that is big on HN but not elsewhere. I'm thinking of all the JS frameworks of the day, for one. For instance, Meteor JS was the bees knees for awhile.
- OJFord 5y ago
  You'd have to also somehow decide that they're 'past it' too, and it's not merely that HN is still ahead on them! Easy with a JS framework that's been discontinued say, hard in general, especially if you want some kind of objective rule for it.
deleted 5y ago
[deleted]
sbussard 5y ago
Remote work is so obvious to the hacker news crowd that there’s no need for it to trend here. What’s artificial is going to an office to use a computer there. There better be food to make up for the traffic.
Ensorceled 5y ago
I think a lot of people on HN are already working remote (I've been 100% remote since before the pandemic). It's a business process and business management issue that doesn't really come up on Hacker News except in productivity discussions.
asciimov 5y ago
No. This is just confirmation bias. The author didn't control for what was on the front page, how many points the article received, or the amount of discussion on the topic. Not to mention the myriad of technologies that are on Hacker News that fizzle.
nix23 5y ago
No, pretty much the opposite.
wiradikusuma 5y ago
Hacker News, like any news aggregator, pops up new stuff all the time. The hardest part is keeping an open mind for jumping on early. Hard because 99% turn out to be dud anyway, not to mention our crowd are skeptical bunch. My biggest "regret" is ignoring Bitcoin/Ether. Saw Bitcoin posts when it was $5.. nah it's a scam. Oh it's gone up to $10.. nah it won't go up further. Went up further.. nah won't buy bcoz already too expensive. Ether announced.. lolz nobody going to buy another coin since there's already Bitcoin.. Ether goes up.. WTF.
- nemo44x 5y ago
  I guess there's 2 ways to approach something that appears to be a scam. One is to avoid it. The other is to get in on it.
- bcrosby95 5y ago
  Yes, crystal balls are in short supply. That said, unless you were a True Believer, if you bought bitcoin at $5 you probably wouldn't have any left by now. You likely would have sold at $100. Or $1,000. I know a lot of people that got in early. But only one made millions because he thought it would be the next big thing. Everyone else jumped ship at some point because they weren't True Believers.
- langitbiru 5y ago
  So what are you going to do with NFTs? :)
- handrous 5y ago
  I picked up 0.1-and-some-change Bitcoin, largely from "faucets" and a tiny amount of collaborative mining. Basically free, a few hours of screwing-around being my only real "investment". I neglected the wallet, since it wasn't worth anything anyway, and lost the file in a format-and-reinstall at some point. Did something similar with Doge, later. Whoops.
xyzzy21 5y ago
No really. Unless you think the ONLY future tech is certain niches in software.
deleted 5y ago
[deleted]
enriquto 5y ago
if it was, everybody would be coding in rust using neovim, which does not really seem to happen
chubot 5y ago
Hm never saw this thread about "fixing RAZR" as a pre-approved YC idea (pre-Android and pre-iPhone), but it's pretty interesting: https://news.ycombinator.com/item?id=13465 https://news.ycombinator.com/item?id=13465
prewett 5y ago
HN is pretty good at identifying trends. I generally read some comment here way before I hear about it somewhere else. Does HN accurately predict where the trend goes? Not necessarily. But if something actually does trend, I will have heard low-level chatter about it on HN for a long time. Remote work trended because of the pandemic, for example, but there was constant chatter about remote work anytime the topic of employment came up, and the monthly job match-making posts even post whether the job / worker is open to remote. Functional programming has had low-level chatter from the beginning (given that PG viewed it as one of the reasons for the success of his company), and indeed, functional programming has grown more mainstream. I started exploring Python for my Perl-type tasks because it kept getting good mentions on HN. I don't think any of the above ever got to the point where it got so many mentions it would hit a strong 100 in his methodology compared to Google trends. I'm also not sure you could identify successful trends early on, but I think you hear chatter on HN long before other places.
- stackbutterflow 5y ago
  For news related to programming trends, the Venn diagram between here and r/programming is pretty much a circle.
mettamage 5y ago
From the top of my head (a very biased account): * Dolphin emulator progress reports (I love it when they show up) * Random posts on x86-64 emulation or emulating a gameboy or something like that. Fabrice Bellard is a hero. * People who post their very first side project ever made either via no-code solutions or some React/Bootstrap type of thing. In many cases, these things are job boards for a specific niche. * HN praising Apple for standing up to the FBI (or CIA?). HN hating Apple because of recent events with scanning pictures on iCloud and all that. * All the things the blog post talks about * Hardcore learning resources on many topics, one that I actually used: http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/ (tons of books on Calculus as well, I remember a 1900 textbook called Calculus Made Easy) * The random physics or biological discovery * And so much more The way I see it: HN talks about tech (and interesting stuff outside tech), it's bound to cover tech that eventually becomes popular. But will the Dolphin progress reports become as popular as Bitcoin? I don't think so. It'd be interesting times if they would ;-)
- microtherion 5y ago
  Another Apple perennial is blog posts about switching away from Apple to Linux/Windows/ChromeOS. This successfully predicted that 300% of Apple's user base is gone now.
turdnagel 5y ago
What is the prediction being made? Whether or not something is a "trend"? What is the definition of a trend? > Can I convince myself that checking the HN front page multiple times a day is a useful and productive exercise? That seems to be a different topic entirely and should probably be the headline of the article. Also, it's clearly a bit cheeky, but confirmation bias abounds nonetheless.
f154hfds 5y ago
Two things I learned on HN before they became mainstream: * Rust * Covid For the latter I remember thinking that it was going to become a big deal soon but no one seemed to be paying attention except for those paranoid HNers.
ChicagoBoy11 5y ago
Of things which are "interesting," absolutely. But to the stickiness of these things I'd say the record is mixed. If you lurk here chances are you'll hear about any single tech topic earlier than most other places. But that the ones you hear about here (or are most upvoted here) also tend to be the ones with greater stickiness is not clear to me at all.
deleted 5y ago
[deleted]
yuchi 5y ago
Only tangential but still interesting. During the infamous leap second fiasco of Java in 2012 which brought down all JVM servers by saturating the CPU, I saved the day because I was reading about it here on HN while my colleagues were struggling to undestrand why all our environments stopped working. It took me a few minutes to realize that we were indeed hit by what I just read, look at ten of my colleagues each closely inspecting logs at different ssh sessions, and gleefully give them a precise description of the problem, an approach to verify it and a solution, adding “it’s all over the news guys, you should keep yourself updated”.
keewee7 5y ago
The mood on Hacker News (and reddit/twitter) has become too pessimistic to be a good predictor of the future. The long-term trend is that the world is becoming a better place. Pessimistic people are biased against that fundamental trend that other trends will develop on. Groups of pessimistic people will not be good predictors of the future.
- Toine 5y ago
  Define "better place" ?
  - marcusverus 5y ago
    Fewer people in extreme poverty. Hunger is falling. Child labor is on the decline. People in developed countries are working less. The share of income spent on food is down (in the US). Life expectancy is rising. Child mortality is down. Death in childhood is down. Guinea worm has almost been eradicated. Teen births are down (in the US). Smoking is down (in the US). Violent crime is down. Nuclear weapon stockpiles are down. More people are living in democracies. Literacy is up. Internet access is increasing. Solar energy is getting cheaper. Source: https://www.vox.com/2014/11/24/7272929/global-poverty-health-crime-literacy-good-news https://www.vox.com/2014/11/24/7272929/global-poverty-health...
    - consumer451 5y ago
      It's unfortunate that spreading such news is not as profitable as fearmongering. It is only recently that I have begun to suspect that there are strong political reasons for suppressing this as well. Feeling pretty naive about that.
- long_time_gone 5y ago
  Oddly, this strikes me as one of the most pessimistic posts I’ve seen on HN. - What is your definition of “long-term”? - Can a certain industry or geography get worse while the broader world becomes a “better place”? - Aren’t optimists just lying to themselves in the opposite way as pessimists? - Is any group of a single type a good predictor of anything? - Is everyone on HN exactly the same with the exact same outlook?
OJFord 5y ago
> Remote Work [...] I thought [in contradistinction to the graph] that Hacker News would have been more ahead of the trend on this one. > For each topic I counted how many times it had been mentioned in a post title I suspect that's the problem. A lot of trendy discussion happens in comments; I think that's going to be particularly true for a topic like remote work. People are going to extol its virtues when it's tangentially related to a submission, but especially pre-pandemic there's only so many stories you could submit about it? 'Company goes full-remote'? 'How we handle remote working at Company'? It'd be interesting to search comments too I think. hn.algolia.com supports it.
- sam0x17 5y ago
  Agreed. This analysis would be more useful if it was done against comments.
- drpancake 5y ago
  Author here. I agree with you but the only recent HN data dump that I could find contained posts and not comments. I suspect the trend lines would look similar though.
- dredmorbius 5y ago
  Both the selection (survivorship bias) and weighting (mentions rather than votes/discussion) are problematic. Discussion itself can be a very poor metric, as when there's a great deal of general agreement there's frequently little (or at least less) as when compared with highly contentious topics. Research methodology can often be restricted by data availability. Finding novel ways to torture^W explore data can be insightful, though raises its own risks (p-hacking, as an example).
bane 5y ago
HN is fantastic for surfacing up and coming technologies. Reading HN daily is almost like a superpower. The HN community is...not so great...at predicting which things are exciting and the industry should adopt. If it were the industry would have settled on a LISP dialect long ago, the web would have adopted good readability standards (based on thousands of meta-complaints on virtually every top rated post on any topic), and mobile app development would still be a growing and profitable business (and Apple would finally be catering to niche HN reader needs).
MontyCarloHall 5y ago
Relevant XKCD: https://xkcd.com/1497/ https://xkcd.com/1497/
- dredmorbius 5y ago
  Also relevant: https://xkcd.com/1827/ https://xkcd.com/1827/
brundolf 5y ago
I would say it's more a good radar for what (software, mostly web) technologies are entering general awareness. If something might be successful one day, you'll probably find out about it. And then lots of those things will sputter out, but not all of them.
dqpb 5y ago
Now do the other study - where on the Internet were these things first talked about.
devops000 5y ago
You forgot the "coronavirus" or "pandemic" in the analysis
mateo1 5y ago
As with all online communities HN is suffering from it's increased popularity. If you remember how Reddit was a good 10 years back, on every thread there would be 1-2 actual experts posting substantiated and well-sourced commentary on news relevant to their field. Usually undergraduates and young professionals passionate about their field. At some point this was discovered by journalists and for a brief period of time major news organizations would rely on reddit users for sources. Many if not most discussions would be done in good faith or at least with solid arguments. The userbase was usually 18+. As it got more popular it got worse, mainly because of the "eternal september", altering userbase due to smartphone proliferation and changes in ranking algorithms and business model. HN is not the same as reddit and isn't doomed to the same faults, but as as a forum gets more popular, experts reduce in percentage, they're more reluctant and careful on what and how they comment, the "populism" of the voting system becomes harmful and all the attention creates financial incentives for bad actors. As for using it as some kind of blackbox betting algorithm for startups and trends, that's just a bad idea. I'm certain that you will get the same graph structure for all things that never became popular. Obviously any forum focused on leading edge technologies will discuss advancements long before they become a subject of interest in the general populace. But they'll also discuss everything that will fail or never reach the mainstream sphere.
yablak 5y ago
https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias
- andruby 5y ago
  Upvoted because I also immediately thought of survivorship bias when reading the post. You might want to elaborate a little bit instead of posting just a link to wikipedia. I think they should have also looked at trends that peaked on HN but didn't (yet) peak on google.
vmception 5y ago
Just find whatever is being ridiculed on HN and that’s the alpha bet
- drexlspivey 5y ago
  What is currently being ridiculed on HN (other than crypto) ? Asking for a friend
  - vmception 5y ago
    Nothing pops out aside from crypto, to me, like you said. But for anyone passing by, crypto assets are like 20 distinct sectors and ever expanding, with many solutions and competing services providers in each sector. HN typically gets stuck on surface level arguments such as limitations in the "currency" concept and the merchant use sector. It also gets reliable stuck on environmental impact, as well as surface level use of NFTs. Solutions providers in oracles, insurance, IOT, yield farming, yield farming aggregators, SaaS products (information, tax), marketplaces and many more entire sectors are almost completely ignored. All alpha bets.
  - johnnycerberus 5y ago
    Node.js (by Deno fans and JavaScript haters) Deno (by overly invested Node.js users) Clojure (ClojureScript) (by type-driven programming aficionados) Scala (by keep-it-simple squad) C++ (by literally everyone else that does not develop in C++) Rust ("why don't you rewrite that in Rust?"-jokes squad) Docker (by the "I still use a simple VM to boot SkyNet"-crowd) TypeScript (by "don't pollute my JSDoc annotated codebase with types"-crowd) Kubernetes (by the "I still haven't installed Docker"-crowd) JVM, CLR, WASM (by the "metal"-people) Bitcoin (by reasonable people)
    - handrous 5y ago
      > C++ (by literally everyone else that does not develop in C++) Also, those who do.
jpswade 5y ago
No, it predicted Dropbox would be dead.
doo_daa 5y ago
(How) did you disambiguate Tesla as in Tesla Cars from Tesla as in Nikola Tesla from the either source?
m1117 5y ago
Hacker News's a great source of rage.
chaostheory 5y ago
I would say no. I made the first post on Bitcoin here. https://news.ycombinator.com/item?id=599852 https://news.ycombinator.com/item?id=599852
- drpancake 5y ago
  I'm afraid someone pipped you by a few months: https://news.ycombinator.com/item?id=463793 https://news.ycombinator.com/item?id=463793
modeless 5y ago
HN is the best way of learning about future tech trends before they happen. But it's not because there's any kind of prescient consensus here. The important stuff is a small minority. You have to do your own filtering. I was ahead of the curve on Bitcoin, VR, and deep learning because I read about them early here, but they weren't the most popular things at the time. HN hates Bitcoin to this day. But I learned about it from HN and decided that HN was wrong about it. When lots of people are wrong about something, I see it as an opportunity.
- gnulinux 5y ago
  It's not clear to me if this is predictive. E.g. HN has been right about AMD. Years ago when people were talking about AMD it was worth peanuts and look at it now. I don't think it's a case of a group of people being right/wrong, it has more to do with since you're fed all the perspectives, you can find the right/wrong ones yourself and take action accordingly.
  - modeless 5y ago
    Yes, I'm not saying HN is wrong about everything. HN isn't opposed to my other examples of VR or deep learning (though there are certainly people who argue strenuously that deep learning is a dead end and GOFAI is the way forward). I'm just saying that when the consensus about something is wrong, that's an opportunity to get ahead of the curve. And reading HN is a great way to gather the information you need to decide whether the consensus is right or not. But you have to figure that out for yourself.
    - bdcravens 5y ago
      How do you measure what is wrong? In the example of Bitcoin, if the measurement is price, I think that's a misunderstanding of what people are criticizing.
      - modeless 5y ago
        The criticisms have changed over time. But I'm pretty sure the early criticisms were mostly wrong, and I'm certain that the people who criticized it early missed big opportunities, even if Bitcoin itself ultimately fails (which is of course still a possibility).
privatdozent 5y ago
Cool analysis!
bdcravens 5y ago
It's barely an indicator of what's currently popular, skewing heavily to subsections of the tech world
esjeon 5y ago
As backed by this post, HN isn’t precisely good at prediction. It simply has far less noise than the general internet, making it easier to notice the upcoming trends.
tomcooks 5y ago
Cue in joke about that Dropbox thread comment
justinzollars 5y ago
It looks like interest in Bitcoin has peaked on HN and in decline; We could extrapolate the same will happen on google in two years.
hackitup7 5y ago
I find that when Hacker News really likes a piece of highly technical SaaS technology it's usually pretty good. The best example I can think of is Datadog which got rave reviews here years ago and has continued to be enormously successful. Particularly for enterprise tech as there tends to be less herd behavior up-market. If I'm diligencing a technology I'll often actually look at HN to see the opinions.
p0nce 5y ago
Hacker News skews towards richer, more technical, and higher-educated and then makes systematic bias against pedestrian, working-class, and uncomplicated solutions. If it's too popular on HN I'd say it's a red flag.
artembugara 5y ago
https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias
drdrey 5y ago
The serverless post on the Scoble blog is not even about what we think of as Serverless now, just good old cloud. As in "you don't have any physical servers?"
lend000 5y ago
I've read about an indie trading algorithm that works like this. It's basically a PageRank for professional forums, which apparently did quite well for the author. A more fundamental-based form of sentiment analysis.
wetpaws 5y ago
Dropbox folks would smile at this post
jsnell 5y ago
One thing worth nothing is that HN didn't launch publicly until 2007 February. There is some older content in the DB, but it looks mostly like very low volume manual testing. So the data set definitely shouldn't be extended that far back, and is affecting a few of the be graphs.
salynchnew 5y ago
This is a great example of survivorship bias! A+ work. :) https://en.wikipedia.org/wiki/Survivorship_bias#As_a_general_experimental_flaw https://en.wikipedia.org/wiki/Survivorship_bias#As_a_general...
- throwawayFries 5y ago
  This is a great example of an HN comment (and also in general why reading HN comments as opposed to the posts, is a poor predictor of anything).
peter303 5y ago
A class project would be to use machine learning on these graphs to spot trends. HN has lots of data examples.
lamontcg 5y ago
this suffers from survivorship bias though and isn't looking at stuff that hacker news yaps about but never happens.
Razengan 5y ago
On the contrary. The very nature of vote-based forums like Reddit and HN means that you must agree with the masses and uphold the status quo to even be seen at all let alone heard, so it’s very rare that you get a glimpse of upcoming disruption or anything new at all. TL;DR: No, most people here are sticks in the mud. If you listened to them you’d think that future laptops would still have parallel ports and floppy drives and communicate via IRC.
- Razengan 5y ago
  And phones would still have scrollbars.
resheku 5y ago
no, it isn't
reducesuffering 5y ago
If you payed attention and believed in HN, you would've invested in: AMD, Square, Cloudflare, Fiverr, Apple, Nvidia, TSM, Twilio, CrowdStrike, Zoom, Atlassian, Moderna, and Shopify. Maybe a 5 year 500% return compared to the general market's 100%? I'd say your fat stacks of $ would have you believing it's a pretty good predictor of something.
- drpancake 5y ago
  Author of the article here. I would love to run an unbiased analysis on that, but it would tricky to list and identify all of the companies that flopped. I guess you could grab a list of every successful and defunct company listed on Crunchbase/AngelList/etc, but there would still be bias, as flopped companies may never have got to the point of creating a profile there, or they scrubbed the profile later.
  - reducesuffering 5y ago
    I'd stick to public companies and run sentiment analysis on all tickers and company mentions. Then identify the correlation between positive sentiment and historical market return. I'm sure there are at least a few funds incorporating this data into some portion of a portfolio.
- fomine3 5y ago
  Was Atlassian cool at HN ever?
  - reducesuffering 5y ago
    Doesn't need to be cool. The prevailing attitude has always been basically that JIRA sucks, but 3/4 of devs are using it and the alternatives are worse.
jbverschoor 5y ago
For cloud storage and collaboration (Dropbox) -> Nope Blockchain -> Nope
deleted 5y ago
[deleted]
lvl100 5y ago
This place is pretty right leaning and mostly front-end.

344 points by drpancake 5y ago

Is there a single comment here that understands the mathematics behind these graphs and as such can understand how to read them? This is not like comparing Google Trends. That's before we can get into positive vs negative posts and alleged confirmation/survivor bias and how they matter to what these graphs represent.

oakpond 5y ago

Regarding your conclusion: > My conclusion: Hacker News is typically ahead of the mainstream, often by a few years, but you would need to be paying very close attention to catch the early mentions of a new tech trend. Most of the linked posts and comments have very few upvotes and probably wouldn't even make it to the front page. HN also tracks front-page history: news.ycombinator.com/front. To go back to a particular date all you need to do is give the date, i.e. news.ycombinator.com/front?day=2020-08-24. Seems to go back to 2006-10-09.

chansiky 5y ago

Is no one going to mention Rust? If HN is good at predicting trends, then we're about to see a lot more Rust in a few years.

New_California 5y ago

Bitcoin was completely and totally missed by HN and more broadly by Silicon Valley. Only recently the SV/HN community is catching up and waking up to cryptocurrencies - and still hesitantly.

johnnycerberus 5y ago
It wasn't missed, I still remember checking the sourceforge source circa 2009. I was not really that interested in it and I am still not. https://news.ycombinator.com/item?id=599852 https://news.ycombinator.com/item?id=599852
- fastball 5y ago
  So... you're admitting that you are a bad predictor of future trends?
  - johnnycerberus 5y ago
    We cannot predict future trends in technology, if we could, then Bitcoin would have today 0 value because it is boosted only by FOMO (fear of missing out) feelings, not because it is useful. In 2009, I was starting with Scala, which had functional features way beyond what was achievable in other languages at that time. Scala was a game changer, it rode later on the Big Data (Spark, Kafka) wave (circa 2015). Many academic things that Scala proved to be production-ready were backported after to Java, JS, etc. I have built systems with Scala that were actually useful for people and organisations, while Bitcoin as a technology... I am just glad that I have predicted worse in this case.
    - moreshadow 5y ago
      Bitcoin's value partially derives from permissionless transactions on an immutable ledger, if you'd actually been following it from 2009 you'd know that (but you don't). > Bitcoin would have today 0 value because it is boosted only by FOMO (fear of missing out) An incredibly tired take (but it's cute that you spelled out "FOMO" and abbreviated it). Compared to 2009, 2021 is the future, where the value of BTC is not 0. We don't have to speculate incorrectly as you do when you say "Bitcoin would have today 0 value," we know what value it is. You completely failed to predict the effect of BTC and cryptocurrency on the global financial market, which supports your idea that "we cannot predict future trends." We can make educated guesses though.
      - johnnycerberus 5y ago
        > Bitcoin's value derives from permissionless transactions on an immutable ledger The imbecilic way of saying supply and demand?
        5y ago
drpancake 5y ago
I suspect the folks over at Coinbase would beg to differ.
- New_California 5y ago
  Coinbase is successful b/c it went against the HN crowd wisdom. On the very HN thread Brian Armstrong announced his Coinbase idea the participants widely expressed skepticism.
  - throwawayFries 5y ago
    But he probably first heard about Bitcoin from something he read on HN.

FridayoLeary 5y ago

I like to see analysis like this that sheds light on the collective 'mind' of HN. Are there more of these out there?

leugim 5y ago

I'd say no. Lots of (successful) ideas have been dismissed by this community. And that is okey.

davemp 5y ago
Can't forget the famous dropbox dismissal: https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224
- dcolkitt 5y ago
  I think that example really exemplifies HN's biggest blindspot. A lot of successful companies are built around taking an existing technology, then packaging it in a way that drastically reduces the frictions for the user. The typical HN user doesn't understand those pain point, because they're technically sophisticated enough that they're mostly invisible. So, you take something like Dropbox. Yeah, it pretty much is just mounted FTP. And the average HN is looking at that and thinking "I could do the same thing with a bash script, what's the big deal".
  - luffapi 5y ago
    It’s not a blindspot for many people. It’s open source and open protocols vs closed source spyware. SCP is far better than Dropbox if you value privacy for instance. A lot of resistance to things like Dropbox come from the experience of being screwed over by closed source software. If you look at the state of Dropbox today, those initial comments look pretty spot on.
  - sumtechguy 5y ago
    It is a blind spot of knowledge. Sure you can make a script that does it. But most people can not do that. Heck just yesterday something in my hacked up system did something odd. It spit out an error that I will now have to research (probably find nothing) and then have to dig out the code on. Then spend several days probably reverse engineering some tech API that I have never seen before, in a style that will be different than what I am used to. But if I had bought a product that did the same thing. More than likely 50 other people would have the same issue and a few dozen work arounds. If those failed I could have a shot at just taking it to whoever sold it to me and saying 'fix it'. But as I hacked together something on my own. I own it, warts and all. I enjoy doing that but most people have zero idea where to even begin in fixing something like that. Buying something that 'just works' solves so many issues (and creates others). Probably the biggest one that sticks out in my mind was when the iPad came out and all the tech boards like this were down on it. But it was what most people wanted to use a computer for. Most people did not want a computer. They wanted a media device. For many years the only way to get that was to get a computer. Now you can get that media device without most of the headaches of a computer.

kkdaemas 5y ago

In case of NFTs these were called "Colored Coins" much earlier.

theonlybutlet 5y ago

Would be interesting to see the prefix "meta"

slingnow 5y ago

Using total number of mentions as a way to predict future tech trends seems dubious at best. I understand coming up with a better metric would be difficult, but this one seems useless. For example, with this metric, how can we tell the difference between people constantly dismissing Bitcoin or people constantly praising it? Just because you mention it doesn't mean you're mentioning it in a positive light.

hnbad 5y ago
HN commenters famously dismissed Dropbox when it was first posted. I would love to see a variant of this based on sentiment analysis and whether the conclusion would turn into "yes, but not the way you think".
- xx511134bz 5y ago
  A lot of people fall into the trap of negativity = IQ. Your brain can spit out a reason that DropBox will succeed or fail, but the failure case sounds smarter.
  - luffapi 5y ago
    Alternative take, people didn’t like Dropbox because it was the repackaging and close sourcing of existing solutions. It was more polished and financially successful but that doesn’t mean it was good for tech. It’s corporate software, yes it became valuable but it definitely represents an anti-user trend that actually would best have been resisted. You should never give closed source software root!
    - ratww 5y ago
      > Alternative take, people didn’t like Dropbox because it was the repackaging and close sourcing of existing solutions It wasn't. Even the famous comment just pointed it was possible to replicate with existing tools. And apart from one person the thread was actually pretty welcoming. That comment was just a grandfather of the very popular "Why should I use this instead of [completely different solution]?", that is also so very often also used against open source projects themselves.

black_13 5y ago

A lot of self involved bay area types

hnbad 5y ago

"My conclusion: Hacker News is typically ahead of the mainstream, often by a few years, but you would need to be paying very close attention to catch the early mentions of a new tech trend. Most of the linked posts and comments have very few upvotes and probably wouldn't even make it to the front page." That's a long way of saying "no".

AnimalMuppet 5y ago
Yeah. "HN mentioned the future, but no one noticed even on HN" != "HN predicts the future".

creaghpatr 5y ago

Love the visualizations, I would also want to compare against false positives though. From my time here, I feel like there's been a lot of flops that never broke into the public conscious.

MattGaiser 5y ago

1. For things like remote work, it only became a trend because of the pandemic. You would have needed to predict the pandemic to predict that management would permit remote work. It was not really a trend, but rather an emergency measure that morphed into a desirable benefit. 2. This only looks at successful things. We must examine unsuccessful things to see if there is any real predictive ability.

em500 5y ago
Regarding point 2, an old joke comes to mind: "economists have successfully predicted nine of the last five recessions".
- MattGaiser 5y ago
  I am constantly seeing articles about how a market crash is coming and have been seeing them since 2014.
  - dragonwriter 5y ago
    > I am constantly seeing articles about how a market crash is coming and have been seeing them since 2014. I’ve been seeing them nonstop since the early 80s, and I suspect I would have seen them earlier except that my media consumption before elementary school was fairly constrained.
  - qnsi 5y ago
    I think part of this is media enjoying amplifying clickable stories. A lot of people (myself included) just can't help but click about news like "market crash is comming" "housing prices are about to crash" etc. Current media business models sadly are driven by sensational news. See also: all those revolutionary stories like cancer cure, that happen to only work in mice for now and will probably never work in humans, but we still click on those
    - cyberge99 5y ago
      I agree. I think The Great Recession of 2007 created an ongoing paranoid that marketing folks capitalize on.

shotta 5y ago

This only looked at submissions it seems. I gather a lot more useful information lurking the comments. I think discussions in the comments here are far ahead of the mainstream.

jedimastert 5y ago
> I gather a lot more useful information lurking the comments I don't know if it's a good thing or not, but I've definitely gotten into the habit of opening the comments of a post before the link itself.
- lotsofpulp 5y ago
  Considering the prevalence of clickbait, it is an effective way to prevent wasting time if someone has already identified it.
  - jedimastert 5y ago
    It definitely catches a lot of clickbait, but there's a lot of false positives if the post goes against the hivemind
- mellavora 5y ago
  As an old slashdot user, I've almost never done anything else :)
- jasondigitized 5y ago
  +1 on this. All the good stuff is in the comments.
- gnulinux 5y ago
  It's a bad habit since it encourages commenting before reading the submission. But I do the same thing as well, and it's not clear to me how someone could do otherwise without spending extraordinary time on HN. I find HN a very valuable source, so scanning some amount of comments is more important to me than reading all the submissions.

dougSF70 5y ago

Basically, no one knows what is going to catch in until it has. As one poster mentioned, false positives could be plentiful. Also, for companies or brands such as Tesla, their PR machine will be working harder than HN to spread the word, so not a good data point.

deleted 5y ago

[deleted]

yodsanklai 5y ago

HN is certainly a source to know what is trendy. I'd be interested to see a list of trends based on HN top posts and comment.

tommek4077 5y ago

Yes, if JavaScript is a future trend ;)

bpodgursky 5y ago

LOL Hacker News is where you to to learn that self-hosted rack-mount servers in your own basement running a LAMP stack provisioned with self-generated bash scripts is the future.

afavour 5y ago

Feels like a great case study in confirmation bias: you only looked at the topics that ended up being successful. How many tech trends did Hacker News hype up only to have them go nowhere? Without that info we don’t really know what HN’s success rate is.

mempko 5y ago
This! Since people have limited time, if HN leads more more often then not on some tech goose chase, then it doesn't matter if some tech trend is mentioned early on HN and has interest. But ultimately what makes something successful is investment in that thing in both time and money. HN is very VC heavy tech crowd and of course it should lead in trends where VC tech crowd creates out of pure effort and money. Telsa and 3D printing is an interesting example. If it isn't strictly a digital good, HN might not be ahead given the audience.
bena 5y ago
Not only confirmation bias, but also they're essentially polling a very specific subset of the population. This is a segment of the population that would be replete with model train enthusiasts, RC plane hobbyists, and HAM radio operators back in the day. So, yeah, things of interest in the tech space would be of even more interest to people on HN. But yeah, they also picked the "winners", they really have to filter for all topics discussed by HN and then find out how many of those became "future tech" and how many fell by the wayside. Because if it's 1 in 100, it's not a really good predictor. And if something becomes mainstream, then of course it'll be talked about. That's kind of how it works.
hirako2000 5y ago
Exactly. Or at last a comparative analysis with similar news feeds. Would reddit tech posts not yield similar results? If not then perhaps one has an edge in term of early discovery future trends. The variety of tech subjects submitted by hacker newsers is very large, and mostly about the most innovative and financially rewarding domain: tech. So sure, hacker news does predict every upcoming trend since someone will be posting something about each and every plausibly promising thing.
5y ago

mFixman 5y ago

> For each topic I counted how many times it had been mentioned in a post title, then calculated a 0-100 index, where 100 is the month with the most mentions. This is not really useful data then. A time series with a big spike will "lose" against an identical one without the spike. Maybe the amount of days when a certain word was in the front page would be a better comparison metric on the HN side?

timbaboon 5y ago

Hacker News causes success of future tech trends :P

jedimastert 5y ago

Egdelord opinion time: I think HN has way too much of a hivemind to really think outside the box enough to catch trends.

luffapi 5y ago
I think it’s more that many of the “leaders” of this community are investors and entrepreneurs who have vested interests and use this site and community for promotion or strategic purposes. This site’s contents and commentary are mostly ads, thus not good at prediction.
goodpoint 5y ago
Anybody talking about the hivemind gets downvoted by the hivemind.
- dredmorbius 5y ago
  The hivemind practices humility.

username91 5y ago

I'm not aware of HN doing any predicting at large. If you mean 'will the average person later act like a given segment of HN people do now?' - then it depends on how good you are at selecting your segment, which makes it no better or worse than using any other sample of people for prediction. Less abstractly, productisation is what takes things from obscurity (e.g. HN) to mainstream audiences, so future trends might grow out of stuff that you see here, but the people who can figure out whether certain tech can become a product (would-be predictors) -hopefully- also have the means to turn it into a product themselves.

rafale 5y ago

They missed crypto and NFTs. For some reason HN doesn't appreciate the potential of these tech. Even when some prominent Silicon Valley investors do. I wouldn't have expected this from a technical and freedom minded audience.

Ensorceled 5y ago
Both crypto and NFTs are strugging to find productive real world applications; predicting those are like predicting which toy will be the "one to get" this Christmas or the Beanie Baby fad.
- whitepaint 5y ago
  > are struggling to find productive real world applications You can lend and borrow on compound.finance, aave. You can play no-loss lotteries like PoolTogether. You can bet on your beliefs and hedge your investments on markets platforms like Polymarket. You can gamble. More on: https://ethereum.org/en/dapps/ https://ethereum.org/en/dapps/ Remember, anyone can use all of this without any arbitrary restrictions imposed by governments or companies.
  - Ensorceled 5y ago
    Financial shenanigans and gambling aren't really productive.
    - whitepaint 5y ago
      > Financial shenanigans I am curious, what's your job? Are you involved in finance in any way?
      - Ensorceled 5y ago
        No. I'm also curious, do people in Finance think various forms of gambling (and hedging of speculations) are productive for society.

heywherelogingo 5y ago

Skimming the front page there's nothing on future tech.

ArtWomb 5y ago

If something looks useful, I'll sign up day one. Even if it doesn't pan out. That's how you get early adopter status. Found out about: Figma, Airtable, Stripe, Coinbase, Robinhood, Alpaca. All through HN posts ;)

alexnapiernomad 5y ago

I AGRII JAMES. BTC AND NFT ARE THE FUTURE! BUT Y NO DOGE?!?!

littleweep 5y ago

If so, we might have (actually) seen the year of the Linux desktop.

deleted 5y ago

[deleted]

throwthere 5y ago

This is a good start. It does not do much to answer the question of "predicting" trends. You would need to quantify false positives somehow. That is, tech that is big on HN but not elsewhere. I'm thinking of all the JS frameworks of the day, for one. For instance, Meteor JS was the bees knees for awhile.

OJFord 5y ago
You'd have to also somehow decide that they're 'past it' too, and it's not merely that HN is still ahead on them! Easy with a JS framework that's been discontinued say, hard in general, especially if you want some kind of objective rule for it.

deleted 5y ago

[deleted]

sbussard 5y ago

Remote work is so obvious to the hacker news crowd that there’s no need for it to trend here. What’s artificial is going to an office to use a computer there. There better be food to make up for the traffic.

Ensorceled 5y ago

I think a lot of people on HN are already working remote (I've been 100% remote since before the pandemic). It's a business process and business management issue that doesn't really come up on Hacker News except in productivity discussions.

asciimov 5y ago

No. This is just confirmation bias. The author didn't control for what was on the front page, how many points the article received, or the amount of discussion on the topic. Not to mention the myriad of technologies that are on Hacker News that fizzle.

nix23 5y ago

No, pretty much the opposite.

wiradikusuma 5y ago

Hacker News, like any news aggregator, pops up new stuff all the time. The hardest part is keeping an open mind for jumping on early. Hard because 99% turn out to be dud anyway, not to mention our crowd are skeptical bunch. My biggest "regret" is ignoring Bitcoin/Ether. Saw Bitcoin posts when it was $5.. nah it's a scam. Oh it's gone up to $10.. nah it won't go up further. Went up further.. nah won't buy bcoz already too expensive. Ether announced.. lolz nobody going to buy another coin since there's already Bitcoin.. Ether goes up.. WTF.

nemo44x 5y ago
I guess there's 2 ways to approach something that appears to be a scam. One is to avoid it. The other is to get in on it.
bcrosby95 5y ago
Yes, crystal balls are in short supply. That said, unless you were a True Believer, if you bought bitcoin at $5 you probably wouldn't have any left by now. You likely would have sold at $100. Or $1,000. I know a lot of people that got in early. But only one made millions because he thought it would be the next big thing. Everyone else jumped ship at some point because they weren't True Believers.
langitbiru 5y ago
So what are you going to do with NFTs? :)
handrous 5y ago
I picked up 0.1-and-some-change Bitcoin, largely from "faucets" and a tiny amount of collaborative mining. Basically free, a few hours of screwing-around being my only real "investment". I neglected the wallet, since it wasn't worth anything anyway, and lost the file in a format-and-reinstall at some point. Did something similar with Doge, later. Whoops.

xyzzy21 5y ago

No really. Unless you think the ONLY future tech is certain niches in software.

deleted 5y ago

[deleted]

enriquto 5y ago

if it was, everybody would be coding in rust using neovim, which does not really seem to happen

chubot 5y ago

Hm never saw this thread about "fixing RAZR" as a pre-approved YC idea (pre-Android and pre-iPhone), but it's pretty interesting: https://news.ycombinator.com/item?id=13465 https://news.ycombinator.com/item?id=13465

prewett 5y ago

HN is pretty good at identifying trends. I generally read some comment here way before I hear about it somewhere else. Does HN accurately predict where the trend goes? Not necessarily. But if something actually does trend, I will have heard low-level chatter about it on HN for a long time. Remote work trended because of the pandemic, for example, but there was constant chatter about remote work anytime the topic of employment came up, and the monthly job match-making posts even post whether the job / worker is open to remote. Functional programming has had low-level chatter from the beginning (given that PG viewed it as one of the reasons for the success of his company), and indeed, functional programming has grown more mainstream. I started exploring Python for my Perl-type tasks because it kept getting good mentions on HN. I don't think any of the above ever got to the point where it got so many mentions it would hit a strong 100 in his methodology compared to Google trends. I'm also not sure you could identify successful trends early on, but I think you hear chatter on HN long before other places.

stackbutterflow 5y ago
For news related to programming trends, the Venn diagram between here and r/programming is pretty much a circle.

mettamage 5y ago

From the top of my head (a very biased account): * Dolphin emulator progress reports (I love it when they show up) * Random posts on x86-64 emulation or emulating a gameboy or something like that. Fabrice Bellard is a hero. * People who post their very first side project ever made either via no-code solutions or some React/Bootstrap type of thing. In many cases, these things are job boards for a specific niche. * HN praising Apple for standing up to the FBI (or CIA?). HN hating Apple because of recent events with scanning pictures on iCloud and all that. * All the things the blog post talks about * Hardcore learning resources on many topics, one that I actually used: http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/ (tons of books on Calculus as well, I remember a 1900 textbook called Calculus Made Easy) * The random physics or biological discovery * And so much more The way I see it: HN talks about tech (and interesting stuff outside tech), it's bound to cover tech that eventually becomes popular. But will the Dolphin progress reports become as popular as Bitcoin? I don't think so. It'd be interesting times if they would ;-)

microtherion 5y ago
Another Apple perennial is blog posts about switching away from Apple to Linux/Windows/ChromeOS. This successfully predicted that 300% of Apple's user base is gone now.

turdnagel 5y ago

What is the prediction being made? Whether or not something is a "trend"? What is the definition of a trend? > Can I convince myself that checking the HN front page multiple times a day is a useful and productive exercise? That seems to be a different topic entirely and should probably be the headline of the article. Also, it's clearly a bit cheeky, but confirmation bias abounds nonetheless.

f154hfds 5y ago

Two things I learned on HN before they became mainstream: * Rust * Covid For the latter I remember thinking that it was going to become a big deal soon but no one seemed to be paying attention except for those paranoid HNers.

ChicagoBoy11 5y ago

Of things which are "interesting," absolutely. But to the stickiness of these things I'd say the record is mixed. If you lurk here chances are you'll hear about any single tech topic earlier than most other places. But that the ones you hear about here (or are most upvoted here) also tend to be the ones with greater stickiness is not clear to me at all.

deleted 5y ago

[deleted]

yuchi 5y ago

Only tangential but still interesting. During the infamous leap second fiasco of Java in 2012 which brought down all JVM servers by saturating the CPU, I saved the day because I was reading about it here on HN while my colleagues were struggling to undestrand why all our environments stopped working. It took me a few minutes to realize that we were indeed hit by what I just read, look at ten of my colleagues each closely inspecting logs at different ssh sessions, and gleefully give them a precise description of the problem, an approach to verify it and a solution, adding “it’s all over the news guys, you should keep yourself updated”.

keewee7 5y ago

The mood on Hacker News (and reddit/twitter) has become too pessimistic to be a good predictor of the future. The long-term trend is that the world is becoming a better place. Pessimistic people are biased against that fundamental trend that other trends will develop on. Groups of pessimistic people will not be good predictors of the future.

Toine 5y ago
Define "better place" ?
- marcusverus 5y ago
  Fewer people in extreme poverty. Hunger is falling. Child labor is on the decline. People in developed countries are working less. The share of income spent on food is down (in the US). Life expectancy is rising. Child mortality is down. Death in childhood is down. Guinea worm has almost been eradicated. Teen births are down (in the US). Smoking is down (in the US). Violent crime is down. Nuclear weapon stockpiles are down. More people are living in democracies. Literacy is up. Internet access is increasing. Solar energy is getting cheaper. Source: https://www.vox.com/2014/11/24/7272929/global-poverty-health-crime-literacy-good-news https://www.vox.com/2014/11/24/7272929/global-poverty-health...
  - consumer451 5y ago
    It's unfortunate that spreading such news is not as profitable as fearmongering. It is only recently that I have begun to suspect that there are strong political reasons for suppressing this as well. Feeling pretty naive about that.
long_time_gone 5y ago
Oddly, this strikes me as one of the most pessimistic posts I’ve seen on HN. - What is your definition of “long-term”? - Can a certain industry or geography get worse while the broader world becomes a “better place”? - Aren’t optimists just lying to themselves in the opposite way as pessimists? - Is any group of a single type a good predictor of anything? - Is everyone on HN exactly the same with the exact same outlook?

OJFord 5y ago

> Remote Work [...] I thought [in contradistinction to the graph] that Hacker News would have been more ahead of the trend on this one. > For each topic I counted how many times it had been mentioned in a post title I suspect that's the problem. A lot of trendy discussion happens in comments; I think that's going to be particularly true for a topic like remote work. People are going to extol its virtues when it's tangentially related to a submission, but especially pre-pandemic there's only so many stories you could submit about it? 'Company goes full-remote'? 'How we handle remote working at Company'? It'd be interesting to search comments too I think. hn.algolia.com supports it.

sam0x17 5y ago
Agreed. This analysis would be more useful if it was done against comments.
drpancake 5y ago
Author here. I agree with you but the only recent HN data dump that I could find contained posts and not comments. I suspect the trend lines would look similar though.
dredmorbius 5y ago
Both the selection (survivorship bias) and weighting (mentions rather than votes/discussion) are problematic. Discussion itself can be a very poor metric, as when there's a great deal of general agreement there's frequently little (or at least less) as when compared with highly contentious topics. Research methodology can often be restricted by data availability. Finding novel ways to torture^W explore data can be insightful, though raises its own risks (p-hacking, as an example).

bane 5y ago

HN is fantastic for surfacing up and coming technologies. Reading HN daily is almost like a superpower. The HN community is...not so great...at predicting which things are exciting and the industry should adopt. If it were the industry would have settled on a LISP dialect long ago, the web would have adopted good readability standards (based on thousands of meta-complaints on virtually every top rated post on any topic), and mobile app development would still be a growing and profitable business (and Apple would finally be catering to niche HN reader needs).

MontyCarloHall 5y ago

Relevant XKCD: https://xkcd.com/1497/ https://xkcd.com/1497/

dredmorbius 5y ago
Also relevant: https://xkcd.com/1827/ https://xkcd.com/1827/

brundolf 5y ago

I would say it's more a good radar for what (software, mostly web) technologies are entering general awareness. If something might be successful one day, you'll probably find out about it. And then lots of those things will sputter out, but not all of them.

dqpb 5y ago

Now do the other study - where on the Internet were these things first talked about.

devops000 5y ago

You forgot the "coronavirus" or "pandemic" in the analysis

mateo1 5y ago

As with all online communities HN is suffering from it's increased popularity. If you remember how Reddit was a good 10 years back, on every thread there would be 1-2 actual experts posting substantiated and well-sourced commentary on news relevant to their field. Usually undergraduates and young professionals passionate about their field. At some point this was discovered by journalists and for a brief period of time major news organizations would rely on reddit users for sources. Many if not most discussions would be done in good faith or at least with solid arguments. The userbase was usually 18+. As it got more popular it got worse, mainly because of the "eternal september", altering userbase due to smartphone proliferation and changes in ranking algorithms and business model. HN is not the same as reddit and isn't doomed to the same faults, but as as a forum gets more popular, experts reduce in percentage, they're more reluctant and careful on what and how they comment, the "populism" of the voting system becomes harmful and all the attention creates financial incentives for bad actors. As for using it as some kind of blackbox betting algorithm for startups and trends, that's just a bad idea. I'm certain that you will get the same graph structure for all things that never became popular. Obviously any forum focused on leading edge technologies will discuss advancements long before they become a subject of interest in the general populace. But they'll also discuss everything that will fail or never reach the mainstream sphere.

yablak 5y ago

https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias

andruby 5y ago
Upvoted because I also immediately thought of survivorship bias when reading the post. You might want to elaborate a little bit instead of posting just a link to wikipedia. I think they should have also looked at trends that peaked on HN but didn't (yet) peak on google.

vmception 5y ago

Just find whatever is being ridiculed on HN and that’s the alpha bet

drexlspivey 5y ago
What is currently being ridiculed on HN (other than crypto) ? Asking for a friend
- vmception 5y ago
  Nothing pops out aside from crypto, to me, like you said. But for anyone passing by, crypto assets are like 20 distinct sectors and ever expanding, with many solutions and competing services providers in each sector. HN typically gets stuck on surface level arguments such as limitations in the "currency" concept and the merchant use sector. It also gets reliable stuck on environmental impact, as well as surface level use of NFTs. Solutions providers in oracles, insurance, IOT, yield farming, yield farming aggregators, SaaS products (information, tax), marketplaces and many more entire sectors are almost completely ignored. All alpha bets.
- johnnycerberus 5y ago
  Node.js (by Deno fans and JavaScript haters) Deno (by overly invested Node.js users) Clojure (ClojureScript) (by type-driven programming aficionados) Scala (by keep-it-simple squad) C++ (by literally everyone else that does not develop in C++) Rust ("why don't you rewrite that in Rust?"-jokes squad) Docker (by the "I still use a simple VM to boot SkyNet"-crowd) TypeScript (by "don't pollute my JSDoc annotated codebase with types"-crowd) Kubernetes (by the "I still haven't installed Docker"-crowd) JVM, CLR, WASM (by the "metal"-people) Bitcoin (by reasonable people)
  - handrous 5y ago
    > C++ (by literally everyone else that does not develop in C++) Also, those who do.

jpswade 5y ago

No, it predicted Dropbox would be dead.

doo_daa 5y ago

(How) did you disambiguate Tesla as in Tesla Cars from Tesla as in Nikola Tesla from the either source?

m1117 5y ago

Hacker News's a great source of rage.

chaostheory 5y ago

I would say no. I made the first post on Bitcoin here. https://news.ycombinator.com/item?id=599852 https://news.ycombinator.com/item?id=599852

drpancake 5y ago
I'm afraid someone pipped you by a few months: https://news.ycombinator.com/item?id=463793 https://news.ycombinator.com/item?id=463793

modeless 5y ago

HN is the best way of learning about future tech trends before they happen. But it's not because there's any kind of prescient consensus here. The important stuff is a small minority. You have to do your own filtering. I was ahead of the curve on Bitcoin, VR, and deep learning because I read about them early here, but they weren't the most popular things at the time. HN hates Bitcoin to this day. But I learned about it from HN and decided that HN was wrong about it. When lots of people are wrong about something, I see it as an opportunity.

gnulinux 5y ago
It's not clear to me if this is predictive. E.g. HN has been right about AMD. Years ago when people were talking about AMD it was worth peanuts and look at it now. I don't think it's a case of a group of people being right/wrong, it has more to do with since you're fed all the perspectives, you can find the right/wrong ones yourself and take action accordingly.
- modeless 5y ago
  Yes, I'm not saying HN is wrong about everything. HN isn't opposed to my other examples of VR or deep learning (though there are certainly people who argue strenuously that deep learning is a dead end and GOFAI is the way forward). I'm just saying that when the consensus about something is wrong, that's an opportunity to get ahead of the curve. And reading HN is a great way to gather the information you need to decide whether the consensus is right or not. But you have to figure that out for yourself.
  - bdcravens 5y ago
    How do you measure what is wrong? In the example of Bitcoin, if the measurement is price, I think that's a misunderstanding of what people are criticizing.
    - modeless 5y ago
      The criticisms have changed over time. But I'm pretty sure the early criticisms were mostly wrong, and I'm certain that the people who criticized it early missed big opportunities, even if Bitcoin itself ultimately fails (which is of course still a possibility).

privatdozent 5y ago

Cool analysis!

bdcravens 5y ago

It's barely an indicator of what's currently popular, skewing heavily to subsections of the tech world

esjeon 5y ago

As backed by this post, HN isn’t precisely good at prediction. It simply has far less noise than the general internet, making it easier to notice the upcoming trends.

tomcooks 5y ago

Cue in joke about that Dropbox thread comment

justinzollars 5y ago

It looks like interest in Bitcoin has peaked on HN and in decline; We could extrapolate the same will happen on google in two years.

hackitup7 5y ago

I find that when Hacker News really likes a piece of highly technical SaaS technology it's usually pretty good. The best example I can think of is Datadog which got rave reviews here years ago and has continued to be enormously successful. Particularly for enterprise tech as there tends to be less herd behavior up-market. If I'm diligencing a technology I'll often actually look at HN to see the opinions.

p0nce 5y ago

Hacker News skews towards richer, more technical, and higher-educated and then makes systematic bias against pedestrian, working-class, and uncomplicated solutions. If it's too popular on HN I'd say it's a red flag.

artembugara 5y ago

https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias

drdrey 5y ago

The serverless post on the Scoble blog is not even about what we think of as Serverless now, just good old cloud. As in "you don't have any physical servers?"

lend000 5y ago

I've read about an indie trading algorithm that works like this. It's basically a PageRank for professional forums, which apparently did quite well for the author. A more fundamental-based form of sentiment analysis.

wetpaws 5y ago

Dropbox folks would smile at this post

jsnell 5y ago

One thing worth nothing is that HN didn't launch publicly until 2007 February. There is some older content in the DB, but it looks mostly like very low volume manual testing. So the data set definitely shouldn't be extended that far back, and is affecting a few of the be graphs.

salynchnew 5y ago

This is a great example of survivorship bias! A+ work. :) https://en.wikipedia.org/wiki/Survivorship_bias#As_a_general_experimental_flaw https://en.wikipedia.org/wiki/Survivorship_bias#As_a_general...

throwawayFries 5y ago
This is a great example of an HN comment (and also in general why reading HN comments as opposed to the posts, is a poor predictor of anything).

peter303 5y ago

A class project would be to use machine learning on these graphs to spot trends. HN has lots of data examples.

lamontcg 5y ago

this suffers from survivorship bias though and isn't looking at stuff that hacker news yaps about but never happens.

Razengan 5y ago

On the contrary. The very nature of vote-based forums like Reddit and HN means that you must agree with the masses and uphold the status quo to even be seen at all let alone heard, so it’s very rare that you get a glimpse of upcoming disruption or anything new at all. TL;DR: No, most people here are sticks in the mud. If you listened to them you’d think that future laptops would still have parallel ports and floppy drives and communicate via IRC.

Razengan 5y ago
And phones would still have scrollbars.

resheku 5y ago

no, it isn't

reducesuffering 5y ago

If you payed attention and believed in HN, you would've invested in: AMD, Square, Cloudflare, Fiverr, Apple, Nvidia, TSM, Twilio, CrowdStrike, Zoom, Atlassian, Moderna, and Shopify. Maybe a 5 year 500% return compared to the general market's 100%? I'd say your fat stacks of $ would have you believing it's a pretty good predictor of something.

drpancake 5y ago
Author of the article here. I would love to run an unbiased analysis on that, but it would tricky to list and identify all of the companies that flopped. I guess you could grab a list of every successful and defunct company listed on Crunchbase/AngelList/etc, but there would still be bias, as flopped companies may never have got to the point of creating a profile there, or they scrubbed the profile later.
- reducesuffering 5y ago
  I'd stick to public companies and run sentiment analysis on all tickers and company mentions. Then identify the correlation between positive sentiment and historical market return. I'm sure there are at least a few funds incorporating this data into some portion of a portfolio.
fomine3 5y ago
Was Atlassian cool at HN ever?
- reducesuffering 5y ago
  Doesn't need to be cool. The prevailing attitude has always been basically that JIRA sucks, but 3/4 of devs are using it and the alternatives are worse.

jbverschoor 5y ago

For cloud storage and collaboration (Dropbox) -> Nope Blockchain -> Nope

deleted 5y ago

[deleted]

lvl100 5y ago

This place is pretty right leaning and mostly front-end.