4 ms·
What a fantastic writeup. I had some vague ideas regarding the challenges involved to build an application of such scale, but the article really makes it clear
by seankimdesign 10y ago
What a fantastic writeup. I had some vague ideas regarding the challenges involved to build an application of such scale, but the article really makes it clear for everyone the amount of decision points encountered as well as why certain solutions were selected.
I also like the way the article is broken down into the backend, API, frontend and mobile. This isolated approach really highlights the different struggles each aspects of the product has, while dealing with what is essentially a shared concern: performance.
What I also found interesting is the fact that they were able to come up with a pretty accurate guess in terms of the expected traffic.
> "We experienced a maximum tile placement rate of almost 200/s. This was below our calculated maximum rate of 333/s (average of 100,000 users placing a tile every 5 minutes)."
Their guess ended up being a good amount above the actual maximum usage, but it was probably padded against the worst case scenario. The company that I work for consistently fails to come up with accurate guesses even with our very rigid user base, so it's pretty impressive that Reddit could accommodate the unpredictable user base that is the entire Reddit community.
- eblanshey 10y agoAgreed, great write-up. Anyone have other recommended links to write-ups about specific challenges and how to make them scale?
- rajathagasthya 10y agoI really like the "Big Data in Real-Time at Twitter" slides at https://www.slideshare.net/nkallen/q-con-3770885 https://www.slideshare.net/nkallen/q-con-3770885. They explain their original implementation and why it didn't scale, possible new implementations and their current one (it's a bit old though). It's clear and easy to understand.
- josephg 10y agohttp://highscalability.com/ http://highscalability.com/ frequently features content like this. The archives are a treasure trove of practical software architecture wisdom.
- petercooper 10y agoHere are some recent ones that have been popular: http://highscalability.com/blog/2016/4/20/how-twitter-handles-3000-images-per-second.html http://highscalability.com/blog/2016/4/20/how-twitter-handle... https://redditblog.com/2017/1/17/caching-at-reddit/ https://redditblog.com/2017/1/17/caching-at-reddit/ https://blog.twitter.com/2017/the-infrastructure-behind-twitter-scale https://blog.twitter.com/2017/the-infrastructure-behind-twit... https://www.netlify.com/blog/2017/03/16/smashing-magazine-just-got-10x-faster/ https://www.netlify.com/blog/2017/03/16/smashing-magazine-ju... https://nickcraver.com/blog/2016/02/03/stack-overflow-a-technical-deconstruction/ https://nickcraver.com/blog/2016/02/03/stack-overflow-a-tech... https://engineering.linkedin.com/blog/2016/10/instant-messaging-at-linkedin--scaling-to-hundreds-of-thousands- https://engineering.linkedin.com/blog/2016/10/instant-messag... http://techblog.netflix.com/2016/08/building-fastcom.html http://techblog.netflix.com/2016/08/building-fastcom.html http://highscalability.com/blog/2016/9/28/how-uber-manages-a-million-writes-per-second-using-mesos-and.html http://highscalability.com/blog/2016/9/28/how-uber-manages-a... Or a recent writeup on a challenge building IMDB's forums in 2001(!) https://www.beatworm.co.uk/blog/internet/imdb-boards-no-more https://www.beatworm.co.uk/blog/internet/imdb-boards-no-more .. fun and scary in equal measures. (I'm plucking these from things were popular recently in our http://webopsweekly.com/ http://webopsweekly.com/ :-))
- samfisher83 10y agoThat writeup may have taken more time than building it out. It seems quite thorough. Edit: why am I getting down voted for praising a write-up?
- tabeth 10y agoBecause despite being thorough, there's no way this write-up could be completed slower than building the actual thing they're describing.
- specialist 9y agoBelated response: I've done a fair amount of tech writing. Documentation always takes me as long or longer than designing, coding. Most recently, when I released a novel layout manager, the docs, examples, screenshots, etc took roughly twice as much effort as everything else combined.
- virtuabhi 10y agoPeople on HN can be quite uptight. One response takes your comment literally and other downvoters do not approve of your cheeky comment.