5 ms·
Facebook is kind of an interesting example, as they got pretty far into "hyperscale" with mostly PHP, memcached, and Mysql.
by tyingq 6y ago
Facebook is kind of an interesting example, as they got pretty far into "hyperscale" with mostly PHP, memcached, and Mysql.
- ahofmann 6y agoThis could be the final post to this discussion. You can't really get more to the point than that. We are running a large German browser game on individual VPS instances. For many years, with PHP, MySQL and Redis. It works and it runs just fine.
- bombcar 6y agoIt's not one that just recently had to start migrating from Flash, is it?
- jasode 6y ago>Facebook is kind of an interesting example, as they got pretty far into "hyperscale" with mostly PHP, memcached, and Mysql. But PHP and MySql are only the surface level tools we see. There is also the unseen and that's where the immense complexity lies hidden under the surface. E.g. To help with scaling (which Friendster failed at), Facebook had separate MySql instances for each school. An interview with Facebook co-founder Dustin Moskovitz mentioned they had constant stress of emergencies and fires to put out. They had tons of custom tooling, Linux scripts, batch jobs, automation/orchestration/dashboard, etc and it's very likely the combination of all that ends up inventing an "invisible internal Kubernetes" without calling it "Kubernetes" to manage their server fleet. In other words, if Docker/Kubernetes/AWS were around, the early Facebook team might have offloaded some of the complexity onto a 3rd-party to reduce the all-nighters. But the tools didn't exist yet so they had to do it the hard way and invent "deployment orchestration" themselves. [EDIT to reply] Thanks for the "Tupperware" comment. A Google search leads to a Facebook post saying they renamed it to "Twine": https://engineering.fb.com/2019/06/06/data-center-engineering/twine/ https://engineering.fb.com/2019/06/06/data-center-engineerin...
- jeffffff 6y agoit's called 'tupperware'
- tyingq 6y agoThat's recent though. I'm saying they scaled up quite a long way before doing anything like that. From the outside it looks like they didn't really branch out into things like this until 2008 or so.
- jasode 6y ago>From the outside it looks like they didn't really branch out into things like this until 2008 or so. No, even before the Newsfeed feature rollout of September 2006, they already had crazy complexity orchestrating multiple servers for many schools. They were only a "simple PHP+MySql" website when it was February 2004 with only Harvard students on it. Dustin said he was the one in charge of spinning up new servers for each new school and he said it was a nightmare of engineering effort.
- Person5478 6y agoIIRC, they were doing things like patching bittorrent to prefer closer IP's to pull from during their deploys. So it would roll through their network from 1 side to the other without just completely saturating the entire network. But the point still remains, they had challenges to solve, but they were also simple, which made it possible TO solve those challenges.
- mattmanser 6y agoBut today, 16 years of Moore's law later, you could now have gone way, way, way, way past that. They probably could have done most of America on a single server. For example, even in 2009 Stack Overflow was running off two dedicated servers: https://stackoverflow.blog/2009/01/12/new-stack-overflow-server-glamour-shots/ https://stackoverflow.blog/2009/01/12/new-stack-overflow-ser...
- 6y ago
- rualca 6y ago> Facebook is kind of an interesting example, as they got pretty far into "hyperscale" with mostly PHP, memcached, and Mysql. Isn't Facebook known for hiring top world experts such as the likes of Andrei Alexandrescu to hyperoptimize all the way down their whole stack? That hardly sounds like running vanilla PHP with your generic nem ache and mysql. And how do you exactly think they get so many 9s in their reliability? Is this plausible with the "a single linode box ought to do" nonsense?
- tyingq 6y agoHe got there in 2009. Facebook did run for a long time with a fairly "vanilla" setup, yes. A hacker got ahold of their "index.php" in 2007: https://gist.github.com/nikcub/3833406 https://gist.github.com/nikcub/3833406 It is decidedly not a tour-de-force of technical prowess, just typical old school "bag of functions in the same namespace" PHP.
- redis_mlc 6y ago> Facebook is kind of an interesting example, as they got pretty far into "hyperscale" with mostly PHP, memcached, and Mysql. Those are fantastic technologies and each is remarkable at what it does. Besides that, those are some of the easiest skill sets to hire for.