4 ms·
If you have a static website, maybe. If you have a web application with millions of users, several large and complex data stores, and lots of functionality, thi
by skywhopper 11y ago
If you have a static website, maybe. If you have a web application with millions of users, several large and complex data stores, and lots of functionality, things get complicated fast. Most critically, it's not a direct relationship. If you double the number of pieces in your infrastructure, your problems become four times as hard if not more.
You might have a database and a web app. But the database gets slow, so you add a caching layer. Then you need to scale up to accomodate more users. You've got to be able to back up all your users' data. And when your app gets complicated, deployments can break things, so you also need to be able to roll back quickly. Once you have a dozen or more servers going, you realize that if one goes down everything breaks, so you have to add redundancy to each part of the system, which adds even more concerns about how backups and recovery is handled. Then you need to add some search features to your growing dataset, which necessitates another kind of database to index your primary database, and more app servers. Oh, and by this point when things break, it becomes really hard to tell why things are broken, so you need monitoring and centralized logging, and whoops, those are two more kinds of databases and the associated web applications to manage them. And when you're at this scale, you have to coordinate multiple pieces of the system for any change you make, so you have additional software to manage that. And by now, you'll realize your original code was terrible, and while you are wise enough to know not to try to rewrite everything, you instead take various pieces of your infrastructure and break them into microservices in new languages, and...
Are you getting the picture?
- ryandrake 11y agoGot it, but this guy was trying to deploy a 15-minute sample app.
- skywhopper 11y ago... using a tool designed for the type of app I described. That said, if you want to learn how to use the big complicated tool, it's best to start with a trivial app.
- josegonzalez 11y agoHe didn't need to use docker. FTP deploys for static sites are akin to deploying straight to heroku for rails apps.
- Gigablah 11y agoAnd what was preventing him from just uploading it to his AWS instance? If you want to experiment with an entirely new stack of technologies intended for deploying distributed, multi-tier, scalable apps, that's cool, but don't feign surprise if it takes you more than 30 minutes.
- deleted 11y ago[deleted]
- TeMPOraL 11y agoWas there even a project in which a single person was responsible for all of this? It feels that if you're having scaling issues so bad you need to do all of the things you described, you're already big enough to have 20 people on payroll handling this.
- baudehlo 11y agoWhatsapp probably did.
- TeMPOraL 11y agoI'm reading WhatsApp history on Wikipedia[0]; it seems they got themselves a lot of money before having the need to scale beyond what simple solutions would allow. [0] - https://en.wikipedia.org/wiki/WhatsApp https://en.wikipedia.org/wiki/WhatsApp
- lmm 11y agoI did all that for some projects two jobs ago. The company had ~50 people elsewhere but only three of us in Europe (and for every responsibility we wanted at least two of us to know about it). None of it is hard, honestly - it's all well-documented things that a smart person can teach themselves how to do. And I honestly think that having one person take end-to-end responsibility for delivering one piece of functionality results in much higher quality than having twenty people specializing in a single layer as it applies to twenty different deliverables.
- defenestration 11y agoGreat summary and very recognizable points. It would be useful if there was a guide on how to scale your app wisely. > So what are the basic steps you will go through? > What are caveats and best practices per step? > What are the important choices and when do you make them? Could you recommend a resource for answers on these questions? I took your comment and edited it a bit: Start: database + web app Slow database: adding caching layer Need to backup: starting back up all your users' data App gets complicated, deployments break: being able to roll back quickly You have a dozen or more servers. If one goes down everything breaks: adding redundancy to each part of the system and improve backups and recovery. You need search features: add another kind of database to index your primary database and add more app servers. Now when things break, it becomes really hard to tell why things are broken: add monitoring and centralized logging. Those are two more kinds of databases and web applications to manage them. Coordinate multiple pieces of the system for any change you make: add additional software to manage that. You'll realize your original code was terrible: you take various pieces of your infrastructure and break them into microservices in new languages. And...