3 ms·

I’m really curious about the percentage of total software projects that have real scalability concerns. The majority of my career has been working on products

I’m really curious about the percentage of total software projects that have real scalability concerns. The majority of my career has been working on products that barely get 100k monthly active users. These projects don’t really need to worry about scalability because it’ll never happen. It hasn’t happened in the last 30 years, unlikely to happen in the next two years. With that in mind why spend so much resources on complexity where the only benefits are to the engineers that get to add another buzz word to their resume? I’m guessing the total percentage is less than 5, probably 1% seeing how Wordpress is still the most used framework on the internet. I’ve worked at companies that cared about complexity and it was baked into the code. After the product was released we only got 500 users when the initial projection was 10,000 (this was a company selling Medicare advantage plans). The org was eventually disbanded. We probably spent $20 million in additional “engineering” effort that was never used. At some point you have to question why things are done a certain way if there aren’t material benefits.

sofixa 2y ago
100k monthly active users can result in wildly different numbers of queries/requests/operations depending on what you're doing. 100k MAU for a static blog and 100k MAU for an online game simulating a real world economy will have vastly different usage patterns of the underlying databases/message queues/etc. > With that in mind why spend so much resources on complexity where the only benefits are to the engineers that get to add another buzz word to their resume? It's not the only benefits, it's making things potentially easier down the line; more often than not it can be premature optimisation (preparing for a future that never comes), but there are also plenty of examples of companies outgrowing their initial infrastructure and having to spend lots of time on rearchitecting everything. A lot of that time can be saved by thinking a little bit beforehand and making things theoretically scalable.
- azemetre 2y ago
  I really disagree on making things "easier," that's kinda my point. Forcing in solutions that work at Google or Facebook or Twitter into an experience that gets 0.01% of the traffic is bad engineering IMO. This example of a 100k MAU was for a company that was 150+ years old. They know their customers, they know the business, they know the growth projections. The only people that didn't know this was the technology leadership team that was given a blank check to "modernize" while all it did was create a clusterfuck of services using technology that requires even more workers to maintain rather than going for pragmatic solutions. I completely reject the notion that you think every project requires "thinking" as if workers there don't actually know what the business entails or what future growth projections were (hint, they haven't been changing much for nearly 70 years). You don't need fancy bleeding edge solutions for 99% of businesses out there. You engineer for what you have, not what you want.

sofixa 2y ago

100k monthly active users can result in wildly different numbers of queries/requests/operations depending on what you're doing. 100k MAU for a static blog and 100k MAU for an online game simulating a real world economy will have vastly different usage patterns of the underlying databases/message queues/etc. > With that in mind why spend so much resources on complexity where the only benefits are to the engineers that get to add another buzz word to their resume? It's not the only benefits, it's making things potentially easier down the line; more often than not it can be premature optimisation (preparing for a future that never comes), but there are also plenty of examples of companies outgrowing their initial infrastructure and having to spend lots of time on rearchitecting everything. A lot of that time can be saved by thinking a little bit beforehand and making things theoretically scalable.

azemetre 2y ago
I really disagree on making things "easier," that's kinda my point. Forcing in solutions that work at Google or Facebook or Twitter into an experience that gets 0.01% of the traffic is bad engineering IMO. This example of a 100k MAU was for a company that was 150+ years old. They know their customers, they know the business, they know the growth projections. The only people that didn't know this was the technology leadership team that was given a blank check to "modernize" while all it did was create a clusterfuck of services using technology that requires even more workers to maintain rather than going for pragmatic solutions. I completely reject the notion that you think every project requires "thinking" as if workers there don't actually know what the business entails or what future growth projections were (hint, they haven't been changing much for nearly 70 years). You don't need fancy bleeding edge solutions for 99% of businesses out there. You engineer for what you have, not what you want.