4 ms·

I've seen startups killed because of one or two "influential" programmers deciding they need to start architecturing the project for 1000TPS and 10K daily users

I've seen startups killed because of one or two "influential" programmers deciding they need to start architecturing the project for 1000TPS and 10K daily users, as "that's the proper way to build scalable software", while the project itself hasn't even found product-market fit yet and barely has users. Inevitably, the project needs to make a drastic change which now is so painful to do because it no longer fits the perfect vision the lead(s) had. Cue programmers blaming the product team for "always changing their mind" as they discover what users actually need, and the product team blaming developers for being hesitant to do changes, and when programmers agree, it takes a long time to undo the perfect architecture they've spent weeks fine-tuning against some imaginary future user-base.

the8472 1y ago
1000TPS isn't that much? Engineer for low latency and with a 10ms budget that'd be 10 cores if it were CPU-bound, less in practice since usually part of the time is spent in IO wait.
- drob518 1y ago
  And with CPUs now being shipped with 100+ cores, you can brute force that sucker a long way.
- CaptainOfCoit 1y ago
  > 1000TPS isn't that much? Why does that matter? My argument is: Engineer for what you know, leave the rest for when you know better, which isn't before you have lots of users.
  - the8472 1y ago
    What I'm saying is that "building for 1000TPS" is not what gets you an overengineered 5-layer microservice architecture. If you build for a good user experience (which includes low latency) you get that not-that-big scale without sharding.
- hedora 1y ago
  I doubt much time would be in I/O wait if this was really a scale up architecture. Ignoring the 100's of GB of page cache, it should be sitting on NVMe drives, where a write is just a PCIe round trip, and a read is < 1ms.
otabdeveloper4 1y ago
> 1000TPS and 10K daily users That is not a lot. You can host that on a Raspberry Pi.
- pja 1y ago
  Not if you’re going to be “web scale” (tm) you can’t.
  - hedora 1y ago
    You can host it on 8 raspberry pi's: Three for etcd, three for minio/ceph, and two for Kubernetes workers. (16 if you need geo replication.)
  - moffkalast 1y ago
    You put one Mongo shard on each Pi, they are the secret ingredient in the web scale sauce.
- byroot 1y ago
  That entirely depends on what these transactions are meant to do. I always find these debate weird. How can you compare one app’s TPS with another?
strken 1y ago
I've seen senior engineers get fired and the business suffer a setback because they didn't have any way to scale beyond a single low spec VPS from a budget provider, and their system crashed when a hall full of students tried to sign up together during a demo and each triggered 200ms of bcrypt CPU activity.
- CaptainOfCoit 1y ago
  Wonder which one happens more often? Personally I haven't worked in that kind of "find the person to blame" culture which would led to something like that, so I haven't witnessed what you're talking about, but I believe you it does happen in some places.
- sgarland 1y ago
  That’s a skill issue, not an indictment on the limitations of the architecture. You can spin up N servers and load-balance them, as TFA points out. If the server is a snowflake and has nothing in IaC, again, not an architectural issue, but a personnel / knowledge issue.
  - strken 1y ago
    The architecture in TFA is fine, and sounds preferable to microservices for most use cases. I am worried by the talk of 10k daily users and a peak of 1000TPS being too much premature optimisation. Those numbers are quite low. You should know your expected traffic patterns, add a margin of error, and stress test your system to make sure it can handle the traffic. I disagree that self-inflicted architectural issues and personnel issues are different.
- kunley 1y ago
  I frankly don't believe that in a workplace where an userbase can be characterized as a "hall full of students" anyone was fired overnight. Doesn't happen at these places. Reprimanded, maybe
  - hedora 1y ago
stavros 1y ago
> 1000TPS and 10K daily users I absolutely agree with your point, but I want to point out, like other commenters here, that the numbers should be much larger. We think that, because 10k daily users is a big deal for a product, they're also a big deal for a small server, but they really aren't. It's fantastic that our servers nowadays can easily handle multiple tens of thousands of daily users on $100/mo.
- hamdingers 1y ago
  Users/TPS aren't the right metric in the first place. I have a webhook glue side project that I didn't even realize had ~8k daily users/~300tps until I set up Cloudflare analytics. As a go program doing trivial work, the load is dwarfed by the cpu/memory usage of all my seedbox related software (which has 1 user, not even every day).
  - CaptainOfCoit 1y ago
    > Users/TPS aren't the right metric in the first place. This was my initial point :) Don't focus on trying to achieve some metrics, focus on making sure to build the right thing.
- thewebguyd 1y ago
  > We think that, because 10k daily users is a big deal for a product, they're also a big deal for a small server, but they really aren't. Yeah we seem to forget just how fast computers are now a days. Obviously varies with complexity of the app & what other tech you are using, but for simpler things 10k daily users could be handled by a reasonbly powerful desktop sitting under my desk without breaking a sweat.
systems 1y ago
Clearly this project failed for either 1. scaling for a very specific use case, or because 2. it hasn't even found product-market fit Blaming the failure or designing for scale seem misplaced, you can scale while remaining agile and open to change
smoe 1y ago
In my opinion, if those influential programmers actually architected around some concrete metrics like 1,000 TPS and 10K daily users, they would end up with much simpler systems. The problem I see is much more about extremely vague notions of scalability, trends, best practices, clean code, and so on. For example we need Kafka, because Kafka is for the big boys like us. Not because the alternatives couldn’t handle the actual numbers. CV-driven development is a much bigger issue than people picking overly ambitious target numbers.
jstimpfle 1y ago
Something that does not scale to 10k users is likely so badly architected, it would be faster to iterate on it if it was more scalable hence better architected and more maintainable.
- o11c 1y ago
  For reference, in 1999 10K was still considered a (doable) challenge ... but they were talking "simultaneous" not "per day". The modern equivalent challenge is 10 million simultaneous users per machine.
th0ma5 1y ago
You simply can't get the software or support for a lot of smaller solutions. It can be sometimes easier to do the seemingly more difficult thing, and sometimes because all the money goes to those more difficult seeming technical problems and solutions.
hamburglar 1y ago
I was part of a small team that built a $300M company on Ruby and MySQL that made every scaling mistake you can possibly make. This was also the right decision because it forced us to stay lean and focus on what we needed right now, as opposed to getting starry-eyed about what it was going to be like when we had 10 million users. At every order of magnitude, we had sudden emergencies where some new part of the system had become a bottleneck, and we scrambled like crazy to rearchitect things to accommodate. It was hard, and it was fun. And it was frugal. We eventually hit over 10 million users before I left, and I can’t say I regret the painful approach one bit.
- stavros 1y ago
  I also imagine you were pretty agile by not having tons of complexity to grapple with every time you wanted to add a new feature.
  - hamburglar 1y ago
    Haha, we were tech debt city. It certainly didn’t feel agile.
throwaway894345 1y ago
On the flip side, I've seen a project fail because it was built on the unvalidated assumption that the naive architecture would scale to real world loads only to find that a modest real world workload was exceeding targets by a factor of 100X. You really do need technical leadership with good judgment and experience; we can't substitute it with facile "assume low scale" or "assume large scale" axioms.

the8472 1y ago

1000TPS isn't that much? Engineer for low latency and with a 10ms budget that'd be 10 cores if it were CPU-bound, less in practice since usually part of the time is spent in IO wait.

drob518 1y ago
And with CPUs now being shipped with 100+ cores, you can brute force that sucker a long way.
CaptainOfCoit 1y ago
> 1000TPS isn't that much? Why does that matter? My argument is: Engineer for what you know, leave the rest for when you know better, which isn't before you have lots of users.
- the8472 1y ago
  What I'm saying is that "building for 1000TPS" is not what gets you an overengineered 5-layer microservice architecture. If you build for a good user experience (which includes low latency) you get that not-that-big scale without sharding.
hedora 1y ago
I doubt much time would be in I/O wait if this was really a scale up architecture. Ignoring the 100's of GB of page cache, it should be sitting on NVMe drives, where a write is just a PCIe round trip, and a read is < 1ms.

otabdeveloper4 1y ago

> 1000TPS and 10K daily users That is not a lot. You can host that on a Raspberry Pi.

pja 1y ago
Not if you’re going to be “web scale” (tm) you can’t.
- hedora 1y ago
  You can host it on 8 raspberry pi's: Three for etcd, three for minio/ceph, and two for Kubernetes workers. (16 if you need geo replication.)
- moffkalast 1y ago
  You put one Mongo shard on each Pi, they are the secret ingredient in the web scale sauce.
byroot 1y ago
That entirely depends on what these transactions are meant to do. I always find these debate weird. How can you compare one app’s TPS with another?

strken 1y ago

I've seen senior engineers get fired and the business suffer a setback because they didn't have any way to scale beyond a single low spec VPS from a budget provider, and their system crashed when a hall full of students tried to sign up together during a demo and each triggered 200ms of bcrypt CPU activity.

CaptainOfCoit 1y ago
Wonder which one happens more often? Personally I haven't worked in that kind of "find the person to blame" culture which would led to something like that, so I haven't witnessed what you're talking about, but I believe you it does happen in some places.
sgarland 1y ago
That’s a skill issue, not an indictment on the limitations of the architecture. You can spin up N servers and load-balance them, as TFA points out. If the server is a snowflake and has nothing in IaC, again, not an architectural issue, but a personnel / knowledge issue.
- strken 1y ago
  The architecture in TFA is fine, and sounds preferable to microservices for most use cases. I am worried by the talk of 10k daily users and a peak of 1000TPS being too much premature optimisation. Those numbers are quite low. You should know your expected traffic patterns, add a margin of error, and stress test your system to make sure it can handle the traffic. I disagree that self-inflicted architectural issues and personnel issues are different.
kunley 1y ago
I frankly don't believe that in a workplace where an userbase can be characterized as a "hall full of students" anyone was fired overnight. Doesn't happen at these places. Reprimanded, maybe
- hedora 1y ago

stavros 1y ago

> 1000TPS and 10K daily users I absolutely agree with your point, but I want to point out, like other commenters here, that the numbers should be much larger. We think that, because 10k daily users is a big deal for a product, they're also a big deal for a small server, but they really aren't. It's fantastic that our servers nowadays can easily handle multiple tens of thousands of daily users on $100/mo.

hamdingers 1y ago
Users/TPS aren't the right metric in the first place. I have a webhook glue side project that I didn't even realize had ~8k daily users/~300tps until I set up Cloudflare analytics. As a go program doing trivial work, the load is dwarfed by the cpu/memory usage of all my seedbox related software (which has 1 user, not even every day).
- CaptainOfCoit 1y ago
  > Users/TPS aren't the right metric in the first place. This was my initial point :) Don't focus on trying to achieve some metrics, focus on making sure to build the right thing.
thewebguyd 1y ago
> We think that, because 10k daily users is a big deal for a product, they're also a big deal for a small server, but they really aren't. Yeah we seem to forget just how fast computers are now a days. Obviously varies with complexity of the app & what other tech you are using, but for simpler things 10k daily users could be handled by a reasonbly powerful desktop sitting under my desk without breaking a sweat.

systems 1y ago

Clearly this project failed for either 1. scaling for a very specific use case, or because 2. it hasn't even found product-market fit Blaming the failure or designing for scale seem misplaced, you can scale while remaining agile and open to change

smoe 1y ago

In my opinion, if those influential programmers actually architected around some concrete metrics like 1,000 TPS and 10K daily users, they would end up with much simpler systems. The problem I see is much more about extremely vague notions of scalability, trends, best practices, clean code, and so on. For example we need Kafka, because Kafka is for the big boys like us. Not because the alternatives couldn’t handle the actual numbers. CV-driven development is a much bigger issue than people picking overly ambitious target numbers.

jstimpfle 1y ago

Something that does not scale to 10k users is likely so badly architected, it would be faster to iterate on it if it was more scalable hence better architected and more maintainable.

o11c 1y ago
For reference, in 1999 10K was still considered a (doable) challenge ... but they were talking "simultaneous" not "per day". The modern equivalent challenge is 10 million simultaneous users per machine.

th0ma5 1y ago

You simply can't get the software or support for a lot of smaller solutions. It can be sometimes easier to do the seemingly more difficult thing, and sometimes because all the money goes to those more difficult seeming technical problems and solutions.

hamburglar 1y ago

I was part of a small team that built a $300M company on Ruby and MySQL that made every scaling mistake you can possibly make. This was also the right decision because it forced us to stay lean and focus on what we needed right now, as opposed to getting starry-eyed about what it was going to be like when we had 10 million users. At every order of magnitude, we had sudden emergencies where some new part of the system had become a bottleneck, and we scrambled like crazy to rearchitect things to accommodate. It was hard, and it was fun. And it was frugal. We eventually hit over 10 million users before I left, and I can’t say I regret the painful approach one bit.

stavros 1y ago
I also imagine you were pretty agile by not having tons of complexity to grapple with every time you wanted to add a new feature.
- hamburglar 1y ago
  Haha, we were tech debt city. It certainly didn’t feel agile.

throwaway894345 1y ago

On the flip side, I've seen a project fail because it was built on the unvalidated assumption that the naive architecture would scale to real world loads only to find that a modest real world workload was exceeding targets by a factor of 100X. You really do need technical leadership with good judgment and experience; we can't substitute it with facile "assume low scale" or "assume large scale" axioms.