6 ms·

And run it all on a single piece of big iron: https://thmsmlr.com/cheap-infra https://thmsmlr.com/cheap-infra I'm not joking either.

hobs 2y ago
It's mostly right, but the problem lies in when the requests per second are not average. Even then most people just do some unscalable stuff on the cloud for minimum 10x the cost.
ndriscoll 2y ago
Where big iron might be something between a raspberry pi with an NVMe hat and a gaming PC. My 6th gen i5 (4 cores) can render ~15k page views per second with database queries, or several times that serving cached pages from nginx. 150 requests/second wouldn't be noticeable if I were using it for gaming at the same time.
- deleted 2y ago
  [deleted]
- SavageBeast 2y ago
  Thanks for the metric - I wouldn't have naively assumed that kind of through put was possible on such a setup.
  - ndriscoll 2y ago
    There are some caveats to that, like you can't get anywhere near that with TLS unless you have hardware offload (I've seen claims that it's ~1% overhead, which seems to be 15 years out of date. For cached content, TLS handshakes cause more like a 90% reduction in throughput on my computer). Unfortunately Intel segments this, so that offload isn't on consumer chips. You might be able to use a cheap firewall device with QAT as a reverse proxy to terminate TLS though (it is on some cheap atom processors). I don't have one to measure what it can do. Obviously that's not something a large corporation would do, but for a small bootstrapped project, it ought to take you way farther than you could need. Also batching database queries is important for high throughput.
- kgeist 2y ago
  Another team's product at our company runs inside a Kubernetes cluster which consists of lots of microservices. CPU's are at 80% all the time. Redis, RabbitMQ, MySQL, Elastic Search etc. There's no number crunching, mostly CRUD with some business logic on top. It requires a separate operations team to manage it (otherwise this goldberg machine easily explodes). A few days ago I asked our ops team, what's our rps? ~500. Out of curiosity, I ran a few benchmarks to stress test my monolithic pet project in Go with a single DB for everything and it easily handled 15k/s on my home computer, too, without any special optimizations. I understand its load is totally different from our production server, and my benchmark is most likely flawed, but the order of magnitude of difference is interesting. I was told the production cluster also has additional 500 rps just for communication between the microservices. I suspect there's just too much overhead in their setup: communication between services, serialization of data between different DBs, etc. If it was a single modular monolith with a single DB, I suspect it could be faster and easier to maintain... Another issue is that they also sell it as an on-premise solution. And with the complexity growing, it now often fails to run on clients' infra.
  - rovr138 2y ago
    At first I was "eh, if it runs okay and can just be scaled, it's fine as long as they know the limits, how it scales, and it's maintained". > And with the complexity growing, it now often fails to run on clients' infra. then I saw this and... yeah completely different beast
  - earnesti 2y ago
    But imagine how many billable hours were created with this thing
    - Nextgrid 2y ago
      Unless it's a consultancy project it's less about billable hours and more about internal busywork and resume fodder. In a ZIRP/startups environment this kind of BS was rewarded by the market and thus "engineers" would take any opportunity to build up such unproductive over-engineering skills to polish up their resume for the next opportunity.
- vineyardmike 2y ago
  > Where big iron might be something between a raspberry pi with an NVMe hat and a gaming PC. At my last startup (with ~3 technical employees), I was talking to the CTO about our cloud spend and planning for properly scaling infra after we got our first customer. We weren't "a SaaS website", but (hand-waving) we did complex physics modeling and analytics on data streams. The (physicist) CTO had heard of k8s and cloud stuff and wanted me to investigate how we should scale our cloud. He was convinced it'd be expensive to do "all those physics calculation with 64 bit numbers". I showed him our entire cloud - gateway, database(s), physics modeling, metrics collection, log aggregation, etc - running at 1000x estimated 1-customer load from a MacBook. I offered to set him up a personal raspberry pi to stress test before a k8s cluster. We ended up with a sensible single medium EC2 instance running everything, with some extra stuff for fail-over. AFAIK the only change made after I left was using a cloud-vendor DB.
tkiolp4 2y ago
I agree with everything said there except one thing: high availability is difficult to achieve with a single box. So, I would use two.
- ffsm8 2y ago
  I will have to disagree with your disagreement. High availability is not difficult to achieve with a single box, it's just depending on luck I've had boxes run with 100% availability for multiple years
  - SavageBeast 2y ago
    In terms of hardware issues, Ive had a single EC2 instance running application servers and MySQL (YUP - ran a decently traffic intensive site like this). All software was run on a virtual instance. I configured all the necessary things to run on startup of the machine and the worst thing AMZN ever did was occasionally reboot the host machine my instance was running on. I was never down and there were "planned restarts" that I'd get the odd email about. In the event some real failure happened, I had a clone of the AMI and could easily dump it onto new hardware - after I knew about it of course. All that said - that situation wasn't my Job - and if Im being paid to have uptime, Id better cover my ass and have a hot failover on standby. Since Im speaking AWS here - of course I used EBS for the virtual filesystem such that a machine crash didn't lose my SSD based filesystem. As I'm sure many here know, you can run a database on an EBS backed filesystem (and I have to assume even mount a swap file there too). I thought it was crack-smoking a when someone suggested that could easily be done. "But if its that simple - why is everyone running all these N tier setups and living with all this unnecessary complexity?!?!"
gymbeaux 2y ago
The industry really has become bloated. Now we have DevOps and Agile Coaches and Cloud Engineers and microservices and JavaScript Everywhere(TM). It’s all bullshit. DevOps is really so complex that a medium-sized company needs an entire team? Earlier today I was trying to set up some elaborate Postgres database backup and couldn’t get it to work so I just put the command in the cron tab and called it a day. Took 1 minute.
- vineyardmike 2y ago
  > DevOps is really so complex that a medium-sized company needs an entire team? Depends on your goals, but most people probably don't need it. > I just put the command in the cron tab and called it a day Obviously this will lead to data loss if any data is written and then a corruption event or issue occurs in-between cron events. The road to over-complicated infrastructure is paved with mistakes and outages. The home-lab server running on a pi in my closet can use cron jobs and scripts, but at work we maintain higher standards because our data loss would cost more than a few extra days of engineer time. Everything should be as simple as possible, but let's not pretend everyone is a chump because their engineering tolerances don't allow for data loss.
  - gymbeaux 2y ago
    I can have a second DB server with replication and still forego the DevOps team.

hobs 2y ago

It's mostly right, but the problem lies in when the requests per second are not average. Even then most people just do some unscalable stuff on the cloud for minimum 10x the cost.

ndriscoll 2y ago

Where big iron might be something between a raspberry pi with an NVMe hat and a gaming PC. My 6th gen i5 (4 cores) can render ~15k page views per second with database queries, or several times that serving cached pages from nginx. 150 requests/second wouldn't be noticeable if I were using it for gaming at the same time.

deleted 2y ago
[deleted]
SavageBeast 2y ago
Thanks for the metric - I wouldn't have naively assumed that kind of through put was possible on such a setup.
- ndriscoll 2y ago
  There are some caveats to that, like you can't get anywhere near that with TLS unless you have hardware offload (I've seen claims that it's ~1% overhead, which seems to be 15 years out of date. For cached content, TLS handshakes cause more like a 90% reduction in throughput on my computer). Unfortunately Intel segments this, so that offload isn't on consumer chips. You might be able to use a cheap firewall device with QAT as a reverse proxy to terminate TLS though (it is on some cheap atom processors). I don't have one to measure what it can do. Obviously that's not something a large corporation would do, but for a small bootstrapped project, it ought to take you way farther than you could need. Also batching database queries is important for high throughput.
kgeist 2y ago
Another team's product at our company runs inside a Kubernetes cluster which consists of lots of microservices. CPU's are at 80% all the time. Redis, RabbitMQ, MySQL, Elastic Search etc. There's no number crunching, mostly CRUD with some business logic on top. It requires a separate operations team to manage it (otherwise this goldberg machine easily explodes). A few days ago I asked our ops team, what's our rps? ~500. Out of curiosity, I ran a few benchmarks to stress test my monolithic pet project in Go with a single DB for everything and it easily handled 15k/s on my home computer, too, without any special optimizations. I understand its load is totally different from our production server, and my benchmark is most likely flawed, but the order of magnitude of difference is interesting. I was told the production cluster also has additional 500 rps just for communication between the microservices. I suspect there's just too much overhead in their setup: communication between services, serialization of data between different DBs, etc. If it was a single modular monolith with a single DB, I suspect it could be faster and easier to maintain... Another issue is that they also sell it as an on-premise solution. And with the complexity growing, it now often fails to run on clients' infra.
- rovr138 2y ago
  At first I was "eh, if it runs okay and can just be scaled, it's fine as long as they know the limits, how it scales, and it's maintained". > And with the complexity growing, it now often fails to run on clients' infra. then I saw this and... yeah completely different beast
- earnesti 2y ago
  But imagine how many billable hours were created with this thing
  - Nextgrid 2y ago
    Unless it's a consultancy project it's less about billable hours and more about internal busywork and resume fodder. In a ZIRP/startups environment this kind of BS was rewarded by the market and thus "engineers" would take any opportunity to build up such unproductive over-engineering skills to polish up their resume for the next opportunity.
vineyardmike 2y ago
> Where big iron might be something between a raspberry pi with an NVMe hat and a gaming PC. At my last startup (with ~3 technical employees), I was talking to the CTO about our cloud spend and planning for properly scaling infra after we got our first customer. We weren't "a SaaS website", but (hand-waving) we did complex physics modeling and analytics on data streams. The (physicist) CTO had heard of k8s and cloud stuff and wanted me to investigate how we should scale our cloud. He was convinced it'd be expensive to do "all those physics calculation with 64 bit numbers". I showed him our entire cloud - gateway, database(s), physics modeling, metrics collection, log aggregation, etc - running at 1000x estimated 1-customer load from a MacBook. I offered to set him up a personal raspberry pi to stress test before a k8s cluster. We ended up with a sensible single medium EC2 instance running everything, with some extra stuff for fail-over. AFAIK the only change made after I left was using a cloud-vendor DB.

tkiolp4 2y ago

I agree with everything said there except one thing: high availability is difficult to achieve with a single box. So, I would use two.

ffsm8 2y ago
I will have to disagree with your disagreement. High availability is not difficult to achieve with a single box, it's just depending on luck I've had boxes run with 100% availability for multiple years
- SavageBeast 2y ago
  In terms of hardware issues, Ive had a single EC2 instance running application servers and MySQL (YUP - ran a decently traffic intensive site like this). All software was run on a virtual instance. I configured all the necessary things to run on startup of the machine and the worst thing AMZN ever did was occasionally reboot the host machine my instance was running on. I was never down and there were "planned restarts" that I'd get the odd email about. In the event some real failure happened, I had a clone of the AMI and could easily dump it onto new hardware - after I knew about it of course. All that said - that situation wasn't my Job - and if Im being paid to have uptime, Id better cover my ass and have a hot failover on standby. Since Im speaking AWS here - of course I used EBS for the virtual filesystem such that a machine crash didn't lose my SSD based filesystem. As I'm sure many here know, you can run a database on an EBS backed filesystem (and I have to assume even mount a swap file there too). I thought it was crack-smoking a when someone suggested that could easily be done. "But if its that simple - why is everyone running all these N tier setups and living with all this unnecessary complexity?!?!"

gymbeaux 2y ago

The industry really has become bloated. Now we have DevOps and Agile Coaches and Cloud Engineers and microservices and JavaScript Everywhere(TM). It’s all bullshit. DevOps is really so complex that a medium-sized company needs an entire team? Earlier today I was trying to set up some elaborate Postgres database backup and couldn’t get it to work so I just put the command in the cron tab and called it a day. Took 1 minute.

vineyardmike 2y ago
> DevOps is really so complex that a medium-sized company needs an entire team? Depends on your goals, but most people probably don't need it. > I just put the command in the cron tab and called it a day Obviously this will lead to data loss if any data is written and then a corruption event or issue occurs in-between cron events. The road to over-complicated infrastructure is paved with mistakes and outages. The home-lab server running on a pi in my closet can use cron jobs and scripts, but at work we maintain higher standards because our data loss would cost more than a few extra days of engineer time. Everything should be as simple as possible, but let's not pretend everyone is a chump because their engineering tolerances don't allow for data loss.
- gymbeaux 2y ago
  I can have a second DB server with replication and still forego the DevOps team.