4 ms·

If your entire application fits in one VM forever - fair enough. If you're running a whole fleet of VMs and need to manage them the pain of doing that is quick

by YZF 3y ago

If your entire application fits in one VM forever - fair enough. If you're running a whole fleet of VMs and need to manage them the pain of doing that is quickly going to exceed the "pain" of using k8s. Do you need something like k8s ingress? Do you need the ability to move workloads if a machine goes down? certs? external-dns? Are you going to deploying tooling that's available off the shelf for k8s? (lessay Prometheus/Grafana?). VMs have overhead and likely waste a lot of resources. I've seen this "monolith" and I've seen containerized/k8s. For anything that needs to scale and needs similar abstractions you get out of k8s you definitely should not try to reproduce this using VMs.

toast0 3y ago
> If your entire application fits in one VM forever - fair enough. A lot of applications can fit on a single machine. Single machines have gotten so big: 256+ cores, terrabytes of ram, huge amounts of fast storage, 100+Gbps NICs. Yeah, you should have 2-4 for continuity, but that doesn't need orchestration. You really don't need to start thinking about horizontal scaling unless you're getting close to filling a single big machine. Especially since the longer it takes for you to scale to big machines, the bigger the machines will be when you get there. If your application is very latency sensitive, then maybe you need to run machines in more places, and maybe you want something to orchestrate that...
- YZF 3y ago
  I agree one physical box can do a lot. As you say though, if you want HA/DR now you're already expanding beyond a single box. You're likely talking multiple boxes in multiple locations. Now you have to think about how your data is potentially replicated across. You have to think about monitoring. You have think about how you upgrade your software and maintain your services. You have OS/package upgrades. Where do you test? Do you have a staging environment? How do you monitor all this stuff? Does your business only have this one app or does it grow to offer more apps? Internal services? I'm sure plenty of things fit this model. But many do not. There's going to be a point where you're taking more pain if you chose the wrong model.
lakomen 3y ago
Why do you need a fleet of VMs? Go can serve about 500M users per month on Xeon E3 v2 CPUs. Why, except for HA do you need more? Do you know how much 5000 (and more in reality) concurrent requests are, and I'm just talking HTTP1.1 requests? And with HTTP/2 and 48 cores 96 threads and even 24c/48t commodity hardware that's multiples of 5k concurrent rps. Of course if you insist on using low performance php, node, ruby, elixir that's your own fault. "Scale" my ass. Just another FUD buzzword to sell expensive proprietary services.
- rented_mule 3y ago
  I'm not who you're responding to, but... > Go can serve about 500M users per month on Xeon E3 v2 CPUs. > Why, except for HA do you need more? Because not everything is a simple CRUD website. The last place I worked had a website that appeared to only be moderately more complex than such a site, and had well under 500M users per month. But parts of the application involved complicated processing of many dozens of TBs per day of newly collected data to optimize the content being served (some stream processing, some batch). We had multiple clusters of 1,000+ machines coming and going for an hour or two at a time (often overlapping).
- YZF 3y ago
  The problem isn't the 5000 request/s on your Go code. What do these requests do? Is there a database? Is it replicated? How large does it need to be? How complex are the transactions/queries? What happens when your one server goes down? What happens when the Data Center hosting your server(s) goes down? Do you have a large dev team? How do they develop? How do they test? How large is your application? What happens when your company builds new applications? How do you do updates/upgrades of your software and the dependencies? Do you have a staging environment? Most of my experience is with high performance software. C++/C and yeah lots of Go. My observations aren't related to performance. There's more to scale and providing a reliable service than performance. I am totally onboard with the idea that people often go horizontally when they don't need to performance-wise because they choose slow tools. This is not what I'm talking about here. That said, there's probably a class of problems that are fine with just one VM. So if that's what you have, go for it (which is what I said in my original comment).

by YZF 3y ago

toast0 3y ago

> If your entire application fits in one VM forever - fair enough. A lot of applications can fit on a single machine. Single machines have gotten so big: 256+ cores, terrabytes of ram, huge amounts of fast storage, 100+Gbps NICs. Yeah, you should have 2-4 for continuity, but that doesn't need orchestration. You really don't need to start thinking about horizontal scaling unless you're getting close to filling a single big machine. Especially since the longer it takes for you to scale to big machines, the bigger the machines will be when you get there. If your application is very latency sensitive, then maybe you need to run machines in more places, and maybe you want something to orchestrate that...

YZF 3y ago
I agree one physical box can do a lot. As you say though, if you want HA/DR now you're already expanding beyond a single box. You're likely talking multiple boxes in multiple locations. Now you have to think about how your data is potentially replicated across. You have to think about monitoring. You have think about how you upgrade your software and maintain your services. You have OS/package upgrades. Where do you test? Do you have a staging environment? How do you monitor all this stuff? Does your business only have this one app or does it grow to offer more apps? Internal services? I'm sure plenty of things fit this model. But many do not. There's going to be a point where you're taking more pain if you chose the wrong model.

lakomen 3y ago

Why do you need a fleet of VMs? Go can serve about 500M users per month on Xeon E3 v2 CPUs. Why, except for HA do you need more? Do you know how much 5000 (and more in reality) concurrent requests are, and I'm just talking HTTP1.1 requests? And with HTTP/2 and 48 cores 96 threads and even 24c/48t commodity hardware that's multiples of 5k concurrent rps. Of course if you insist on using low performance php, node, ruby, elixir that's your own fault. "Scale" my ass. Just another FUD buzzword to sell expensive proprietary services.

rented_mule 3y ago
I'm not who you're responding to, but... > Go can serve about 500M users per month on Xeon E3 v2 CPUs. > Why, except for HA do you need more? Because not everything is a simple CRUD website. The last place I worked had a website that appeared to only be moderately more complex than such a site, and had well under 500M users per month. But parts of the application involved complicated processing of many dozens of TBs per day of newly collected data to optimize the content being served (some stream processing, some batch). We had multiple clusters of 1,000+ machines coming and going for an hour or two at a time (often overlapping).
YZF 3y ago
The problem isn't the 5000 request/s on your Go code. What do these requests do? Is there a database? Is it replicated? How large does it need to be? How complex are the transactions/queries? What happens when your one server goes down? What happens when the Data Center hosting your server(s) goes down? Do you have a large dev team? How do they develop? How do they test? How large is your application? What happens when your company builds new applications? How do you do updates/upgrades of your software and the dependencies? Do you have a staging environment? Most of my experience is with high performance software. C++/C and yeah lots of Go. My observations aren't related to performance. There's more to scale and providing a reliable service than performance. I am totally onboard with the idea that people often go horizontally when they don't need to performance-wise because they choose slow tools. This is not what I'm talking about here. That said, there's probably a class of problems that are fine with just one VM. So if that's what you have, go for it (which is what I said in my original comment).