6 ms·

When I see these stories about horizontal scaling on thousands of nodes, I wonder how many people don't know how much performance can be squeezed out of beefy m

When I see these stories about horizontal scaling on thousands of nodes, I wonder how many people don't know how much performance can be squeezed out of beefy machines. At my job we build probes for network monitoring, and we're able to ingest quite a lot of traffic (up to 14Mpps per link, for a fully saturated 10G link in our hardest tests), ingest a good volume logs or metrics from remote systems, do some extra processing and serve a web application in just a single instance. We use C/C++ for the traffic capture and processing, Postgres/Elastic for storage (depending on the balance performance/flexibility), some AWK for quick file processing (you'd be surprised how fast can it be) and then Python for plumbing code. On the other hand, I've seen technical blogposts and whitepapers on how to optimize a "big data ingestion system" that requires clustering and tweaking and whatnot for a fraction of what we're able to do with a single machine. People should try harder to scale up before starting a cluster and scaling out.

christophilus 5y ago
I was in a discussion on HN just yesterday where the assertion was made that startups should default to the cloud because the upfront cost of allocating real hardware is too high. I’ve got a $20 Netcup server that can probably handle the traffic of 90% of any projects I’ve ever worked on (including my stint at big tech cos). It’s amazing how cheap and fast modern computing is when compared to the standard cloud offerings.
- CogitoCogito 5y ago
  If they don't want to pay upfront for hardware, they can also "default to cloud" while just using cloud-based compute instances.
- fullstackchris 5y ago
  ... but netcup is just another cloud offering? What are you getting at here? I mean, it always makes sense to shop around and look at pricings before choosing one, but my unresearched guess is that there are so many providers nowadays that the pricings across all of them are within cents / dollars for the same service (based on RAM, storage, etc)
lelanthran 5y ago
> When I see these stories about horizontal scaling on thousands of nodes, I wonder how many people don't know how much performance can be squeezed out of beefy machines. Beefy machines? You can get decent throughput (https://news.ycombinator.com/item?id=29659986 https://news.ycombinator.com/item?id=29659986) on the cheapest VM your cloud provider ... uh ... provides.
cameronh90 5y ago
Beefy machines don't give you redundancy though. Beefy machines arguably make redundancy more expensive. If you run on a single beefy VM, you need at least two of those running for redundancy, meaning you're 100% overprovisioned (m6a.32xlarge x2 = $11/hr). If instead you run on 4 smaller instances split across multiple regions (or AZs or whatever), you only need to overprovision by 25% if you want to handle a single failure (m6a.8xlarge x5 = $7/hr). Obviously if you're doing offline or non-latency sensitive processing, that doesn't matter.
- tsimionescu 5y ago
  Conversely, since no one is actually doing true resilience anyway (just look at how many services failed when one AZ in us-east-1 went down), why not just start a new machine with the same code and data (either through active replication or shared hard disks), take the 1h downtime until that's up and running, and be golden? You can even deduct it from your customer's monthly bill.
- hinkley 5y ago
  But if you have 20 of them then the frequency of problems goes up. In the context of this conversation, where people are bringing up situations with 100’s of servers, 4 servers still counts as vertical. For AWS, where you can lose an AZ (and lately that feels like a certainty), there are arguments to be made that 3 or 6 servers are the least you should ever run, even if they sell a machine that could handle the whole workload. You should go down 2 steps and spread out. But don’t allocate 40+ 2xlarge machines to do the work. That is also dumb. What is also screwing up the math here is that AWS is doing something untoward with pricing. With dedicated hardware, for the last six decades, you have a bimodal distribution of cost per instruction. The fastest hardware costs more per cycle, but if your problem doesn’t fit on one (or three) machines you have to pay the price to stay competitive. At the other end the opportunity costs of continuing to produce the crappiest machines available, or the overhead of slicing up a shared box, start to dominate. If you just need a little bit of work done, you’re going to pay almost as much as your neighbor who does three times as much. Half as much at best.
- gjulianm 5y ago
  Depends on what kind of redundancy you want. You can achieve quite a decent uptime for a single instance with local redundancy, process health watchdogs to restart things, crash isolation… Yes, some people will need a cluster with no downtime on failure events, but the clustering itself adds complexity and can even increase downtime if it causes problems. For example, a single machine can’t have load balancer issues, or replication errors. Also, you’re assuming an active-active redundancy scheme, which might not be always necessary, and not counting the cost of the elements required to support the cluster.
hinkley 5y ago
I get the feeling that for non-interactive workloads you want to go as big as you can get. There are many problems you can just crunch through using a computer engineer’s cleverness instead of overdoing it with your own. These are usually throughput problems and the solution domain is distinct. When humans are involved you don’t want a single failure to be felt by everyone at once. And they get grumpy when we don’t pretend their button click is the most important thing we will handle today.

christophilus 5y ago

I was in a discussion on HN just yesterday where the assertion was made that startups should default to the cloud because the upfront cost of allocating real hardware is too high. I’ve got a $20 Netcup server that can probably handle the traffic of 90% of any projects I’ve ever worked on (including my stint at big tech cos). It’s amazing how cheap and fast modern computing is when compared to the standard cloud offerings.

CogitoCogito 5y ago
If they don't want to pay upfront for hardware, they can also "default to cloud" while just using cloud-based compute instances.
fullstackchris 5y ago
... but netcup is just another cloud offering? What are you getting at here? I mean, it always makes sense to shop around and look at pricings before choosing one, but my unresearched guess is that there are so many providers nowadays that the pricings across all of them are within cents / dollars for the same service (based on RAM, storage, etc)

lelanthran 5y ago

> When I see these stories about horizontal scaling on thousands of nodes, I wonder how many people don't know how much performance can be squeezed out of beefy machines. Beefy machines? You can get decent throughput (https://news.ycombinator.com/item?id=29659986 https://news.ycombinator.com/item?id=29659986) on the cheapest VM your cloud provider ... uh ... provides.

cameronh90 5y ago

Beefy machines don't give you redundancy though. Beefy machines arguably make redundancy more expensive. If you run on a single beefy VM, you need at least two of those running for redundancy, meaning you're 100% overprovisioned (m6a.32xlarge x2 = $11/hr). If instead you run on 4 smaller instances split across multiple regions (or AZs or whatever), you only need to overprovision by 25% if you want to handle a single failure (m6a.8xlarge x5 = $7/hr). Obviously if you're doing offline or non-latency sensitive processing, that doesn't matter.

tsimionescu 5y ago
Conversely, since no one is actually doing true resilience anyway (just look at how many services failed when one AZ in us-east-1 went down), why not just start a new machine with the same code and data (either through active replication or shared hard disks), take the 1h downtime until that's up and running, and be golden? You can even deduct it from your customer's monthly bill.
hinkley 5y ago
But if you have 20 of them then the frequency of problems goes up. In the context of this conversation, where people are bringing up situations with 100’s of servers, 4 servers still counts as vertical. For AWS, where you can lose an AZ (and lately that feels like a certainty), there are arguments to be made that 3 or 6 servers are the least you should ever run, even if they sell a machine that could handle the whole workload. You should go down 2 steps and spread out. But don’t allocate 40+ 2xlarge machines to do the work. That is also dumb. What is also screwing up the math here is that AWS is doing something untoward with pricing. With dedicated hardware, for the last six decades, you have a bimodal distribution of cost per instruction. The fastest hardware costs more per cycle, but if your problem doesn’t fit on one (or three) machines you have to pay the price to stay competitive. At the other end the opportunity costs of continuing to produce the crappiest machines available, or the overhead of slicing up a shared box, start to dominate. If you just need a little bit of work done, you’re going to pay almost as much as your neighbor who does three times as much. Half as much at best.
gjulianm 5y ago
Depends on what kind of redundancy you want. You can achieve quite a decent uptime for a single instance with local redundancy, process health watchdogs to restart things, crash isolation… Yes, some people will need a cluster with no downtime on failure events, but the clustering itself adds complexity and can even increase downtime if it causes problems. For example, a single machine can’t have load balancer issues, or replication errors. Also, you’re assuming an active-active redundancy scheme, which might not be always necessary, and not counting the cost of the elements required to support the cluster.

hinkley 5y ago

I get the feeling that for non-interactive workloads you want to go as big as you can get. There are many problems you can just crunch through using a computer engineer’s cleverness instead of overdoing it with your own. These are usually throughput problems and the solution domain is distinct. When humans are involved you don’t want a single failure to be felt by everyone at once. And they get grumpy when we don’t pretend their button click is the most important thing we will handle today.