6 ms·

> Vertical scaling — a bigger, exponentially more expensive server This is in practice not true at all. Vertical scaling is typically a sublinear cost increase

> Vertical scaling — a bigger, exponentially more expensive server This is in practice not true at all. Vertical scaling is typically a sublinear cost increase (up to a point, but that point is a ridiculous beast of a machine), since you're (typically) upgrading just the CPU and/or just the RAM or just the storage; not all of them at once. There are instances where you can get nearly 10x the machine for 2x the cost.

geodel 3y ago
The idea is don't let the logic come in the way of promoting "web scale" software.
morelisp 3y ago
The kind of server you'd run Kafka on tends to already be pretty far up the curve. I don't think I can get 10x our default broker for 20x the cost. Maybe 100x the cost. (I could probably get 2x it for 2x the cost but once you value HA the practical inflection point starts below the actual cost intersection.)
teawrecks 3y ago
For small consumer products sure, but we're talking at the extreme end of performance and physical capabilities. Sure you can get a 2Ghz CPU for ~2x the price of a 200Mhz CPU, but how much are you going to pay for a 6.0Ghz CPU vs 5.0Ghz? 6.1Ghz vs 6.0Ghz?
- Sohcahtoa82 3y ago
  Think cores instead of clock speeds. In the case of cloud instances, doubling cores is frequently less than 100% more expensive.
  - The_Colonel 3y ago
    Increasing core count is not really vertical scaling. It's a hybrid between vertical and horizontal scaling, having some characteristics of both. It also tops out quite early (especially its cost-effectiveness for many use cases, but there's an absolute upper limit as well).
  - morelisp 3y ago
    https://aws.amazon.com/msk/pricing/ https://aws.amazon.com/msk/pricing/ prices scale linearly with CPU beginning with m5.large, and I wouldn't really want to run a production Kafka on anything less than m5.xlarge. (They do at least keep linearly scaling all the way up.) Speculating wildly, I could probably have run some of our real clusters on the equivalent of a 8xlarge, but of course 32 core systems were not widely available at that time. The cluster I run today, even a hypothetical 48xlarge would struggle. YMMV for non-managed stuff, but really, you can only bump cores like 3 times realistically, 4 if you started really shitty, before you start getting into special pricing brackets.
- marginalia_nu 3y ago
  You can go from a 8T/16C Epyc 7xxx series CPU to a 32T/64C CPU and not even double the cost.
  - fluoridation 3y ago
    That's more like horizontal scaling, though. You get more throughput (transactions per second) but not lower latency (seconds per transaction). Though it may be more cost-effective to have a single 32-core machine than two 16-core machines.
    - marginalia_nu 3y ago
      I disagree with this definition of horizontal scaling. If you're moving to a bigger computer rather than more computers, then you're scaling vertically and not horizontally. (and fwiw, wikipedia agrees with this definition: https://en.wikipedia.org/wiki/Scalability#Horizontal_(scale_out)_and_vertical_scaling_(scale_up) https://en.wikipedia.org/wiki/Scalability#Horizontal_(scale_... )
      - fluoridation 3y ago
        Then it sounds like you have a disagreement of terminology with FTA, since the article is using the terms like I am. Vertical scaling means increasing the serial performance of the system, and horizontal scaling means increasing the parallel performance of the system. In this sense, vertical scaling past a certain point does indeed get exponentially more expensive, while horizontal scaling almost always scales linearly in cost, or better.
        marginalia_nu 3y ago
        What I'm commenting on is this phrasing from the article > Vertical scaling — a bigger, exponentially more expensive server > Horizontal scaling — distribute the load over more servers
        3y ago
dekhn 3y ago
Disagree- typically vertical scaling is lumpy, and even worse- CPU and RAM upgrades are typically not linear, because you're limited by the number of slots/sockets and the manufacturers intentionally charge higher (expoentially) prices for the largest RAM and fastest CPUs.
- morelisp 3y ago
  Kafka is also a system that can make pretty good general use of more CPUs and more storage, but doesn't have much need for RAM. Tying the CPU and RAM together whether by CPU model or cloud vendor offerings is annoying if you're trying to scale only vertically.
  - defendBanana 3y ago
    Kafka can keep a decent bit of data in RAM using file system pages. Often times you end up wasting CPUs on kafka nodes, not memory i think. https://docs.confluent.io/platform/current/kafka/deployment.html https://docs.confluent.io/platform/current/kafka/deployment....
    - morelisp 3y ago
      I find that if you are seeking lots of consumers around large topics no amount of RAM is really sufficient, and if you are mostly sticking to the tails like a regular Kafka user, even 64GB is usually way more than enough. CPU isn't usually a problem until you start using very large compactions, and then suddenly it can be a massive bottleneck. (Actually I would love to abuse more RAM here but log.cleaner.dedupe.buffer.size has a tiny maximum value!) Kafka Streams (specifically) is also configured to transact by default, even though most applications aren't written to be able to actually benefit from that. If you run lots of different consumer services this results in burning a lot of CPU on transactions in a "flat profile"-y way that's hard to illustrate to application developers since each consumer, individually, is relatively small - there's just thousands of them.
- vegabook 3y ago
  If they charge these big numbers more it's precisely because they're trying to capture some of the embarrassingly better value you get from vertical scaling. It's a testament to vertical scaling's effectiveness that they _can_ do so.
KRAKRISMOTT 3y ago
Also beyond a certain point, it makes sense to go straight to dedicated bare metal. The AWS tax is not worth paying if your workload is mostly fixed, somewhat fault tolerant (i.e. failed hardware on the weekends can be replaced on Monday without major interruption to business operations), and CPU bound. Get a high end machine on Hetzner and put everything behind a VPN or API auth and you will save more than 50% in spending.
- RhodesianHunter 3y ago
  I haven't found this to be true generally unless your workloads are truly completely static, which I've never actually experienced. Given what engineers at this level cost, their costs per hour dealing with all of the nonsense clouds handle for you (networking, storage, elastic scaling, instant replacement of faulty servers, load balancing, yadda yadda) end up being higher than whatever tax you're paying for using the cloud. Economies of scale are real.

geodel 3y ago

The idea is don't let the logic come in the way of promoting "web scale" software.

morelisp 3y ago

The kind of server you'd run Kafka on tends to already be pretty far up the curve. I don't think I can get 10x our default broker for 20x the cost. Maybe 100x the cost. (I could probably get 2x it for 2x the cost but once you value HA the practical inflection point starts below the actual cost intersection.)

teawrecks 3y ago

For small consumer products sure, but we're talking at the extreme end of performance and physical capabilities. Sure you can get a 2Ghz CPU for ~2x the price of a 200Mhz CPU, but how much are you going to pay for a 6.0Ghz CPU vs 5.0Ghz? 6.1Ghz vs 6.0Ghz?

Sohcahtoa82 3y ago
Think cores instead of clock speeds. In the case of cloud instances, doubling cores is frequently less than 100% more expensive.
- The_Colonel 3y ago
  Increasing core count is not really vertical scaling. It's a hybrid between vertical and horizontal scaling, having some characteristics of both. It also tops out quite early (especially its cost-effectiveness for many use cases, but there's an absolute upper limit as well).
- morelisp 3y ago
  https://aws.amazon.com/msk/pricing/ https://aws.amazon.com/msk/pricing/ prices scale linearly with CPU beginning with m5.large, and I wouldn't really want to run a production Kafka on anything less than m5.xlarge. (They do at least keep linearly scaling all the way up.) Speculating wildly, I could probably have run some of our real clusters on the equivalent of a 8xlarge, but of course 32 core systems were not widely available at that time. The cluster I run today, even a hypothetical 48xlarge would struggle. YMMV for non-managed stuff, but really, you can only bump cores like 3 times realistically, 4 if you started really shitty, before you start getting into special pricing brackets.
marginalia_nu 3y ago
You can go from a 8T/16C Epyc 7xxx series CPU to a 32T/64C CPU and not even double the cost.
- fluoridation 3y ago
  That's more like horizontal scaling, though. You get more throughput (transactions per second) but not lower latency (seconds per transaction). Though it may be more cost-effective to have a single 32-core machine than two 16-core machines.
  - marginalia_nu 3y ago
    I disagree with this definition of horizontal scaling. If you're moving to a bigger computer rather than more computers, then you're scaling vertically and not horizontally. (and fwiw, wikipedia agrees with this definition: https://en.wikipedia.org/wiki/Scalability#Horizontal_(scale_out)_and_vertical_scaling_(scale_up) https://en.wikipedia.org/wiki/Scalability#Horizontal_(scale_... )
    - fluoridation 3y ago
      Then it sounds like you have a disagreement of terminology with FTA, since the article is using the terms like I am. Vertical scaling means increasing the serial performance of the system, and horizontal scaling means increasing the parallel performance of the system. In this sense, vertical scaling past a certain point does indeed get exponentially more expensive, while horizontal scaling almost always scales linearly in cost, or better.
      - marginalia_nu 3y ago
        What I'm commenting on is this phrasing from the article > Vertical scaling — a bigger, exponentially more expensive server > Horizontal scaling — distribute the load over more servers
        3y ago

dekhn 3y ago

Disagree- typically vertical scaling is lumpy, and even worse- CPU and RAM upgrades are typically not linear, because you're limited by the number of slots/sockets and the manufacturers intentionally charge higher (expoentially) prices for the largest RAM and fastest CPUs.

morelisp 3y ago
Kafka is also a system that can make pretty good general use of more CPUs and more storage, but doesn't have much need for RAM. Tying the CPU and RAM together whether by CPU model or cloud vendor offerings is annoying if you're trying to scale only vertically.
- defendBanana 3y ago
  Kafka can keep a decent bit of data in RAM using file system pages. Often times you end up wasting CPUs on kafka nodes, not memory i think. https://docs.confluent.io/platform/current/kafka/deployment.html https://docs.confluent.io/platform/current/kafka/deployment....
  - morelisp 3y ago
    I find that if you are seeking lots of consumers around large topics no amount of RAM is really sufficient, and if you are mostly sticking to the tails like a regular Kafka user, even 64GB is usually way more than enough. CPU isn't usually a problem until you start using very large compactions, and then suddenly it can be a massive bottleneck. (Actually I would love to abuse more RAM here but log.cleaner.dedupe.buffer.size has a tiny maximum value!) Kafka Streams (specifically) is also configured to transact by default, even though most applications aren't written to be able to actually benefit from that. If you run lots of different consumer services this results in burning a lot of CPU on transactions in a "flat profile"-y way that's hard to illustrate to application developers since each consumer, individually, is relatively small - there's just thousands of them.
vegabook 3y ago
If they charge these big numbers more it's precisely because they're trying to capture some of the embarrassingly better value you get from vertical scaling. It's a testament to vertical scaling's effectiveness that they _can_ do so.

KRAKRISMOTT 3y ago

Also beyond a certain point, it makes sense to go straight to dedicated bare metal. The AWS tax is not worth paying if your workload is mostly fixed, somewhat fault tolerant (i.e. failed hardware on the weekends can be replaced on Monday without major interruption to business operations), and CPU bound. Get a high end machine on Hetzner and put everything behind a VPN or API auth and you will save more than 50% in spending.

RhodesianHunter 3y ago
I haven't found this to be true generally unless your workloads are truly completely static, which I've never actually experienced. Given what engineers at this level cost, their costs per hour dealing with all of the nonsense clouds handle for you (networking, storage, elastic scaling, instant replacement of faulty servers, load balancing, yadda yadda) end up being higher than whatever tax you're paying for using the cloud. Economies of scale are real.