7 ms·

Everytime I read one of these, no one tells me what resource was the bottleneck in vertical scaling. Does it feel like silicon valley is addicted to horizontal

Everytime I read one of these, no one tells me what resource was the bottleneck in vertical scaling. Does it feel like silicon valley is addicted to horizontal scaling?

John23832 3y ago
Cost (due to higher effective utilization) and single points of failure? Vertical scaling is more expensive without the benefit of failover. With modern clouds, Horizontal scaling is cheaper with commodity hardware and gives the added benefit of failover redundancy.
- wongarsu 3y ago
  If you are already in one of the big 3 cloud providers, vertical scaling is more expensive, especially once you add failover. If you rent bare metal it's often the other way around. So maybe one of the core reasons for Silicon Valley's love for horizontal scaling is due to plentiful AWS credits.
  - deleted 3y ago
    [deleted]
  - deleted 3y ago
    [deleted]
  - John23832 3y ago
    Whether people use modern clouds is orthogonal to whether they choose to use vertical scaling. I can both horizontally and vertically scale in AWS. I can both horizontally and vertically scale in my home office. Horizontally scaling is just more cost efficient if you can dynamically adjust to load. Clouds enable that because they have massive reserve compute. In the startup world, there infinite compute, but finite money. It’s cost savings.
    - CaptainOfCoit 3y ago
      > In the startup world, there infinite compute, but finite money. It’s cost savings. If you're using cloud stuff like EC2 on AWS for cost savings, you're in for a bad time. It makes sense for stuff that you need to be able to dynamically scale on a whim, but once you know what kind of setup you need, it's almost always cheaper to go dedicated.
    - wongarsu 3y ago
      Bare metal, doubling the amount of RAM and using a more powerful CPU is cheaper than adding a whole server that needs space, network, its own Motherboard, case, power supplies etc. This makes vertical scaling much more cost effective (up to a point, which is currently at about 2TB RAM). At AWS's data center AWS also buys big servers. But their advantage is that they only have big servers, and then rent it out in pieces. They don't care whether they rent you 2 48 core 96GB instances or one 96 core 192GB instances. Both options run as VMs on much larger servers and take the same resources, so both are priced the same (a machine of the same category with twice the resources costs exactly twice as much). Thus there's a much smaller benefit to vertical scaling. If you compare actual prices on large-ish instance types between AWS and regular dedicated servers (rented in a DC, not doing your own thing) then AWS is very expensive. And the hopes of paying less due to better utilization somehow never pan out. AWS has lots of advantages, but price isn't one of them; and their price structure will influence the architecture you choose.
  - klooney 3y ago
    AWS charges the same money per core and gig of ram, whether you get a big instance or 5 small instances. I guess their top end isn't as top end as hardware you can get in your DC, though.
  - roncesvalles 3y ago
    The 21st century love for horizontal scaling is for not wanting to be locked in with IBM, because in the limit vertical scaling means buying an IBM mainframe.
dub 3y ago
Typically the price of not having horizontal scaling is felt more by the engineers than the users, at first: - Data migrations, schema changes, backfills, backups & restores, etc., take so long that they can either cause or risk outages or just waste a ton of engineer time waiting around for operations to complete. If you have serious service level objectives regarding time to restore from backup, that alone could be a forcing function for horizontal sharding (doing a point-in-time backup of a 40TB database while dropping some unwanted bad DELETE transaction or something like that from the transaction log is going to be very slow and cause a long outage). - The lack of fault isolation means that any rogue user or process making expensive queries impacts performance and availability for all users, vs being able to limit unavailability to a single shard - When people don't have horizontal scalability, I've seen them normalize things like not using transactions and not using consistent reads even when both would substantially improve developer and end-user experience, with the explanation being a need to protect the primary/write database. It's kind of like being in an abusive relationship: you internalize the fear of overloading the primary/write server and start to think it's normal not to be able to consistently read back the data you just wrote at scale or not to be able to use transactions that span multiple tables or seconds as appropriate.
- dalyons 3y ago
  To your first point I find in these discussions the “just buy a bigger server” crowd massively underestimate the operational problems with giant db servers. They are no fun to babysit, and change gets really hard and tedious to not accidentally bring the whole thing down. It becomes a massive drain on the velocity and agility of the business.
  - baobun 3y ago
    Giant sharded DB clusters aren't that much more fun or less precarious... Ever run Cassandra or Clickhouse at scale? IME vertically scaled replicas/hot-stand-bys are a lot more stable to operate if your requirements allow you to get away with it. OtoH you better already be prepared if/when you hit scaling limits.
baz00 3y ago
It's always the accountants. Those big ass SQL boxes cost serious money.
angarg12 3y ago
For us blast radius is a reason to intentionally limit the size of our servers. We could have smaller fleets with bigger servers, but then if anything went wrong it would impact a larger number of customers. That being said I'm partial of having fleets as small as possible to simplify operations. While trying different instance sizes, I found our bottleneck ended up being network. The largest instance types saturate the network first, and hence we can't use servers as efficiently.

Everytime I read one of these, no one tells me what resource was the bottleneck in vertical scaling. Does it feel like silicon valley is addicted to horizontal scaling?

John23832 3y ago

Cost (due to higher effective utilization) and single points of failure? Vertical scaling is more expensive without the benefit of failover. With modern clouds, Horizontal scaling is cheaper with commodity hardware and gives the added benefit of failover redundancy.

wongarsu 3y ago
If you are already in one of the big 3 cloud providers, vertical scaling is more expensive, especially once you add failover. If you rent bare metal it's often the other way around. So maybe one of the core reasons for Silicon Valley's love for horizontal scaling is due to plentiful AWS credits.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- John23832 3y ago
  Whether people use modern clouds is orthogonal to whether they choose to use vertical scaling. I can both horizontally and vertically scale in AWS. I can both horizontally and vertically scale in my home office. Horizontally scaling is just more cost efficient if you can dynamically adjust to load. Clouds enable that because they have massive reserve compute. In the startup world, there infinite compute, but finite money. It’s cost savings.
  - CaptainOfCoit 3y ago
    > In the startup world, there infinite compute, but finite money. It’s cost savings. If you're using cloud stuff like EC2 on AWS for cost savings, you're in for a bad time. It makes sense for stuff that you need to be able to dynamically scale on a whim, but once you know what kind of setup you need, it's almost always cheaper to go dedicated.
  - wongarsu 3y ago
    Bare metal, doubling the amount of RAM and using a more powerful CPU is cheaper than adding a whole server that needs space, network, its own Motherboard, case, power supplies etc. This makes vertical scaling much more cost effective (up to a point, which is currently at about 2TB RAM). At AWS's data center AWS also buys big servers. But their advantage is that they only have big servers, and then rent it out in pieces. They don't care whether they rent you 2 48 core 96GB instances or one 96 core 192GB instances. Both options run as VMs on much larger servers and take the same resources, so both are priced the same (a machine of the same category with twice the resources costs exactly twice as much). Thus there's a much smaller benefit to vertical scaling. If you compare actual prices on large-ish instance types between AWS and regular dedicated servers (rented in a DC, not doing your own thing) then AWS is very expensive. And the hopes of paying less due to better utilization somehow never pan out. AWS has lots of advantages, but price isn't one of them; and their price structure will influence the architecture you choose.
- klooney 3y ago
  AWS charges the same money per core and gig of ram, whether you get a big instance or 5 small instances. I guess their top end isn't as top end as hardware you can get in your DC, though.
- roncesvalles 3y ago
  The 21st century love for horizontal scaling is for not wanting to be locked in with IBM, because in the limit vertical scaling means buying an IBM mainframe.

dub 3y ago

Typically the price of not having horizontal scaling is felt more by the engineers than the users, at first: - Data migrations, schema changes, backfills, backups & restores, etc., take so long that they can either cause or risk outages or just waste a ton of engineer time waiting around for operations to complete. If you have serious service level objectives regarding time to restore from backup, that alone could be a forcing function for horizontal sharding (doing a point-in-time backup of a 40TB database while dropping some unwanted bad DELETE transaction or something like that from the transaction log is going to be very slow and cause a long outage). - The lack of fault isolation means that any rogue user or process making expensive queries impacts performance and availability for all users, vs being able to limit unavailability to a single shard - When people don't have horizontal scalability, I've seen them normalize things like not using transactions and not using consistent reads even when both would substantially improve developer and end-user experience, with the explanation being a need to protect the primary/write database. It's kind of like being in an abusive relationship: you internalize the fear of overloading the primary/write server and start to think it's normal not to be able to consistently read back the data you just wrote at scale or not to be able to use transactions that span multiple tables or seconds as appropriate.

dalyons 3y ago
To your first point I find in these discussions the “just buy a bigger server” crowd massively underestimate the operational problems with giant db servers. They are no fun to babysit, and change gets really hard and tedious to not accidentally bring the whole thing down. It becomes a massive drain on the velocity and agility of the business.
- baobun 3y ago
  Giant sharded DB clusters aren't that much more fun or less precarious... Ever run Cassandra or Clickhouse at scale? IME vertically scaled replicas/hot-stand-bys are a lot more stable to operate if your requirements allow you to get away with it. OtoH you better already be prepared if/when you hit scaling limits.

baz00 3y ago

It's always the accountants. Those big ass SQL boxes cost serious money.

angarg12 3y ago

For us blast radius is a reason to intentionally limit the size of our servers. We could have smaller fleets with bigger servers, but then if anything went wrong it would impact a larger number of customers. That being said I'm partial of having fleets as small as possible to simplify operations. While trying different instance sizes, I found our bottleneck ended up being network. The largest instance types saturate the network first, and hence we can't use servers as efficiently.