16 ms·

It's worse than that. Your database will be the first major thing to cave under load. The traditional tools most folks use (postgres/mysql/maria/etc) want to

It's worse than that. Your database will be the first major thing to cave under load. The traditional tools most folks use (postgres/mysql/maria/etc) want to scale vertically, not horizontally (you can do it, but it's hard, and has lots of drawbacks). Eventually - You will hit hardware limits. Providers only have instances that are so big. There is a ceiling on how much you can scale your DB without having to do major, major work. I've seen this pattern play out now at 3 high growth companies I've worked at. The one that was using lots of stored procs and materialized views got hit the fastest and the hardest.

pjmlp 4y ago
When it happens it is time to actually use the big boys databases.
- scarface74 4y ago
  I’ve worked with the big boys and seen plenty of times where databases fell over no matter how much data you through at it because of locking issues and other database contention issues. It’s much easier to horizontally scale a bunch of application servers than database servers.
  - pjmlp 4y ago
    That is why big boys databases do clusters.
    - scarface74 4y ago
      “clusters” only help when you can deal with eventual consistency (which is often admittedly more than most businesses realize). Sure you can scale your read only instances. BTW: I work in the consulting department of a little company I am sure you have heard of that knows something about dealing with “the big boys” at scale.
      - pjmlp 4y ago
        Great, me too, several of them actually.
- AdieuToLogic 4y ago
  > When it happens it is time to actually use the big boys databases. For years Oracle had a very small limit of how many CPU's stored procs could use baked into the DBMS engine. IIRC, it was something like 4. Maybe that's changed now, I don't know. I do know Oracle qualifies for most as being one of "the big boys databases."
- Shorel 4y ago
  MySQL doesn't have materialized views, but it has better replication (at least using Aurora in AWS) than the big boys databases. Adding more replicas and changing the instance size scales much better than fine-tuning the big complexity of Oracle and its materialized views. So, maybe it is not the tool, but how to use it what matters. And fuck Oracle if by big boys databases you mean Oracle.
nijave 4y ago
More generically, scaling stateful things is generally hard than scaling stateless things (because you have to figure out what to do with the state). Even horizontally scaling systems you either have to redistribute the data on scale actions or keep the data in place and send new data to new infra (which may or may not be helpful)

pjmlp 4y ago

When it happens it is time to actually use the big boys databases.

scarface74 4y ago
I’ve worked with the big boys and seen plenty of times where databases fell over no matter how much data you through at it because of locking issues and other database contention issues. It’s much easier to horizontally scale a bunch of application servers than database servers.
- pjmlp 4y ago
  That is why big boys databases do clusters.
  - scarface74 4y ago
    “clusters” only help when you can deal with eventual consistency (which is often admittedly more than most businesses realize). Sure you can scale your read only instances. BTW: I work in the consulting department of a little company I am sure you have heard of that knows something about dealing with “the big boys” at scale.
    - pjmlp 4y ago
      Great, me too, several of them actually.
AdieuToLogic 4y ago
> When it happens it is time to actually use the big boys databases. For years Oracle had a very small limit of how many CPU's stored procs could use baked into the DBMS engine. IIRC, it was something like 4. Maybe that's changed now, I don't know. I do know Oracle qualifies for most as being one of "the big boys databases."
Shorel 4y ago
MySQL doesn't have materialized views, but it has better replication (at least using Aurora in AWS) than the big boys databases. Adding more replicas and changing the instance size scales much better than fine-tuning the big complexity of Oracle and its materialized views. So, maybe it is not the tool, but how to use it what matters. And fuck Oracle if by big boys databases you mean Oracle.

nijave 4y ago

More generically, scaling stateful things is generally hard than scaling stateless things (because you have to figure out what to do with the state). Even horizontally scaling systems you either have to redistribute the data on scale actions or keep the data in place and send new data to new infra (which may or may not be helpful)