3 ms·

Databases are notoriously better scaled with more iron. Web servers on the other hand are a great fit for horizontal. Horizontally scaling your database is a l

by Swizec 6y ago

Databases are notoriously better scaled with more iron. Web servers on the other hand are a great fit for horizontal. Horizontally scaling your database is a last resort used at truly mindboggling scales.

louwrentius 6y ago
I bet that you can handle a tremendous amount of traffic with a single machine (2 in ha for availability behind an LB). Especially with today’s hardware or simple cloud vms you’ll come a long way.
cortesoft 6y ago
I don’t think the size where you need to horizontally scale your database is as ‘mindboggling’ly large as you make it out to be. There are a lot of engineers working at a lot of companies who are at and above that scale.
- josephg 6y ago
  Most websites (probably 99.9% of websites) never reach that scale. And I suspect most working engineers work on products that aren't at that scale. I mean, how many websites, anywhere in the world are bigger than stackoverflow? 100? There's interesting tech out there, but I think adding "interesting" technology to your stack without a very good reason is usually a mistake.
  - cortesoft 6y ago
    Sure, but not everyone is making websites.
    - josephg 6y ago
      True. And likewise, not everyone needs an ETL pipeline. "Keep your architecture as simple as possible, but no simpler". There are important problems kafka solves. But its worth checking if you have one of those problems before using it.
- Swizec 6y ago
  I think many engineers underestimate a well-tuned postgres with a good memcache layer. Most of your traffic likely won’t even hit the database. And a good ORM supports this out of the box.
  - cortesoft 6y ago
    This assumes that everyone is making websites.
    - Swizec 6y ago
      No? It assumes most people using databases have more reads than writes and find relations useful.
- paulryanrogers 6y ago
  Scaling the writeable DB horizontally is the hard part. You can get pretty far with tuning, bare metal (if load isn't too lumpy), many read only copies, caches, and other boring solutions.

by Swizec 6y ago

louwrentius 6y ago

I bet that you can handle a tremendous amount of traffic with a single machine (2 in ha for availability behind an LB). Especially with today’s hardware or simple cloud vms you’ll come a long way.

cortesoft 6y ago

I don’t think the size where you need to horizontally scale your database is as ‘mindboggling’ly large as you make it out to be. There are a lot of engineers working at a lot of companies who are at and above that scale.

josephg 6y ago
Most websites (probably 99.9% of websites) never reach that scale. And I suspect most working engineers work on products that aren't at that scale. I mean, how many websites, anywhere in the world are bigger than stackoverflow? 100? There's interesting tech out there, but I think adding "interesting" technology to your stack without a very good reason is usually a mistake.
- cortesoft 6y ago
  Sure, but not everyone is making websites.
  - josephg 6y ago
    True. And likewise, not everyone needs an ETL pipeline. "Keep your architecture as simple as possible, but no simpler". There are important problems kafka solves. But its worth checking if you have one of those problems before using it.
Swizec 6y ago
I think many engineers underestimate a well-tuned postgres with a good memcache layer. Most of your traffic likely won’t even hit the database. And a good ORM supports this out of the box.
- cortesoft 6y ago
  This assumes that everyone is making websites.
  - Swizec 6y ago
    No? It assumes most people using databases have more reads than writes and find relations useful.
paulryanrogers 6y ago
Scaling the writeable DB horizontally is the hard part. You can get pretty far with tuning, bare metal (if load isn't too lumpy), many read only copies, caches, and other boring solutions.