4 ms·
It’s been the same experience for me. Until you get into petabyte levels of data a single replicated and vertically scaled pg is probably going to be just fine.
by SOLAR_FIELDS 4y ago
It’s been the same experience for me. Until you get into petabyte levels of data a single replicated and vertically scaled pg is probably going to be just fine. Quite a few orgs probably realized that eventually after sinking $MM into some Hadoop/Spark setup in the mid 2010’s.
If what you’re doing is
1. Easily parallelizable
2. CPU intensive
3. In 10’s of petabytes or more
Then one of these machine gun like setups makes sense in 2022. Otherwise YAGNI (you aren’t gonna need it)
- ryanjkirk 4y ago"For Basecamp, we take a somewhat different path. We are on record about our feelings about sharding. We prefer to use hardware to scale our databases as long as we can, in order to defer the complexity that is involved in partitioning them as long as possible — with any luck, indefinitely." — Mark Imbriaco, 37Signals These quotes are from 2009 and 2010, and yet here most of us are in 2022, having learned the lesson the hard way over the last decade that there is no refuting this simple logic. I'll add my own truism: All else being equal, designing and maintaining simpler systems will always cost less than complex ones. quote references: https://signalvnoise.com/posts/2479-nuts-bolts-database-servers https://signalvnoise.com/posts/2479-nuts-bolts-database-serv... http://37signals.com/svn/posts/1509-mr-moore-gets-to-punt-on-sharding http://37signals.com/svn/posts/1509-mr-moore-gets-to-punt-on...
- eb0la 4y agoDon't forget evolution: Business do change. A simple sistem far easier to evolve than a complex one, too.