5 ms·

don't do it until you know you have to If you wait until you have to, then isn't it too late to plan and test your solution? So you suffer from degraded perfor

• zeroxfe 5y ago
  They said wait until _you know you have to_, not wait until _you have to_. If you're monitoring your systems appropriately, you should know well before you have to. (modulo inorganic growth explosions, which are hard to predict.) This is just a specific case of premature optimization.
  • jamesfinlayson 5y ago
    Agreed - at a previous job someone tried to force me into implementing some horrible sharding solution they'd dreamt up. I didn't: three years later the project was nearly dead and the database had never gotten anywhere close to maxing out a single machine.
• dexterdog 5y ago
  Yeah Notion waited too long because their performance was atrocious for a long time.
• ozgune 5y ago
  (Ozgun, Citus co-founder) That's a good point. Sharding has an interesting trade-off. Early on, you don't need it and there's an overhead to distributing work across multiple machines (you're taking an additional network hop). Later when/if you need it, sharding becomes painful to introduce. You may need to change your data model for performance and move data. For years, I also cautioned against introducing sharding as a premature optimization. Recently, we changed our approach and also introduced sharding on a single VM. I think that offers a pretty good trade-off. If you're interested in this topic, I’d be curious to hear what you think. https://www.citusdata.com/blog/2021/03/20/sharding-postgres-on-a-single-citus-node/ https://www.citusdata.com/blog/2021/03/20/sharding-postgres-...