4 ms·
Hi, author here. 1) We use software called Citus[0] which sits on top of Postgres and handles all of the read time sharding logic for us. Citus could handle th
by malisper 10y ago
Hi, author here.
1) We use software called Citus[0] which sits on top of Postgres and handles all of the read time sharding logic for us. Citus could handle the write time logic, except we wrote our own logic because we are able to perform several optimizations for our specific use case that don't work in general.
2) Right now, we have a 40 vanilla PostgreSQL instances running on ZFS on Linux and a single Citus master node. We perform writes by sending them directly to the proper vanilla PostgreSQL instance (why we don't use Citus for the write path). We perform reads by sending a vanilla SQL query to the citus master, which will automatically figure out what data it needs from each worker, send queries to fetch all of the data, and then aggregate the results and return them.
3) We wrote our own code for handling resharding and rebalancing different shards across our cluster.
[0] https://www.citusdata.com/ https://www.citusdata.com/
- pradeepchhetri 10y agoVery cool. Thank you for the answers.