5 ms·

I think it's the extent of SQL coverage. As an example, PostgreSQL covers all join types (inner, outer, anti, lateral) that I can think of. Postgres has been b

by ozgune 11y ago

I think it's the extent of SQL coverage. As an example, PostgreSQL covers all join types (inner, outer, anti, lateral) that I can think of. Postgres has been building its SQL coverage for over a decade, and compares favorably to others databases: https://news.ycombinator.com/item?id=9018129 https://news.ycombinator.com/item?id=9018129

mercurial 11y ago
Well, of course it does, but "join" in a distributed database is notoriously different (and often not present at all) from "join" in a non-distributed scenario, so this doesn't make it a strong argument for Postgres.
- mslot 11y ago
  The problem with joins and reason why most distributed databases don't support them is primarily that the local machinery is complicated to implement. Given local join machinery, a join between 2 distributed tables can be achieved by joining the partitions of the tables locally after ensuring that tables are co-partitioned and partitions are co-located. That is, if performing A JOIN B ON (A.X = B.Y), ensure that A is partitioned by X and B is partitioned by Y with the same partitioning scheme such that every partition in A has a corresponding partition in B with which it can be joined. A distributed join can then be performed (in parallel) by joining each pair of partitions and concatenating the results. Ideally, tables are co-partitioned and partitions are co-located in advance. The article describes scenarios in which the tables need to be repartitioned during the query, which can cause a lot of network traffic. However, the amount of traffic is limited in practice since filters and projections can be applied before shuffling any data. The great thing about implementing a distributed database on top of PostgreSQL is that the local machinery already exists, and it is very good.

5 ms·

I think it's the extent of SQL coverage. As an example, PostgreSQL covers all join types (inner, outer, anti, lateral) that I can think of. Postgres has been b

by ozgune 11y ago

mercurial 11y ago
Well, of course it does, but "join" in a distributed database is notoriously different (and often not present at all) from "join" in a non-distributed scenario, so this doesn't make it a strong argument for Postgres.
- mslot 11y ago
  The problem with joins and reason why most distributed databases don't support them is primarily that the local machinery is complicated to implement. Given local join machinery, a join between 2 distributed tables can be achieved by joining the partitions of the tables locally after ensuring that tables are co-partitioned and partitions are co-located. That is, if performing A JOIN B ON (A.X = B.Y), ensure that A is partitioned by X and B is partitioned by Y with the same partitioning scheme such that every partition in A has a corresponding partition in B with which it can be joined. A distributed join can then be performed (in parallel) by joining each pair of partitions and concatenating the results. Ideally, tables are co-partitioned and partitions are co-located in advance. The article describes scenarios in which the tables need to be repartitioned during the query, which can cause a lot of network traffic. However, the amount of traffic is limited in practice since filters and projections can be applied before shuffling any data. The great thing about implementing a distributed database on top of PostgreSQL is that the local machinery already exists, and it is very good.