7 ms·

You mean cstore[0]? No. We've some jsonb but very little. We know what our data look like, so we don't have to. [0] https://github.com/citusdata/cstore_fdw htt

by doh 9y ago

You mean cstore[0]? No. We've some jsonb but very little. We know what our data look like, so we don't have to. [0] https://github.com/citusdata/cstore_fdw https://github.com/citusdata/cstore_fdw

barrkel 9y ago
That's the bit. Not schema free, but column oriented instead of row oriented. I evaluated Citus a while back comparing to Impala w/ Parquet on HDFS and a couple of other systems. Citus was underwhelming in that context, and the syntax for the fdw awkward owing to keeping base Postgres stock.
- doh 9y ago
  Understood. Different use cases. We like schema, gives us an opportunity to keep data in check. We also communicate through protobuf so there is no benefit in schema-free database for us. I know that Heap Analytics is using jsonb to have schema-free setup on citus and I think it's working well for them.
  - barrkel 9y ago
    That's the second time you've thought I was talking about schema free. Column stores have nothing to do with schema free. Completely orthogonal. Column-oriented stores normally have just as much schema as row-oriented stores. What they're faster at is scans where the full row tuple is not required to compute the result. Storing the data in columnar format rather than row also means much better compression: similar data is brought closer together.
    - doh 9y ago
      You're correct, I misunderstood you. Column stores are great, for some tasks and have disadvantages in others. Redshift was I believe the first implementation at top of PG so it's doable, however not sure that PG itself is a good engine for it. In our case, where we need to have access to multiple columns at once, column stores (including the ones that clump them as families/groups) proved to be slower and required more resources than the row-oriented stores. I think this really goes case by case based on your needs. If we would benefit from column-oriented DB we would not chose PG/Citus but something else (probably would stay still on Hbase).

by doh 9y ago

You mean cstore[0]? No. We've some jsonb but very little. We know what our data look like, so we don't have to. [0] https://github.com/citusdata/cstore_fdw https://github.com/citusdata/cstore_fdw

barrkel 9y ago

That's the bit. Not schema free, but column oriented instead of row oriented. I evaluated Citus a while back comparing to Impala w/ Parquet on HDFS and a couple of other systems. Citus was underwhelming in that context, and the syntax for the fdw awkward owing to keeping base Postgres stock.

doh 9y ago
Understood. Different use cases. We like schema, gives us an opportunity to keep data in check. We also communicate through protobuf so there is no benefit in schema-free database for us. I know that Heap Analytics is using jsonb to have schema-free setup on citus and I think it's working well for them.
- barrkel 9y ago
  That's the second time you've thought I was talking about schema free. Column stores have nothing to do with schema free. Completely orthogonal. Column-oriented stores normally have just as much schema as row-oriented stores. What they're faster at is scans where the full row tuple is not required to compute the result. Storing the data in columnar format rather than row also means much better compression: similar data is brought closer together.
  - doh 9y ago
    You're correct, I misunderstood you. Column stores are great, for some tasks and have disadvantages in others. Redshift was I believe the first implementation at top of PG so it's doable, however not sure that PG itself is a good engine for it. In our case, where we need to have access to multiple columns at once, column stores (including the ones that clump them as families/groups) proved to be slower and required more resources than the row-oriented stores. I think this really goes case by case based on your needs. If we would benefit from column-oriented DB we would not chose PG/Citus but something else (probably would stay still on Hbase).