2 ms·

That's a very cool demo, good job! I have a few questions if you don't mind. - Could you share the specs of the hardware powering the demo? - For IoT-type usa

That's a very cool demo, good job! I have a few questions if you don't mind. - Could you share the specs of the hardware powering the demo? - For IoT-type usage, one common access pattern is "get the readings of sensor X from device Y in the given time frame". But unfortunately, the data set in the demo doesn't include something like "cab_id" column that would allow me to simulate such a query. Is there a demo that has such a column? - Does QuestDB require rows to be inserted strictly in order, or is there some leeway? - Are you using any sort of compression? Timeseries columns are extremely compressible, and a good compression algorithm can greatly reduce IO wait times. We're using the Postgres-based TimescaleDb at work, and its compression capabilities have been a game changer [0]. - Do you plan on supporting GROUP BY clauses? [0]: https://blog.timescale.com/blog/building-columnar-compression-in-a-row-oriented-database/ https://blog.timescale.com/blog/building-columnar-compressio...

sirffuzzylogik 6y ago
Thanks for your feedback, I am pleased to know that you like it! - This is a c5.metal instance (AWS) with 48 physical cores and 192 GB of memory. We use only 24 of those 48 cores. - If I understand your example well, the query "select * from trips where trip_type = 'Dispatch' and pickup_datetime = '2018-03' and vendor_id = 'VTS';" should be similar. There is no limit in how many filters you can specify in the where clause, it will also have a very minor (if any) impact on the performance. - Currently yes but this is something we are working on. Out of order insertions is one of our priority and you can expect this to be implemented in a very near future. - Currently no. However, the way data is stored means that columns are extremely compressible as you mentioned. Our internal API (how we store, access/retrieve columns) would make it easy for us to implement compression, we will do it if this is something important for our users. In the meantime, a filesystem like BTRFS or ZFS would do a nice job (if disk usage is your concern). - GROUP BY is automatically inferred, users don't need to specify it explicitly, you can find more details about this here: https://questdb.io/docs/sqlExtensions#important-differences-from-standard-sql https://questdb.io/docs/sqlExtensions#important-differences-... EDIT: fix typos

sirffuzzylogik 6y ago

Thanks for your feedback, I am pleased to know that you like it! - This is a c5.metal instance (AWS) with 48 physical cores and 192 GB of memory. We use only 24 of those 48 cores. - If I understand your example well, the query "select * from trips where trip_type = 'Dispatch' and pickup_datetime = '2018-03' and vendor_id = 'VTS';" should be similar. There is no limit in how many filters you can specify in the where clause, it will also have a very minor (if any) impact on the performance. - Currently yes but this is something we are working on. Out of order insertions is one of our priority and you can expect this to be implemented in a very near future. - Currently no. However, the way data is stored means that columns are extremely compressible as you mentioned. Our internal API (how we store, access/retrieve columns) would make it easy for us to implement compression, we will do it if this is something important for our users. In the meantime, a filesystem like BTRFS or ZFS would do a nice job (if disk usage is your concern). - GROUP BY is automatically inferred, users don't need to specify it explicitly, you can find more details about this here: https://questdb.io/docs/sqlExtensions#important-differences-from-standard-sql https://questdb.io/docs/sqlExtensions#important-differences-... EDIT: fix typos