3 ms·
Not yet. It is quite a bit rough around the edges and far from production use. Besides, it will require creation of multiple tables/column for each schema type
by gauravphoenix 4y ago
Not yet. It is quite a bit rough around the edges and far from production use. Besides, it will require creation of multiple tables/column for each schema type (i.e. github schema might conflict with gitlab schema). As such, we decided to flatten the JSONs and use map data type and have our own SQL layer that translates the queries to underlying ClickHouse queries. This allows us to add a lot of syntactic sugar e.g. https://docs.dassana.cloud/docs/query/intro https://docs.dassana.cloud/docs/query/intro
We might start using that feature in future.
- magundu 4y agoWhat do you mean by flatten the JSON? How it is stored in clickhouse?
- gauravphoenix 4y agoWe flatten it to a map and store the map. When you query, we dynamically generate ClickHouse query that queries the map.
- magundu 4y agoIf the JSON is nested, how do you put it the map? Since map key is one type, how do you handle multiple types of values?
- gauravphoenix 4y agowe flatten nested JSONs too. Not sure what you meant by second question, can you rephrase or provide an example? Also, feel free to drop me an email or join slack[1]- it will be easier to discuss tech details there. [1] https://dassanacommunity.slack.com/join/shared_invite/zt-teo6d5ed-xkWDNSaH4m6pC8PAJnrD8g#/shared-invite/email https://dassanacommunity.slack.com/join/shared_invite/zt-teo...