4 ms·
Are there any examples on how the resultant SQL tables look like in postgres or clickhouse for a given event schema? I'd like to know how generic it is per even
by polskibus 5y ago
Are there any examples on how the resultant SQL tables look like in postgres or clickhouse for a given event schema? I'd like to know how generic it is per event type (is it sth like (event id, blob), or tries to decompose each event field into a column - what about nested objects then, etc.)). Knowing this would greatly improve my understanding on reusability of jitsu for various event-collection tasks I may have.
- vklmn 5y agoThat's what the website missing indeed. We have a few words about that in docs, but it's still not enough https://jitsu.com/docs/internals/jitsu-server#mapping-step https://jitsu.com/docs/internals/jitsu-server#mapping-step Overall, Jitsu tries to decompose (aka flatten) JSON as deep as possible. E.g. {a: {b:1, c:2}} will become a_b=1, a_c=2. If column is missing, it will be created. We don't decompose arrays so far
- atombender 5y agoCan one disable the flattening? BigQuery, for example, supports nested objects just fine, and flattening them for no particular reason seems counter-productive. I work on an application where we already have a schema in BQ, but we'd like to start moving events through something like Jitsu or Rudderstack. This uses nested objects extensively. Looking at Jitsu, it looks like we wouldn't be able to keep our existing table schema. PS. Whoever wrote your BigQuery code does not understand Go contexts. Only functions should take a context argument; you should almost never store contexts in structs!