3 ms·
> You decide to store the raw JSON, as normalization is non-trivial. You lost me. If you have to build the index, normalization is immediately trivialized. Thi
by derpmcterp 5y ago
> You decide to store the raw JSON, as normalization is non-trivial.
You lost me. If you have to build the index, normalization is immediately trivialized. This is just always a sloppy hack so this advice is just a big “temporary workaround” until normalizing for hot paths.
- jmt_ 5y agoTake various data collection analysis tasks as an example - there's many cases where you may not have a good idea of how everything fits together until you can run queries on the data, with JSON being a very common format. You many may need to combine many JSON datasets, with different layouts and potentially deeply nested structures, as well as more traditional tabular data. Being able to query all of this by putting everything into a temporary sqlite file is incredibly powerful. Sure, you're moving much more into SQL territory once you're creating indexes on JSON data as you say, but I'm not sure that always means "sloppy hack". For data tasks at least, being able to combine the power of relational + noSQL databases is incredibly productive. Doesn't mean I would necessarily put something like this into production, but for figuring stuff out combining the two approaches has a lot of merit in my experience.
- jitl 5y agoEDIT: see my comment here: https://news.ycombinator.com/item?id=31399397 https://news.ycombinator.com/item?id=31399397
- gfody 5y agonormalizing from raw json isn't terribly complicated but it's definitely more complicated than just indexing some paths into it. if the json is loosely defined normalization might not be practical (ie requiring too many tables or needing to be revisited too frequently as the json format evolves too often, etc, etc.)
- tracker1 5y agoThe example I like to give is different payment types... CC/Payflow, Paypal, Amazon Checkout, etc... all you really care about is the type and amount and success... The additional details, different transaction id's etc are different... this allows one to use a single field for the "extra" details, while normalizing a few bits that may come back from different services in a different way, but a single type of collective data. Normalizing the above would mean at least 4 tables, with subqueries and joins that really aren't valueable at the database layer.