6 ms·
To me, "warm" (not hot) data falls into one of two boxes: - schema on write: You know what future queries you will ask. Use SQL. - schema on read: You don't k
by anticristi 6y ago
To me, "warm" (not hot) data falls into one of two boxes:
- schema on write: You know what future queries you will ask. Use SQL.
- schema on read: You don't know what future queries you will ask. Use object storage.
Of course, in practice, part of your data will be schema on write and part on read.
- aidos 6y agoCan you explain this more? I’d argue that if you don’t know the patterns you’ll use to access your data, relational is the way to go. That’s the point, no? Flexibility over how you access your data. With an object store you mostly have to have decided upfront which data is collocated together.
- anticristi 6y agoMaybe I explained it badly. :) SQL essentially forces you to first define your data schema, then write data according to that schema. This works great for reading a user profile given an email; or read all comments of all users who are born before 1995. Since you have a schema on write, you can add indexes to optimise these queries. In contrast, if you need to do "anomaly detection" over an opaque dataset (where the definition of "anomaly" moves faster than you can rewrite your schemas), then it's better to just do a full dataset scan and create a schema on read. My 2 cents.
- aidos 6y agoStill not sure I buy it. I can extract a document representation from my relational data that matches your document store at read time. I can effectively choose my schema when I read, there’s no need to rewrite it at the storage layer.
- rwoerz 6y agoMaybe I do not get your point but I have a complete opposite opinion. If you do not know about your clients' queries, you are better off with SQL since it enables clients to query exactly the data they need. If you know about future queries, you can shape your object or document storage or db beforehand to match these queries.
- justsomeuser 6y agoI agree with you. SQL allows for a much larger set of possible unknown-at-design-time queries as the data is broken down into atomic units. With a document store, you often find yourself de-normalising and copying data to new collections on-write to deal with the limited query language. This means your writers often implicitly encode the possible queries. As an example, in Redis if you need a 3 way join/filter, you will either a) have your application read all the data and filter it or b) maintain the filtered collection via writers copying the data.