3 ms·

That's an interesting thought, but rather than focusing on write vs read balance, I would claim that OLAP and OLTP are most distinguished by the nature of queri

That's an interesting thought, but rather than focusing on write vs read balance, I would claim that OLAP and OLTP are most distinguished by the nature of queries they need to support. OLAP is characterized by fairly low-volume aggregation queries that touch very large volumes of data ("what fraction of tweets in English come from non-english speaking countries?"). Sure, the ingest is large, but it tends to be batch and has fairly loose real-time (in the EE sense) requirements. What makes OLAP hard is the sheer volume of data from which an aggregation must be calculated. OLTP is characterized by very selective queries with much tighter real-time bounds ("what are the last 20 things that the 100 people I follow said?"). The overall size of the dataset might even be the same, but each individual query needs fast access to a tiny fraction of the dataset. In many applications, this is accompanied by very high QPS and in Twitter's case, extremely high write volume.

jacques_chester 13y ago
Thanks, I hadn't mentally partitioned OLAP/OLTP that way. I still think that the read/write distinction more correctly forms the dividing line. That's because the classical OLTP approach is to use relational databases. Relational databases do well in writes because in a properly normalised DB there is one, and only one, place for each datum to go. There is no fan out and potentially no coordination. A place for everything and everything in its place. That reduces the amount of write traffic required to support complex models of the problem domain by a lot. But of course the relational view of a problem domain doesn't really look like the humanistic view of a problem domain. And building the humanistic view of the problem domain usually means denormalising and breaking the things that made OLTP useful. Enter ETL pipes and OLAP systems. From there it's very humanistic and it requires no understanding of the relational model. Tools can easily turn the dimensional tables into dropdown filter lists that look like a spreadsheet -- or even run inside a spreadsheet.

jacques_chester 13y ago

Thanks, I hadn't mentally partitioned OLAP/OLTP that way. I still think that the read/write distinction more correctly forms the dividing line. That's because the classical OLTP approach is to use relational databases. Relational databases do well in writes because in a properly normalised DB there is one, and only one, place for each datum to go. There is no fan out and potentially no coordination. A place for everything and everything in its place. That reduces the amount of write traffic required to support complex models of the problem domain by a lot. But of course the relational view of a problem domain doesn't really look like the humanistic view of a problem domain. And building the humanistic view of the problem domain usually means denormalising and breaking the things that made OLTP useful. Enter ETL pipes and OLAP systems. From there it's very humanistic and it requires no understanding of the relational model. Tools can easily turn the dimensional tables into dropdown filter lists that look like a spreadsheet -- or even run inside a spreadsheet.