2 ms·
Most column-oriented database architectures have a giant weakness that limits their utility, which was briefly mentioned at the end of the article: they offer p
by jandrewrogers 8y ago
Most column-oriented database architectures have a giant weakness that limits their utility, which was briefly mentioned at the end of the article: they offer poor performance for high-throughput mixed workloads, especially if there is a real-time component to it. Getting data into the system fast enough is a real problem. As more analytics become operational in nature, being able to scale online mixed workloads becomes the key feature. In practice, these still use vector-ish internal organization but the techniques differ considerably from many of the ones in the article, since online write performance is critical. Scalable online ingestion performance is a killer feature for many analytics systems but few database kernels are designed for it (I specialize in high-throughput mixed workload kernel design).
Also, time-series database implemented as you suggest will work in some cases, and better than some might expect if you are clever, with the caveat that it has significant weaknesses and scalability limitations. Many real-world time-series data models and workloads would run poorly on a database organized this way. Sorting or sharding on time in the data organization is usually a mistake in my opinion. Some high-performance time-series databases don't use time as a sort/shard key at all, instead leveraging the approximate partial order of the underlying data sources directly. This often offers orders of magnitude better write performance with little or no loss of query performance in practice and it generalizes better.
This is all a very complex but fantastically interesting technical design area. I firmly believe that scalable high-throughput mixed-workload analytical databases are possible, but they largely don't exist. Yet.
- manigandham 8y agoThey all have pretty fast ingest, and some can stream in data continuously at millions of rows/sec. MemSQL and SQL Server already support the HTAP combined use case with constant updates to both row and columnstore tables in the database while running large OLAP queries, so I'm not sure what you mean by the databases not existing yet - are you talking about something else? Yes, time as a secondary sort key works great. We have trillion row time-series tables with sub-second query times.
- jandrewrogers 8y agoI believe we are talking different scales, neither MemSQL nor SQL Server are designed for scale-out high-throughput mixed workloads for many practical definitions of "scale-out" and "high-throughput". First, online insertion (parsing, processing, indexing, disk storage) of millions of records/second should be per node, basically saturating the ingress path of each node's 10/25-GbE interface, and scale-out additively. You'll need the aggregate insertion throughput of several nodes for many live sensor data sources, never mind trying to load several petabytes of historical data in a reasonable amount of time. Second, by implication, the workload is far too large for an in-memory database -- those online indexes you are dynamically constructing need to be on disk and distributed. Indexing several columns through disk storage at wire speed under these parameters is not something either MemSQL or SQL Server (or any database really) is designed to do, which was my point. Being in-memory often has few implications for query performance in well-designed parallel database kernels; on modern hardware, storage I/O is rarely the throughput bottleneck for mixed workloads if you are doing everything else optimally. The core problem is that online disk-backed secondary index construction in a distributed environment that can use the full bandwidth of the switch fabric isn't feasible. A modern storage engine and scheduler will get you the raw throughput in most cases but that is a separate question from how do you support low-latency queries at the same time. Like I alluded to with time-series databases, the way you avoid the low throughput of index construction is by not constructing indexes -- other, sometimes exotic, selectivity mechanism exist that have orders of magnitude lower construction cost than a classic index while providing relatively efficient search. Even if those search mechanisms are slightly less selective than a traditional index, you make up for it with the intrinsically higher throughput of those mechanisms. re: time-series, I don't doubt that you can do sub-second queries on a time-series table with a trillion rows -- I've done it with tens of trillions of rows in a single table. Time-series is one of the easiest data models to scale if you have a modicum of cleverness. The challenge is ensuring this query performance while inserting tens of millions of records per second into the index at the same time and/or you need to index more dimensions than a timestamp.
- manigandham 8y agoYou have a fascinating comment history and I actually favorited your previous comments on SpaceCurve. It seems all I can find about this is from your blog - is there anywhere else to read how that database actually works with discrete topology? And is it only for geospatial?
- espeed 8y agoOn a related note, this upcoming paper on succinct graph representations will be presented at PACT next month (http://www.pactconf.org http://www.pactconf.org) -- it's more data model than database, but I think you'll find it interesting... Log(Graph): A Near-Optimal High-Performance Graph Representation [pdf] https://people.csail.mit.edu/jshun/papers/loggraph.pdf https://people.csail.mit.edu/jshun/papers/loggraph.pdf For something more database specific, check out the new RedisGraph module [1] implemented with GraphBLAS [2] by the venerable Tim Davis [3] himself, who as you know implements the underlying sparse matrix algos used in everything from MATLAB to Google Maps... RedisGraph in the Language of Linear Algebra with GraphBLAS, co-presented by RedisLabs and Tim Davis [video] https://www.youtube.com/watch?v=xnez6tloNSQ https://www.youtube.com/watch?v=xnez6tloNSQ GraphBLAS is not Redis-specific, but the Redis data structures and new modular design made it an ideal candidate for the first popular database implementation. This is significant because GraphBLAS is more than 10 years in the making, the culmination of the initial D4M matrix model design by Jeremy Kepner [4] and his team at MIT Lincoln Laboratory Supercomputing Center. And for the last ~5 years or so the software model has been designed in collaboration with hardware teams at Intel, NVIDIA, IBM, and the labs [5] to make chips and architectures optimized for these new matrix models and capable of exascale. It all came together this summer with the release of GraphBLAS 1.0 [6]. Now that GPU and TPU accelerators are populating the data centers and linear algebra has come en vogue, hopefully enough software engineers will be ready with the background understanding and a working mental model for the underpinning architectural paradigm shift [7]. [1] RedisGraph https://oss.redislabs.com/redisgraph/ https://oss.redislabs.com/redisgraph/ [2] GraphBLAS http://graphblas.org http://graphblas.org [3] Tim Davis http://faculty.cse.tamu.edu/davis/ http://faculty.cse.tamu.edu/davis/ [4] Jeremy Kepner http://www.mit.edu/~kepner/ http://www.mit.edu/~kepner/ [5] GraphBLAS: Building Blocks For High Performance Graph Analytics https://crd.lbl.gov/news-and-publications/news/2017/graphblas-building-blocks-for-high-performance-graph-analytics/ https://crd.lbl.gov/news-and-publications/news/2017/graphbla... [6] Graph algorithms via SuiteSparse:GraphBLAS: triangle counting and K-truss [pdf] http://faculty.cse.tamu.edu/davis/GraphBLAS/HPEC18/Davis_HPEC18.pdf http://faculty.cse.tamu.edu/davis/GraphBLAS/HPEC18/Davis_HPE... [7] David Patterson Says It’s Time for New Computer Architectures and Languages https://news.ycombinator.com/item?id=18009581 https://news.ycombinator.com/item?id=18009581
- 8y ago