3 ms·
Druid's main value add to the data infrastructure space is around power user-facing data applications at scale. The queries it is best at are OLAP/business inte
by fangjin 11y ago
Druid's main value add to the data infrastructure space is around power user-facing data applications at scale. The queries it is best at are OLAP/business intelligence style queries. It isn't really designed to be a general processing tool such as Hadoop or Spark. The open source data space is very complex, and there are many different solutions targeted towards many different use cases. Druid is better than other solutions at some of these use cases, and worse than other solutions at others.
I wrote my interpretation of the current open source data landscape here for anyone interested: http://imply.io/post/2015/11/04/big-data-zoo.html http://imply.io/post/2015/11/04/big-data-zoo.html
- deepanchor 11y agoAs someone who hasn't yet had the opportunity to use many of these systems, this was a great high-level overview of how the various systems fit together. Thanks for writing it!
- scott_s 11y agoYour Wikipedia link to "stream processors" is for the wrong kind of stream processors. For a decades, the digital signal processing and graphics worlds have used the streaming abstraction to design programs and hardware. Their applications are typically expressed in pipelines, and do have to continually process data. "Big data" stream processing is obviously related as its a dataflow programming model, but it's still very different in practice. The streaming abstraction is generally more free-form, and not realized directly in hardware. I contrast the two kinds of streaming in Section 2 of a paper from a few years ago: http://www.scott-a-s.com/files/pact2012.pdf http://www.scott-a-s.com/files/pact2012.pdf