3 ms·
Greenplum is designed for datawarehousing. The analytics it is built for aren't the kind that ElasticSearch is typically used for. Greenplum/datawarehouse sty
by nl 11y ago
Greenplum is designed for datawarehousing.
The analytics it is built for aren't the kind that ElasticSearch is typically used for.
Greenplum/datawarehouse style analytical are things like: What is the average spend of female customers with 2 children who live in postcode AAAA or BBBB. Break it down by day of week and group by their marital status. Now for the top cohort of buyers on Mondays, give me a breakdown of the 3 most popular products and our profit margin on each one. This is often called "Business Analytics" (BI)
You could make ElasticSearch do that of course, but it wouldn't be much fun.
Given that context, the single master limitation turns out not be be a huge problem. Typically Greenplum is setup to load data from the OLAP-style online system, and the amount of data loaded is very predictable.
Same with things like hot standby and replication. In the Datawarehouse world it isn't uncommon for there to be nightly periods where a batch dataload occurs and/or nightly reporting is done, and the availability of external interfaces to the DB during that time maybe restricted.
- lobster_johnson 11y agoThanks, that makes a lot of sense. We have those types of queries, too, which is where ES definitely breaks down. We wouldn't mind switching those parts (the "BI") of the analytics into something with higher latency.
- threeseed 11y agoOne option there is to use Spark. You can then write SQL, Scala, Python, R to interact with ElasticSearch. I can't recall the performance but against Cassandra, HDFS, HBase, MongoDB etc. it is very fast.
- blumkvist 11y agoHey, forgive my ignorance, I'm the guy who uses this stuff, not the guy who builds it. My understanding was that, typically, the DW is where the data gets ETLed from various places and lives in a normalized environment. Then from that big pool, departments get built columnar or OLAP databases for specific analyses they want to do, and I connect not to the DW but to the OLAP instance (we call it Datamart here). Was I living a lie? At first I thought that your master-segment system is the same, but then I read your comment. I looked into Greenplum a while back and wanted it because of Madlib, but my guys said "nope" and went with MS.
- chris_wot 11y agoIt can be done with a dimensional scheme or with a normalised schema. Ralph Kimble argues for the dimensional approach whilst Bill Imon argues for the normalised approach. For myself, I tend to go with Kimble, but there are advantages and disadvantages to both approaches. Wikipedia has a decent summary of this here: https://en.m.wikipedia.org/wiki/Data_warehouse#Dimensional_vs._normalized_approach_for_storage_of_data https://en.m.wikipedia.org/wiki/Data_warehouse#Dimensional_v...