3 ms·
Don't have much experience with Presto, but I have used Hive to query Elastic Search. It works very very well for full-table-scan analytics-type queries. I exp
by joefkelley 11y ago
Don't have much experience with Presto, but I have used Hive to query Elastic Search.
It works very very well for full-table-scan analytics-type queries. I expect PrestoDB would be similar. But when it comes to queries about smaller and smaller pieces of the full dataset, it becomes less and less likely that these types of connectors perform well. Predicate pushdown is rarely well-implemented in these types of "run SQL against any big data" systems (Hive, Presto, Impala, SparkSQL, etc). A simple "select * where id = 1234" will often do a full scan and filter within the query engine, rather than push the point lookup into ES.
- rxin 11y agoActually Spark SQL's data source API has a very expressive predicate pushdown interface and most data sources implement them. id = 1234 should not do a full scan.
- threeseed 11y agoAs mentioned Spark SQL has good predicate push down support. The ElasticSearch, Cassandra, HBase, MongoDB adapters all support it.