4 ms·
Not an answer, just wondering if anyone is using Apache Spark for ETL? Is this approach viable or if it is just marketing: https://databricks.com/blog/2017/01
by heliosAtwork 9y ago
Not an answer, just wondering if anyone is using Apache Spark for ETL?
Is this approach viable or if it is just marketing:
https://databricks.com/blog/2017/01/19/real-time-streaming-etl-structured-streaming-apache-spark-2-1.html https://databricks.com/blog/2017/01/19/real-time-streaming-e...
- thisone 9y agoI use spark for batch data transforms, our data is structurally complicated, but not for streaming. I attended the EU Spark Summit last year, as of then Spark streaming wasn't prime-time ready, I don't know if anything has changed in the last 12 months on the streaming front. You should have a look at Flink if you're looking into streaming transforms.