3 ms·

This post: https://databricks.com/blog/2015/07/30/diving-into-spark-streamings-execution-model.html https://databricks.com/blog/2015/07/30/diving-into-spark-str

by _laf 11y ago

This post: https://databricks.com/blog/2015/07/30/diving-into-spark-streamings-execution-model.html https://databricks.com/blog/2015/07/30/diving-into-spark-str... In the section titled "Future Directions for Spark Streaming" there is a paragraph about Event time and out-of-order data and Backpressure. This would blow my mind to be able to use; this is a real pain currently.

peterstjohn 11y ago
Backpressure is in 1.5+ by setting spark.streaming.backpressure.enabled=true (https://spark.apache.org/docs/latest/streaming-programming-guide.html#requirements https://spark.apache.org/docs/latest/streaming-programming-g...). Like you, I'm looking forward to the out-of-order data support.