3 ms·

Started using Spark 1.6 a few months ago. Excited for the Kafka Connector feature.

• mydpy 11y ago
  Me too. Matei said that Spark 2.0 should be released (stable) late-April or early May. But you can always download and build the source!
• kod 11y ago
  What exactly was the mention of Kafka referring to? Spark has had decent kafka integration for a while now.
  • peterstjohn 11y ago
    I think it's support for Kafka Connect, new in Kafka 0.9: http://kafka.apache.org/090/documentation.html#connect http://kafka.apache.org/090/documentation.html#connect
    • kod 11y ago
      It doesn't have anything to do with Kafka Connect. Matei was actually talking about the existing Spark Kafka direct stream implementation, which has been available since Spark 1.3 The video of the talk is available here: http://livestream.com/fourstream/sparksummiteast2016-tracka/videos/112612459 http://livestream.com/fourstream/sparksummiteast2016-tracka/...
      • peterstjohn 11y ago
        Ah, thanks! I have unfortunately been too busy to watch the streams this week, and assumed it was Connect because I think Confluent is/was doing a talk with Kafka Connect and Spark at the summit.