4 ms·
hadoop, kafka, storm, spark, flink, samza, confluent This tastes an aweful lot like javascript framework hell
by loremipsium 5y ago
hadoop, kafka, storm, spark, flink, samza, confluent
This tastes an aweful lot like javascript framework hell
- sivakon 5y agoYou forgot gearpump http://incubator.apache.org/projects/gearpump.html http://incubator.apache.org/projects/gearpump.html
- querulous 5y agoand beam: beam.apache.org
- ternaryoperator 5y agoIt appears gearpump was retired in 2018
- EdwardDiego 5y ago- Hadoop is an ecosystem. - Kafka is a distributed log. - Storm, Samza & Flink are stream processing engines. - Spark is a Map/Reduce framework that uses memory to cache computations to provide some performance increase over other disk-based frameworks. It can also do some streaming computations if you squint hard enough. - Confluent is a company that sells an enterprise Kafka. Not really sure the comparison you made is apt.
- deleted 5y ago[deleted]
- shipp02 5y agoKafka's homepage[1] advertises stream processing as a feature > Built-in Stream Processing > Process streams of events with joins, aggregations, filters, transformations, and more, using event-time and exactly-once processing. [1]:https://kafka.apache.org/ https://kafka.apache.org/
- fnord77 5y ago"kafka streams" is an add on product to kafka
- thinkharderdev 5y ago"Built-in" is a odd word choice there. Kafka Streams is a framework for building stream-processing applications on top of Kafka topics.
- EdwardDiego 5y agoKafka Streams is a streaming framework that uses Kafka's already existing features to implement itself - so resilience and parallelisation is implemented using consumer groups, exactly once using Kafka's transactions and idempotence, topics are used (as well as RocksDB) to store state for stateful aggregations etc. etc. So unlike Flink, Storm, Spark, Heron etc. it's only useful with Kafka.