3 ms·
Thank you! > What do you mean by this? Oracle. Oracle is trying to milk the cash cow. That's and unpredictable ride. > I occasionally contribute to Apache Ka
by blagie 4y ago
Thank you!
> What do you mean by this?
Oracle. Oracle is trying to milk the cash cow. That's and unpredictable ride.
> I occasionally contribute to Apache Kafka, any pointers on what documentation was lacking, and where you could have used more? I will see if I can contribute something around that :)
Well, if you're contributing, there are three Python interfaces to Kafka: confluent, pykafka, and python-kafka:
- All three have major gaps.
- None use async
Rallying around a single, usable interface would be a huge step. That's where I'd most encourage contribution.
But to see gaps in documentation, take any of the three, and try to fetch the most recent event (without blocking). You'll run into a bunch of holes along the way. I wanted a simple KVS in Kafka -- store with compact, and fetch most recent event. That would allow the system to run without a DB. That was not trivial to figure out.
- pixelmonkey 4y agoI just wanted to mention that pykafka is currently unmaintained and archived on GitHub: https://github.com/Parsely/pykafka https://github.com/Parsely/pykafka pykafka was originally developed and maintained by my team at Parse.ly, but we no longer maintain it. We instead encourage folks to use confluent-kafka-python, which is what we have ourselves switched to in our production systems: https://github.com/confluentinc/confluent-kafka-python https://github.com/confluentinc/confluent-kafka-python (pykafka was developed at a time before Confluent invested in their own Python binding. Some of the history of the project is described in this 2016 blog post[1] and our original 2015 announcement[2].) [1]: https://blog.parse.ly/pykafka-now/ https://blog.parse.ly/pykafka-now/ [2]: https://blog.parse.ly/announcing-pykafka-python-support-for-kafka-0-8-x/ https://blog.parse.ly/announcing-pykafka-python-support-for-...
- EdwardDiego 4y agoHmm, none of those are released by Apache Kafka - confluent-kafka-python is, well, from Confluent, and is based on librdkafka, python-kafka is owned by, I'm not sure who. What do you mean by "without blocking" though? You can set a poll timeout of 0ms, that should prevent blocking.