10 ms·

The only thing that might take "weeks" is procrastination. Presuming absolutely no background other than general data engineering, a decent beginner online cour

The only thing that might take "weeks" is procrastination. Presuming absolutely no background other than general data engineering, a decent beginner online course in Kafka (or Redpanda) will run about 1-2 hours. You should be able to install within minutes.

kragen 11mo ago
I mean, setting up Zookeeper, tweaking the kernel settings, configuring the hardware, the kind of stuff mentioned in guides like https://medium.com/@ankurrana/things-nobody-will-tell-you-setting-up-a-kafka-cluster-3a7a7fd1c92d https://medium.com/@ankurrana/things-nobody-will-tell-you-se... and https://dungeonengineering.com/the-kafkaesque-nightmare-of-modern-message-queues/ https://dungeonengineering.com/the-kafkaesque-nightmare-of-m.... Apparently you can do without Zookeeper now, but that's another choice to make, possibly doing careful experiments with both choices to see what's better. Much more discussion in https://news.ycombinator.com/item?id=37036291 https://news.ycombinator.com/item?id=37036291. None of this applies to Redpanda.
- PeterCorless 11mo ago
  True. Redpanda does not use Zookeeper. Yet to also be fair to the Kafka folks, Zookeeper is no longer default and hasn't been since April 2025 with the release of Apache Kafka 4.0: "Kafka 4.0's completed transition to KRaft eliminates ZooKeeper (KIP-500), making clusters easier to operate at any scale." Source: https://developer.confluent.io/newsletter/introducing-apache-kafka-40 https://developer.confluent.io/newsletter/introducing-apache...
  - kragen 11mo ago
    Right, I was talking about installing Kafka, not installing Redpanda. Redpanda may be perfectly fine software, but bringing it up in that context is a bit apples-and-oranges since it's not open-source: https://news.ycombinator.com/item?id=45748426 https://news.ycombinator.com/item?id=45748426
  - EdwardDiego 11mo ago
    Good on you for being fair in this discussion :)

kragen 11mo ago

I mean, setting up Zookeeper, tweaking the kernel settings, configuring the hardware, the kind of stuff mentioned in guides like https://medium.com/@ankurrana/things-nobody-will-tell-you-setting-up-a-kafka-cluster-3a7a7fd1c92d https://medium.com/@ankurrana/things-nobody-will-tell-you-se... and https://dungeonengineering.com/the-kafkaesque-nightmare-of-modern-message-queues/ https://dungeonengineering.com/the-kafkaesque-nightmare-of-m.... Apparently you can do without Zookeeper now, but that's another choice to make, possibly doing careful experiments with both choices to see what's better. Much more discussion in https://news.ycombinator.com/item?id=37036291 https://news.ycombinator.com/item?id=37036291. None of this applies to Redpanda.

PeterCorless 11mo ago
True. Redpanda does not use Zookeeper. Yet to also be fair to the Kafka folks, Zookeeper is no longer default and hasn't been since April 2025 with the release of Apache Kafka 4.0: "Kafka 4.0's completed transition to KRaft eliminates ZooKeeper (KIP-500), making clusters easier to operate at any scale." Source: https://developer.confluent.io/newsletter/introducing-apache-kafka-40 https://developer.confluent.io/newsletter/introducing-apache...
- kragen 11mo ago
  Right, I was talking about installing Kafka, not installing Redpanda. Redpanda may be perfectly fine software, but bringing it up in that context is a bit apples-and-oranges since it's not open-source: https://news.ycombinator.com/item?id=45748426 https://news.ycombinator.com/item?id=45748426
- EdwardDiego 11mo ago
  Good on you for being fair in this discussion :)