5 ms·
Kafka isn’t a queue, it’s a distributed log. A partitioned topic can take very large volumes of message writes, persist them indefinitely, deliver them to any s
by Joeri 1y ago
Kafka isn’t a queue, it’s a distributed log. A partitioned topic can take very large volumes of message writes, persist them indefinitely, deliver them to any subscriber in-order and at-least-once (even for subscribers added after the message was published), and do all of that distributed and HA.
If you need all those things, there just are not a lot of options.
- holografix 1y agoFinally a valuable answer thank you
- diarrhea 1y agoPerhaps the best terse summary of Kafka I have come across yet.
- edem 1y agoWhat do you think about Temporal?
- cyberpunk 1y agoOkay for small numbers of high value jobs (e.g uber trips or food deliveries etc), prohibitively expensive for anything you need even a few k/sec of.
- HarHarVeryFunny 1y agoWhy do you say log rather than just publish and subscribe?
- exitb 1y agoClients don’t have to subscribe to latest messages, but rather can request any available offset range.
- halifaxbeard 1y agothe log stays on kafka for replay until your per log retention settings delete it
- HarHarVeryFunny 1y agoThe way people choose to use feedback on HN never fails to suprise me - we've got a generally intelligent user base here, but the most common type of feedback voting isn't because something is wrong but rather a childish "I don't like it - I want to suppress this comment". In this case it's something different - this was an honest question, and received two useful replies, so why downvote?! The mental model of people using Kafka is useful to know - in this case the published data being more log-like than stream-like since it's retained per a TTL policy, with each "subscriber" having their own controllable read index.
- MYEUHD 1y ago> Please don't comment about the voting on comments. It never does any good, and it makes boring reading. https://news.ycombinator.com/newsguidelines.html#comments https://news.ycombinator.com/newsguidelines.html#comments
- HarHarVeryFunny 1y agoSame goes for metacomments about voting.
- mannycalavera42 1y agoHERO
- iamcreasy 1y agoWhy the delineation? As long as you have a queue - you can publishing whatever you want in it, right? Be it log or something else.