3 ms·
Why not something like Kafka or Redis?
by phibz 2y ago
Why not something like Kafka or Redis?
- hipadev23 2y agoBecause that’s sane, easy, and boring.
- deleted 2y ago[deleted]
- dewey 2y agoAlso most of the times you already have a database so why not use that instead of adding another service to the pile.
- teraflop 2y agoThe most straightforward reason is that if you need a transactional database anyway, then moving the queue into the DB allows you to atomically en/dequeue messages at the same time as making other updates. Which can massively simplify your architecture because it eliminates an enormous category of possible failure modes. (Or it can massively improve your system correctness, if you didn't realize those failure modes were possible.)
- klysm 2y agoUsing Kafka as a work queue is widely documented as a mistake. Using a single database results in a lot of operational simplicity and you get to skip a lot of distributed systems when all your state is in a single system
- victor106 2y ago> Using Kafka as a work queue is widely documented as a mistake Could not find any good sources on this. Can you please provide any references?
- klysm 2y agodon’t have any off the top of my head sorry, I could be making it up but I’m pretty sure I remember reading that from several sources. Iirc it has to do with the fact that there isn’t really single message acknowledging?
- perfectspiral 2y agoThis is a proposal to extend Kafka to (better) handle queueing use cases: https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A... "For example, a queue is perfect for a situation in which messages are independent work items that can be processed concurrently by a pool of applications, and individually retried or acknowledged as processing completes. This is much easier to achieve using a queue rather than a partitioned topic with a consumer group."