> Queues like Kafka Kafka is a distributed log. A sibling comment to this mentions this as well, but I'm repeating it because I actually think that's were most

> Queues like Kafka Kafka is a distributed log. A sibling comment to this mentions this as well, but I'm repeating it because I actually think that's were most people using kafka go wrong. You _can_ use kafka as a queue, but it'll would feel like using a drill to hammer a nail, would kind of work but would make you think the tool just sucks. > The justification is usually a sort of handwave about scaling, but I tend to be skeptical of that argument. Kafka does help with scaling in so far as you can independently scale the number of producers and consumers but I find kafka the most useful in its ability to allow multiple consumers to process the same message(s) at different rates. For example let's say you have an app that's producing messages like "{foo: 1, bar: 2 }" and you need to count how many foos there are and how many bars there are. You could queue the messages, then have 1 service that does both, or you could use kafka and have a FooCounter and BarCounter service. They each process the same message(s) but ignore the part of the payload they don't care about. While this is really trivial example, I've used, and have seen this pattern used for processing actual logs for example.