3 ms·

One is a tomato, the other is an orange. From a distance they might look alike but they really are two completely different tools. This is a pretty solid explan

One is a tomato, the other is an orange. From a distance they might look alike but they really are two completely different tools. This is a pretty solid explanation of the differences with good illustrations. Rabbit can do everything Kafka does - and much more - in a more configurable manner. Kafka is highly optimized for essentially one use case and does that well. Nothing in life is free, there are trade-offs everywhere. I am not privy to which one is theoretically faster - but once you reach that question methinks the particular workload is the deciding factor.

JohnMakin 3y ago
I am not an expert in either and have only worked with Kafka. At a past job I had to write a connector job to parse and sanitize some extremely dirty, unstructured data and pass it along somewhere else. RabbitMQ supports this? What is the one use case of kafka? I think you have it backwards.
- whalesalad 3y ago
  > parse and sanitize some extremely dirty, unstructured data and pass it along somewhere else can you be more specific? that to me sounds like hello world for either of these tools. "santize data" is an application level concern that neither rabbit or kafka would handle. as far as "pass along somewhere else" again both tools can do.
  - JohnMakin 3y ago
    It was a Sink Connector. I don’t know what it was or wasn’t supposed to do but I was asked to do it, as is often the case in tech. I could have done any number of transformations in that process though, which I’m not sure rabbitmq supports
    - whalesalad 3y ago
      It sounds to me like you aren't really even sure what you built. I have operated both rabbit and kafka at scale I definitely do not have it backwards :)
      - JohnMakin 3y ago
        No, I’m not, because it was years ago, and I’m asking for clarification because what was said immediately sounded wrong to me (I’ve managed a lot of rabbitmq deployments) and you’ve not really given one other than an appeal to authority. guess I have my answer. Can’t find anything that suggests rabbitmq natively supports anything like sink connectors. thanks.
        NovemberWhiskey 3y ago
        > I am not an expert in either and have only worked with Kafka. > I’ve managed a lot of rabbitmq deployments ... ?
        JohnMakin 3y ago
        You do not need to be an expert in something's internal workings to manage/monitor a deployment. Surely this does not need to be explained further.
        sgarland 3y ago
        > I think you have it backwards You do need to be an expert when you start telling other people they’re wrong, though.
        lnenad 3y ago
        A classic exchange on the internet.
        FINDarkside 3y ago
        So let me get this straight. You've used Kafka once, RabbitMQ never. You don't really know what you did with Kafka. But you somehow know that RabbitMQ cannot do the thing which you don't really remember anymore. Doesn't make much sense to be honest. Nobody can really have any sources for RabbitMQ being able to do it if you don't know what it supposedly cannot do. The way you descibed it, is that you simply read data and then did something with the data and passed it to somewhere else. RabbitMQ obviously can do it.
KaiserPro 3y ago
Rabbit is an arse to scale past one broker. It was possible, but a pain, that might have changed. Kafka is just a pain full stop.
- nhumrich 3y ago
  The official rabbitmq controller for kubernetes is a breeze. Scales wonderfully without almost any effort.
- icedchai 3y ago
  At a previous company, about 10 years ago, we had roughly 10 RabbitMQ instances (brokers), all isolated. The system was essentially partitioned by queue server. We had a directory-ish service that would associate clients with their assigned node. It worked well, except if a client got too large we might have to move them to another queue.
relay23 3y ago
>> Rabbit can do everything Kafka does - and much more - in a more configurable manner. Sure, if you're doing like 10's of MB/s. RMQ is fast compared to AK if you're not adding durability, persistence, etc. Try to run gigabytes per second through it though, or stretch across regions, or meet RTO when the broker gets overloaded and crashes.. Get your shovel ready! ;) Kafka itself is dumb but scalable and resilient, it's the client ecosystem that's massive compared to RabbitMQ. Count 10 stream processing, connectivity, ingestion or log harvesting platforms that use RMQ as it's backend, then name 10 languages that have supported libraries for RMQ.. then compare that to Kafka.
- deleted 3y ago
  [deleted]

JohnMakin 3y ago

I am not an expert in either and have only worked with Kafka. At a past job I had to write a connector job to parse and sanitize some extremely dirty, unstructured data and pass it along somewhere else. RabbitMQ supports this? What is the one use case of kafka? I think you have it backwards.

whalesalad 3y ago
> parse and sanitize some extremely dirty, unstructured data and pass it along somewhere else can you be more specific? that to me sounds like hello world for either of these tools. "santize data" is an application level concern that neither rabbit or kafka would handle. as far as "pass along somewhere else" again both tools can do.
- JohnMakin 3y ago
  It was a Sink Connector. I don’t know what it was or wasn’t supposed to do but I was asked to do it, as is often the case in tech. I could have done any number of transformations in that process though, which I’m not sure rabbitmq supports
  - whalesalad 3y ago
    It sounds to me like you aren't really even sure what you built. I have operated both rabbit and kafka at scale I definitely do not have it backwards :)
    - JohnMakin 3y ago
      No, I’m not, because it was years ago, and I’m asking for clarification because what was said immediately sounded wrong to me (I’ve managed a lot of rabbitmq deployments) and you’ve not really given one other than an appeal to authority. guess I have my answer. Can’t find anything that suggests rabbitmq natively supports anything like sink connectors. thanks.
      - NovemberWhiskey 3y ago
        > I am not an expert in either and have only worked with Kafka. > I’ve managed a lot of rabbitmq deployments ... ?
        JohnMakin 3y ago
        You do not need to be an expert in something's internal workings to manage/monitor a deployment. Surely this does not need to be explained further.
        sgarland 3y ago
        > I think you have it backwards You do need to be an expert when you start telling other people they’re wrong, though.
        lnenad 3y ago
        A classic exchange on the internet.
      - FINDarkside 3y ago
        So let me get this straight. You've used Kafka once, RabbitMQ never. You don't really know what you did with Kafka. But you somehow know that RabbitMQ cannot do the thing which you don't really remember anymore. Doesn't make much sense to be honest. Nobody can really have any sources for RabbitMQ being able to do it if you don't know what it supposedly cannot do. The way you descibed it, is that you simply read data and then did something with the data and passed it to somewhere else. RabbitMQ obviously can do it.

KaiserPro 3y ago

Rabbit is an arse to scale past one broker. It was possible, but a pain, that might have changed. Kafka is just a pain full stop.

nhumrich 3y ago
The official rabbitmq controller for kubernetes is a breeze. Scales wonderfully without almost any effort.
icedchai 3y ago
At a previous company, about 10 years ago, we had roughly 10 RabbitMQ instances (brokers), all isolated. The system was essentially partitioned by queue server. We had a directory-ish service that would associate clients with their assigned node. It worked well, except if a client got too large we might have to move them to another queue.

relay23 3y ago

>> Rabbit can do everything Kafka does - and much more - in a more configurable manner. Sure, if you're doing like 10's of MB/s. RMQ is fast compared to AK if you're not adding durability, persistence, etc. Try to run gigabytes per second through it though, or stretch across regions, or meet RTO when the broker gets overloaded and crashes.. Get your shovel ready! ;) Kafka itself is dumb but scalable and resilient, it's the client ecosystem that's massive compared to RabbitMQ. Count 10 stream processing, connectivity, ingestion or log harvesting platforms that use RMQ as it's backend, then name 10 languages that have supported libraries for RMQ.. then compare that to Kafka.

deleted 3y ago
[deleted]