4 ms·

A controversial, but a very pragmatic take. Queues are great for semi-infinite scalability, but you rarely need it. There's numerous subtle benefits to using

by hbrn 4y ago

A controversial, but a very pragmatic take. Queues are great for semi-infinite scalability, but you rarely need it. There's numerous subtle benefits to using db compared to regular message queues that are often overlooked. Being able to delete, reorder, or edit specific messages can be a lifesaver when things go wrong. Most common issue with queue-based systems is getting overwhelmed with messages. Either your consumers went down. Or your producers had a bug / executed too frequently. Being able to recover with a simple SQL query is a blessing.

hbrn 4y ago
Here's just a few examples from my experience: 1. Huge number of messages from test system were accidentally inserted into production. Queue solution: disable consumers, move messages to a temporary queue while filtering them, move messages back to the old queue, enable consumers DB solution: just delete rogue messages 2. We want to store some of the messages, but we're not ready to process them yet Queue solution: create a separate queue for each message type, insert messages to different queues, manually move them when you're ready to process them (and keep in mind that not every queue can persist messages forever, SQS for example can't hold messages longer than 14 days) DB solution: just skip those messages while processing 3. Consumers went down and the queue now contains a big number of duplicate messages. While it was fine to just wait a couple hours to let it stabilize, a whale customer started complaining Queue solution: none (any hacky solution would take longer than it takes for the system to naturally stabilize) DB solution: move whale customer messages to the front of the queue
- EdwardDiego 4y ago
  1. Kafka isn't a message queue, but advance consumer offsets past bad data. Hit person who inserted test data into prod with a cricket bat and then hit person who designed a system where they could do that with the same cricket bat, but harder. 2. Kafka - do what you did with your DB consumers. 3. Kafka - consumers going down can't cause duplicates, what's even going on with your queue?
  - hbrn 4y ago
    > what's even going on with your queue? A message with the latest state of an object gets pushed into queue. When noone is consuming those, you essentially get duplicates: queue contains all of the updates, but you only care about the most recent one.
whalesalad 4y ago
Throughput is often where an RDBMS falls over here. Unless you have partitions/isolation around your high throughput workloads (or throw tons of horsepower/$$$ at it) you can’t achieve the same ops/sec as a dedicated queueing mechanism. But you do have a lot of valid points, particularly about failure recovery. My approach tends to involve a hybrid of the two if high speed is a necessary component somewhere. The system can restore its runtime state from a db, but the active mechanics of it are running on message passing of some kind.

by hbrn 4y ago

hbrn 4y ago

Here's just a few examples from my experience: 1. Huge number of messages from test system were accidentally inserted into production. Queue solution: disable consumers, move messages to a temporary queue while filtering them, move messages back to the old queue, enable consumers DB solution: just delete rogue messages 2. We want to store some of the messages, but we're not ready to process them yet Queue solution: create a separate queue for each message type, insert messages to different queues, manually move them when you're ready to process them (and keep in mind that not every queue can persist messages forever, SQS for example can't hold messages longer than 14 days) DB solution: just skip those messages while processing 3. Consumers went down and the queue now contains a big number of duplicate messages. While it was fine to just wait a couple hours to let it stabilize, a whale customer started complaining Queue solution: none (any hacky solution would take longer than it takes for the system to naturally stabilize) DB solution: move whale customer messages to the front of the queue

EdwardDiego 4y ago
1. Kafka isn't a message queue, but advance consumer offsets past bad data. Hit person who inserted test data into prod with a cricket bat and then hit person who designed a system where they could do that with the same cricket bat, but harder. 2. Kafka - do what you did with your DB consumers. 3. Kafka - consumers going down can't cause duplicates, what's even going on with your queue?
- hbrn 4y ago
  > what's even going on with your queue? A message with the latest state of an object gets pushed into queue. When noone is consuming those, you essentially get duplicates: queue contains all of the updates, but you only care about the most recent one.

whalesalad 4y ago

Throughput is often where an RDBMS falls over here. Unless you have partitions/isolation around your high throughput workloads (or throw tons of horsepower/$$$ at it) you can’t achieve the same ops/sec as a dedicated queueing mechanism. But you do have a lot of valid points, particularly about failure recovery. My approach tends to involve a hybrid of the two if high speed is a necessary component somewhere. The system can restore its runtime state from a db, but the active mechanics of it are running on message passing of some kind.