5 ms·

I understand where you're coming from, but what you're describing is learning how to use a queue to maintain consistency guarantees across a distributed system.

I understand where you're coming from, but what you're describing is learning how to use a queue to maintain consistency guarantees across a distributed system. You can get something simple like AWS SQS working with a few clicks, but then you don't have any of those consistency guarantees.

gtaylor 6y ago
Depending on your usage patterns, SQS can be significantly more expensive, too.
tracker1 6y ago
If you don't need crazy throughput, I find that Azure Storage Queues are crazy easy, built in retry and just simple as can be. Though when I've used it in the past, I've created a slightly simpler to use abstraction. https://docs.microsoft.com/en-us/azure/storage/queues/storage-nodejs-how-to-use-queues https://docs.microsoft.com/en-us/azure/storage/queues/storag... Thinking of doing something that works like an async generator so I can just use it like... const work = queue.subscribe('somequeue'); for await (const {item, done} in work) { // do something with JSON.parsed item from message await done(); // wrapper for the delete/finish }
- AmericanChopper 6y ago
  Azure Storage Queues are about on par with SQS. That is, easy to use, but lacking strict concurrency control. If you need that level of concurrency control (and stricter serialization), then you’d be better off their (more complicated) service bus product [0]. RabbitMQ isn’t more complicated because it’s been improperly designed. It’s more complicated because it’s doing a much more complicated task. A fair amount of that complexity lies in the hosting, so a managed service can take some of that off your hands (for an increased price obviously), but part of it is necessarily going to lie with the message consumer (your application logic). If your use case doesn’t need that level of control, then it doesn’t need that level of complexity either, so something like Rabbit would just be the wrong choice. [0]: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted https://docs.microsoft.com/en-us/azure/service-bus-messaging...

gtaylor 6y ago

Depending on your usage patterns, SQS can be significantly more expensive, too.

tracker1 6y ago

If you don't need crazy throughput, I find that Azure Storage Queues are crazy easy, built in retry and just simple as can be. Though when I've used it in the past, I've created a slightly simpler to use abstraction. https://docs.microsoft.com/en-us/azure/storage/queues/storage-nodejs-how-to-use-queues https://docs.microsoft.com/en-us/azure/storage/queues/storag... Thinking of doing something that works like an async generator so I can just use it like... const work = queue.subscribe('somequeue'); for await (const {item, done} in work) { // do something with JSON.parsed item from message await done(); // wrapper for the delete/finish }

AmericanChopper 6y ago
Azure Storage Queues are about on par with SQS. That is, easy to use, but lacking strict concurrency control. If you need that level of concurrency control (and stricter serialization), then you’d be better off their (more complicated) service bus product [0]. RabbitMQ isn’t more complicated because it’s been improperly designed. It’s more complicated because it’s doing a much more complicated task. A fair amount of that complexity lies in the hosting, so a managed service can take some of that off your hands (for an increased price obviously), but part of it is necessarily going to lie with the message consumer (your application logic). If your use case doesn’t need that level of control, then it doesn’t need that level of complexity either, so something like Rabbit would just be the wrong choice. [0]: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted https://docs.microsoft.com/en-us/azure/service-bus-messaging...