3 ms·
Yes. For simple setups, I'm usually biased towards having less moving parts. Queues sound nice until you start thinking about the edge cases with respect to err
by jillesvangurp 8y ago
Yes. For simple setups, I'm usually biased towards having less moving parts. Queues sound nice until you start thinking about the edge cases with respect to errors, lost messages, rolling restarts, etc. In short, pretty much every system with queues I've ever seen this was a headache. There are good solutions for this of course but they require extra work like checking dead letter queues, devops related to monitoring of and managing the queues and depending systems, etc. If find lots of teams cut corners here and then have to deal with the inevitable fallout of things going wrong.
An alternative that is usually simple to implement is some kind of feed based system that you poll periodically. This can be dynamic or even static files. Feeds may be cached. Depending systems can simply poll this. If they go down, they can catch up when they come back. If the feed system goes down, they will catch up when it comes back up. It also scales to large numbers of subscribers. You can do streaming or non streaming variants of this. You can even poll from a browser.
- kbouck 8y ago> Feeds may be cached. Depending systems can simply poll this. If they go down, they can catch up when they come back. This is how Kafka works. The "feed" of messages is stored as a log. Clients maintain a reference to their latest position in the log, and pick up where they left off upon reconnecting.