4 ms·
It's also an argument against using queues in some cases. Instead of publishing data on a queue, you can make consumers of the data pull the data. Especially if
by jillesvangurp 7y ago
It's also an argument against using queues in some cases. Instead of publishing data on a queue, you can make consumers of the data pull the data. Especially if you have a 1 to many relation ship between producer and consumer of the data that moves the problem of ensuring data is received out of the infrastructure to the consumer side. Staying up to date is now their problem and they can ensure they are up to date by simply pulling regularly.
A good example would be the twitter firehose and consuming that. Other examples would be websites publishing RSS feeds. You can actually combine the two and push the information that "something changed" which would cause the receiver to pull. Caching and CDNs make publishing like this cheap and scalable.
- kqr 7y agoWell, what do the consumers pull from, if not a queue? What is an RSS feed if not a FIFO queue of published pages? Queues inherently lead to a pull-based module. The producer does not publish stuff directly to the consumer, but to the queue. The consumers then pull stuff at their own pace/terms from this queue.
- jillesvangurp 7y agoA rest API. An rss feed is a simple http GET that returns a document. You can cache it, put it in a CDN. There no queues involved here.
- elteto 7y agoCan you expand on this? So each worker gets its own REST endpoint to pull from? Because if all workers pull from a single endpoint then that endpoint is a queue, no?