I never fully understood the need for back end message queues TBH. You can just poll the database or data store every few seconds and process tasks in batches..

I never fully understood the need for back end message queues TBH. You can just poll the database or data store every few seconds and process tasks in batches... IMO, the 'real time' aspect was only ever useful for front end use cases for performance reasons since short polling every second with HTTP (with all its headers/overheads) is prohibitively expensive. Also, HTTP long polling introduces some architectural complexity which is not worth it (e.g. sticky sessions are required when you have multiple app servers). Unfortunately, moving real-time messaging complexity entirely to the back end has been the norm for a very long time. My experience is that, in general, it makes the architecture way more difficult to manage. I've been promoting end-to-end pub/sub as an alternative for over a decade (see https://socketcluster.io/ https://socketcluster.io/) but, although I've been getting great feedback, this approach has never managed to spread beyond a certain niche. I think it's partly because most devs just don't realize how much complexity is added by micromanaging message queues on the back end and figuring out which message belongs to what client socket instead of letting the clients themselves decide what channels to subscribe to directly from the front end (and the back end only focuses on access control). I think part of the problem was the separation between front end and back end developer responsibilities. Back end developers like to ignore the front end as much as possible; when it comes to architecture, their thinking rarely extends beyond the API endpoint definition; gains which can be made from better integrating the back end with the front end is 'not their job'. From the perspective of front-end developers, they see anything performance-related or related to 'architectural simplicity' as 'not their job' either... There weren't enough full stack developers with the required insights to push for integration efficiency/simplicity.