4 ms·

> I enjoy knowing a little bit of queue theory because it is a subject where not only can you gain an advantage in many areas but few people will be able to fig

by jw1224 3y ago

> I enjoy knowing a little bit of queue theory because it is a subject where not only can you gain an advantage in many areas but few people will be able to figure out why Yes! I’ve been building web applications for 15 years, but queuing systems were rarely discussed with any importance. I only picked up on it in the past few years, but since then, I consider it one of the most commercially-beneficial ways to scale and grow most kinds of software. When it comes to scaling, it seems like 90% of the advice just focusses on RDMS, optimising indexes, etc. Database theory is vitally important to understand, but I now consider queue theory essential knowledge for modern application developers too. Case in point: over the years I’ve reviewed thousands of PHP projects, articles, blogs, and other resources in the ecosystem. There are more guides to scaling Nginx or optimising MariaDB than I could ever need. But I simply never see any discussion or usage of the low-level queue features offered by the language, like Deques: https://www.php.net/manual/en/class.ds-deque.php https://www.php.net/manual/en/class.ds-deque.php I appreciate these have niche use cases, but a background queue handler builds on this kind of understanding, and is general purpose. Knowledge of queue theory directly helped me scale my own projects, in ways I couldn’t have imagined otherwise.

idoubtit 3y ago
> But I simply never see any discussion or usage of the low-level queue features offered by the language, like Deques I respectfully disagree: 1. Deques is not a "feature[s] offered by the language". Its part of an external extension (meaning it's a C compiled plug-in to the main engine). It's development is outside of the language scope, see https://pecl.php.net/package/ds https://pecl.php.net/package/ds 2. Deques is barely a "low-level queue feature". It's just a data structure that can be performant for some implementations of queues. Native arrays provide slower but similar features. 3. It's sane that this (Deques) implementation detail (the in-memory data structure) is less discussed than more central and generic subjects, like scaling MariaDB. If I consider the first two problems mentioned in the article (systems for issues and for kanban), I think Deques is unsuitable to them, because it's meant to replace arrays for in-memory works. The problems require a persistent storage, and the optimization of a PHP storage for the queue is probably irrelevant.
- jw1224 3y ago
  You're totally right and I agree too :) I forgot Deques (and DS) were via PECL. It's a poor example to give in relation to the article's real-world cases. Nonetheless, it is relevant to queue theory, and knowledge of how, why, and when these data structures can be more optimal than simple arrays has helped me more broadly in the past. A much more real-world example would be Laravel Horizon, which uses Redis queues to build scalable distributed systems very easily. The last time I set it up I think it took me 5 minutes at most... https://laravel.com/docs/10.x/horizon https://laravel.com/docs/10.x/horizon
kiitos 3y ago
> But I simply never see any discussion or usage of the low-level queue features offered by the language... Assuming we're talking about something that serves requests over the network, it rarely makes sense to define or use queues within the program itself. The network stack already contains many layers of queues which will almost always manage load concerns (e.g. throttling, backpressure, etc.) more effectively than your application can do itself. In general you want your app to accept and process requests as fast as it can, and leave queueing concerns to other authorities.

by jw1224 3y ago

idoubtit 3y ago

> But I simply never see any discussion or usage of the low-level queue features offered by the language, like Deques I respectfully disagree: 1. Deques is not a "feature[s] offered by the language". Its part of an external extension (meaning it's a C compiled plug-in to the main engine). It's development is outside of the language scope, see https://pecl.php.net/package/ds https://pecl.php.net/package/ds 2. Deques is barely a "low-level queue feature". It's just a data structure that can be performant for some implementations of queues. Native arrays provide slower but similar features. 3. It's sane that this (Deques) implementation detail (the in-memory data structure) is less discussed than more central and generic subjects, like scaling MariaDB. If I consider the first two problems mentioned in the article (systems for issues and for kanban), I think Deques is unsuitable to them, because it's meant to replace arrays for in-memory works. The problems require a persistent storage, and the optimization of a PHP storage for the queue is probably irrelevant.

jw1224 3y ago
You're totally right and I agree too :) I forgot Deques (and DS) were via PECL. It's a poor example to give in relation to the article's real-world cases. Nonetheless, it is relevant to queue theory, and knowledge of how, why, and when these data structures can be more optimal than simple arrays has helped me more broadly in the past. A much more real-world example would be Laravel Horizon, which uses Redis queues to build scalable distributed systems very easily. The last time I set it up I think it took me 5 minutes at most... https://laravel.com/docs/10.x/horizon https://laravel.com/docs/10.x/horizon

kiitos 3y ago

> But I simply never see any discussion or usage of the low-level queue features offered by the language... Assuming we're talking about something that serves requests over the network, it rarely makes sense to define or use queues within the program itself. The network stack already contains many layers of queues which will almost always manage load concerns (e.g. throttling, backpressure, etc.) more effectively than your application can do itself. In general you want your app to accept and process requests as fast as it can, and leave queueing concerns to other authorities.