3 ms·

I just realized how useful queues can be. Instead of letting the system fail under heavy load. You put the tasks in a holding place. They will be processed l

I just realized how useful queues can be. Instead of letting the system fail under heavy load. You put the tasks in a holding place. They will be processed later. Computers have spoiled us with instant and now.... But even a 1 hour processing time is amazing if that task would take 3 days manually.

yurishimo 4y ago
There are entire languages built upon this model! Anything using the BEAM is a great example. Erlang, Elixir, Gleam, etc. The runtime acts as an event loop and if a specific program is hogging too much time on the CPU, it will get paused and thrown to the back of the execution stack. This is all transparent (relatively) for the developer. Here’s a video if you want to see an example: https://youtu.be/JvBT4XBdoUE https://youtu.be/JvBT4XBdoUE
AstixAndBelix 4y ago
I had a course about queue theory in my SWE degree. I left the course with a deep sense of dread because I knew most people studying CS and SWE in other universities would be oblivious to such a fundamental piece of knowledge. Basically everything that processes data has a queue attached to it, from your router to your scheduler. Not knowing anything about queue theory is such an insult to the profession of SWE. It's like being a Mech engineer and not knowing about control of dynamic systems.
- OJFord 4y ago
  Why did you think you knew that? I'm also one of your supposed few to have studied it, I never thought that. The dread should have been that most companies/teams won't care, won't engineer on that level, will just set it up however and make it bigger/faster/more-consumed if it's too slow.
  - setr 4y ago
    In my uni, the class was more or less unadvertised and the professor largely ignored — you had to go out of your way to take it (I was actively looking for poorly rated professors, because I figured they were usually rated such because they graded hard… which probably meant they cared more). When I tried to google the subject/code examples afterwards, I largely only found textbooks and one-off blog posts. Also a lot of GitHub repos with names like CS550… Only place I’ve seen it referenced since that class was: 1. HN periodically 2. A YouTube documentary about Disneyland fast pass queues 3. A business optimization consultancy Ultimately, a pretty niche subject area AFAICT. But implementing a fairly efficient queuing simulation seems to be pretty easy, and the subject seems to be stupid powerful and broadly applicable… so I’ve never understood why it’s not more discussed
    - OJFord 4y ago
      Compared to what, as if say information theory comes up (explicitly) all the time? Seems like you could say that about any more theoretically oriented course.
roflyear 4y ago
Except most times people don't implement the surrounding infra correctly. Let's say you consume a job, do something, but that something fails. You have to set it to be rescheduled somewhere. Lets say your things writing jobs to the queue fails. That needs to be handled as well. "They will be processed later [if I use a queue]" is not true by default (without work and thought) and assuming so will get you in trouble, and will also make certain people on your team who do understand that isn't the case upset.

yurishimo 4y ago

There are entire languages built upon this model! Anything using the BEAM is a great example. Erlang, Elixir, Gleam, etc. The runtime acts as an event loop and if a specific program is hogging too much time on the CPU, it will get paused and thrown to the back of the execution stack. This is all transparent (relatively) for the developer. Here’s a video if you want to see an example: https://youtu.be/JvBT4XBdoUE https://youtu.be/JvBT4XBdoUE

AstixAndBelix 4y ago

I had a course about queue theory in my SWE degree. I left the course with a deep sense of dread because I knew most people studying CS and SWE in other universities would be oblivious to such a fundamental piece of knowledge. Basically everything that processes data has a queue attached to it, from your router to your scheduler. Not knowing anything about queue theory is such an insult to the profession of SWE. It's like being a Mech engineer and not knowing about control of dynamic systems.

OJFord 4y ago
Why did you think you knew that? I'm also one of your supposed few to have studied it, I never thought that. The dread should have been that most companies/teams won't care, won't engineer on that level, will just set it up however and make it bigger/faster/more-consumed if it's too slow.
- setr 4y ago
  In my uni, the class was more or less unadvertised and the professor largely ignored — you had to go out of your way to take it (I was actively looking for poorly rated professors, because I figured they were usually rated such because they graded hard… which probably meant they cared more). When I tried to google the subject/code examples afterwards, I largely only found textbooks and one-off blog posts. Also a lot of GitHub repos with names like CS550… Only place I’ve seen it referenced since that class was: 1. HN periodically 2. A YouTube documentary about Disneyland fast pass queues 3. A business optimization consultancy Ultimately, a pretty niche subject area AFAICT. But implementing a fairly efficient queuing simulation seems to be pretty easy, and the subject seems to be stupid powerful and broadly applicable… so I’ve never understood why it’s not more discussed
  - OJFord 4y ago
    Compared to what, as if say information theory comes up (explicitly) all the time? Seems like you could say that about any more theoretically oriented course.

roflyear 4y ago

Except most times people don't implement the surrounding infra correctly. Let's say you consume a job, do something, but that something fails. You have to set it to be rescheduled somewhere. Lets say your things writing jobs to the queue fails. That needs to be handled as well. "They will be processed later [if I use a queue]" is not true by default (without work and thought) and assuming so will get you in trouble, and will also make certain people on your team who do understand that isn't the case upset.