3 ms·

Async queues are far from meaningless. They are the basis of distributed, resumanble/retryable computation. If you can't see the advantage of keeping your proc

Async queues are far from meaningless. They are the basis of distributed, resumanble/retryable computation. If you can't see the advantage of keeping your processors fed with work as close to 100% of the time as humanly possible, I got nothing for ya. Hell, AWS is basically popular because you can have novice programmers toss up synchronously programmed lambdas, and scale out the underlying infrastructure with Amazon magic, which in a non-AWS system translates to "define queue and message format which will be servicing producer/consumer workers, deploy and start dumping messages in/popping messages out of said queue; scale to taste. It takes a while to wrap your head around async, but once you get there, you're building with one of the most powerful computing primitives I've yet to come across. It's just unfortunate that the language and requisite mental model around it is, in my experience, a bit difficult to intuitively reach.

peterhunt 4y ago
I didn’t say they were meaningless, just that they are often overused and can make systems needlessly more complex. How many microservices architectures with dependent webs of async queues emulating distributed transaction semantics could be replaced by one big rdbms? I think a lot!
- taeric 4y ago
  My favorite is when a new project thinks it needs this on day one. I'm willing to place money that most new projects will struggle to saturate a single computer. Much less a distributed infrastructure. (yes, I'm well aware there are some cases. But even most fang projects are doing relatively little work, all told.)
  - ericlewis 4y ago
    Chime famously (but maybe not externally) ran almost all infra off two co-located beefy servers all the way up to their millionth customer. Maybe that is dumb, but we never had any downtime or slowness that was a result of work we did. The APIs we would rely on sometimes though did.
  - z3t4 4y ago
    Another reason there is so much complexity is that the computer speed allows it. If you are constrained by processing power you will chose a more simple architecture... until you start optimizing. A lot on unnecessary complexity is premature optimizations. That "web scale" service you are building with "serverless" and other cloud services will likely run fine on a 1$/moth VPS - but who would like to manage such a small budget :P
    - taeric 4y ago
      You aren't wrong, but I suspect there was more unnecessary complexity in early computers then you'd allow. Is why don't ridiculous demo scene stuff is still created today. :)
    - torginus 4y ago
      It's just so ironic to see this monstrous amount of complexity that is only made tolerable by the massively overpowered hardware beneath it, and which, reduces the throughput of said hardware enough that said complexity becomes necessary.

peterhunt 4y ago

I didn’t say they were meaningless, just that they are often overused and can make systems needlessly more complex. How many microservices architectures with dependent webs of async queues emulating distributed transaction semantics could be replaced by one big rdbms? I think a lot!

taeric 4y ago
My favorite is when a new project thinks it needs this on day one. I'm willing to place money that most new projects will struggle to saturate a single computer. Much less a distributed infrastructure. (yes, I'm well aware there are some cases. But even most fang projects are doing relatively little work, all told.)
- ericlewis 4y ago
  Chime famously (but maybe not externally) ran almost all infra off two co-located beefy servers all the way up to their millionth customer. Maybe that is dumb, but we never had any downtime or slowness that was a result of work we did. The APIs we would rely on sometimes though did.
- z3t4 4y ago
  Another reason there is so much complexity is that the computer speed allows it. If you are constrained by processing power you will chose a more simple architecture... until you start optimizing. A lot on unnecessary complexity is premature optimizations. That "web scale" service you are building with "serverless" and other cloud services will likely run fine on a 1$/moth VPS - but who would like to manage such a small budget :P
  - taeric 4y ago
    You aren't wrong, but I suspect there was more unnecessary complexity in early computers then you'd allow. Is why don't ridiculous demo scene stuff is still created today. :)
  - torginus 4y ago
    It's just so ironic to see this monstrous amount of complexity that is only made tolerable by the massively overpowered hardware beneath it, and which, reduces the throughput of said hardware enough that said complexity becomes necessary.