4 ms·

Async web servers are more complex than thread per connection ones. I think that's the big thing the author is communicating. It's true. And most types of d

Async web servers are more complex than thread per connection ones. I think that's the big thing the author is communicating. It's true. And most types of development don't need async io. It's worth talking about. The author makes the mistake of calling this a language problem and not a web server problem. With python, Java, c#, and hell, even js, you can have a thread per connection web server if you choose to. This will avoid many async woes. It's just not trendy.

hliyan 9y ago
This seems like a good faith comment. I'm not really sure why it's getting downvoted. Someone care to shed some light?
- TomMarius 9y ago
  Nowadays people downvote because they disagree. It's sad because it pointlessly frustrates the poster and kills discussions.
  - steveklabnik 9y ago
    It's not just 'nowadays'. pg a decade ago: https://news.ycombinator.com/item?id=117171 https://news.ycombinator.com/item?id=117171 > I think it's ok to use the up and down arrows to express agreement.
    - TomMarius 9y ago
      From my personal point of view, it's happening more in the last year than it did before.
eecc 9y ago
Except I remember my times as sysadmin, putting out fires on Tomcats brought to their knees by 1 poorly performing query and a few impatient users hitting F5. Yes, async is indeed more complex but it is a measurable improvement over previous - thread based - designs, and it is trivial to hit scenarios where it will make a difference. Just like anything though, it must be applied sensibly: not every single call must be decomposed into a choreography of async collaborators (yes, listen that, it’s “architecture talk”, it’s like that dark language used in Mordor that even Gandalf is scared to pronounce. Ever worked on a DDD/CQRS/ES project? ;)
xtrapolate 9y ago
> "It's just not trendy." The "thread-per-connection" model doesn't scale. It's simply not resource-efficient in any way.
- vosper 9y ago
  I suspect it scales plenty for what most people are actually trying to do. Especially if the cost of throwing a few more web servers at the problem is less than the extra developer time that it costs to live in the async world. You can have quite a lot of threads on a fairly cheap machine these days.
  - adventured 9y ago
    99%+ of all use cases will never need the kind of scaling that async is more ideal for. A small fraction of 1% of all sites/services will ever push more than 10k or 20k uniques per day. With today's infrastructure you can trivially run a large number of the Quantcast US top 5,000 sites on boring old PHP 7.x (which is very fast) with MySQL or Postgres and a cost effective VPS cluster from Digital Ocean. All of these use cases that talk about even a thousand simultaneous connections, are elite 1/10th of 1/10th of 1% of all use case scenarios. Async is comical overkill for nearly everyone, unless you've got a very resource hungry service or a vast number of users. Exceptionally few developers ever run into that scenario. Optimizing that 5% or 10% resource savings when you're pushing 15% of the resources of a single machine, is a waste of time that should be focused on product instead. The first site I ever got into that bracket of traffic, into the Quancast top 3k, it cost me $550 per month for two machines, using Nehalem 5500 processors and 24gb of ram. That was 2011-2012, and 1.x million uniques per month. That was using PHP 5.x and MySQL. And it was overkill by about 50%, while serving up entirely dynamic pages with zero caching and no CDN. US traffic & usage hasn't increased much since 2012, unless you're one of the few hyper scale services. Mostly the audience growth has been stagnant, while the hardware has kept getting better. Today you can run what I was back then for $80 on two VPS machines. Do some use cases need async? Absolutely. Do most? Not even remotely close.
    - kthejoker2 9y ago
2sk21 9y ago
Everytimee I have to wade into Node, I feel like they are reinventing the early days of programming with cooperative multi-tasking.
pjc50 9y ago
Ultimately the problem is "the system needs to retain enough state per connection to resume executing the 'next' step of the communication". This state is currently split into different pieces and spread around the system. The operating system maintains the TCP state, for example. If you use threads to partition connections, the operating system looks after the instruction pointer to resume a non-running thread at. The program (whatever language) will usually have some kind of stream state and maybe buffers. It's all a question of where you want to put the abstraction boundaries. Heavy threads? Light threads? Async? Continuations? Or go the other way and have process-per-core with userland drivers and TCP state, where each process is just listening for interrupts and handles everything from there?
- zbentley 9y ago
  > Or go the other way and have process-per-core with userland drivers and TCP state, where each process is just listening for interrupts and handles everything from there? A kernel. You are describing a kernel, minus workers, plus a web server.
  - pjc50 9y ago
    I was thinking of things like https://shader.kaist.edu/mtcp/ https://shader.kaist.edu/mtcp/ , or https://github.com/snabbco/snabb https://github.com/snabbco/snabb or these kind of things: https://jvns.ca/blog/2016/06/30/why-do-we-use-the-linux-kernels-tcp-stack/ https://jvns.ca/blog/2016/06/30/why-do-we-use-the-linux-kern...
bennofs 9y ago
With green threads, I think you can have async io but with threads. When your current thread is making an async call, suspend the green thread (cheap) and yield back to the event loop. Threads are woken up when their async call finishes. This avoids the overhead of real OS threads per connection.

hliyan 9y ago

This seems like a good faith comment. I'm not really sure why it's getting downvoted. Someone care to shed some light?

TomMarius 9y ago
Nowadays people downvote because they disagree. It's sad because it pointlessly frustrates the poster and kills discussions.
- steveklabnik 9y ago
  It's not just 'nowadays'. pg a decade ago: https://news.ycombinator.com/item?id=117171 https://news.ycombinator.com/item?id=117171 > I think it's ok to use the up and down arrows to express agreement.
  - TomMarius 9y ago
    From my personal point of view, it's happening more in the last year than it did before.

eecc 9y ago

Except I remember my times as sysadmin, putting out fires on Tomcats brought to their knees by 1 poorly performing query and a few impatient users hitting F5. Yes, async is indeed more complex but it is a measurable improvement over previous - thread based - designs, and it is trivial to hit scenarios where it will make a difference. Just like anything though, it must be applied sensibly: not every single call must be decomposed into a choreography of async collaborators (yes, listen that, it’s “architecture talk”, it’s like that dark language used in Mordor that even Gandalf is scared to pronounce. Ever worked on a DDD/CQRS/ES project? ;)

xtrapolate 9y ago

> "It's just not trendy." The "thread-per-connection" model doesn't scale. It's simply not resource-efficient in any way.

vosper 9y ago
I suspect it scales plenty for what most people are actually trying to do. Especially if the cost of throwing a few more web servers at the problem is less than the extra developer time that it costs to live in the async world. You can have quite a lot of threads on a fairly cheap machine these days.
- adventured 9y ago
  99%+ of all use cases will never need the kind of scaling that async is more ideal for. A small fraction of 1% of all sites/services will ever push more than 10k or 20k uniques per day. With today's infrastructure you can trivially run a large number of the Quantcast US top 5,000 sites on boring old PHP 7.x (which is very fast) with MySQL or Postgres and a cost effective VPS cluster from Digital Ocean. All of these use cases that talk about even a thousand simultaneous connections, are elite 1/10th of 1/10th of 1% of all use case scenarios. Async is comical overkill for nearly everyone, unless you've got a very resource hungry service or a vast number of users. Exceptionally few developers ever run into that scenario. Optimizing that 5% or 10% resource savings when you're pushing 15% of the resources of a single machine, is a waste of time that should be focused on product instead. The first site I ever got into that bracket of traffic, into the Quancast top 3k, it cost me $550 per month for two machines, using Nehalem 5500 processors and 24gb of ram. That was 2011-2012, and 1.x million uniques per month. That was using PHP 5.x and MySQL. And it was overkill by about 50%, while serving up entirely dynamic pages with zero caching and no CDN. US traffic & usage hasn't increased much since 2012, unless you're one of the few hyper scale services. Mostly the audience growth has been stagnant, while the hardware has kept getting better. Today you can run what I was back then for $80 on two VPS machines. Do some use cases need async? Absolutely. Do most? Not even remotely close.
  - kthejoker2 9y ago

2sk21 9y ago

Everytimee I have to wade into Node, I feel like they are reinventing the early days of programming with cooperative multi-tasking.

pjc50 9y ago

Ultimately the problem is "the system needs to retain enough state per connection to resume executing the 'next' step of the communication". This state is currently split into different pieces and spread around the system. The operating system maintains the TCP state, for example. If you use threads to partition connections, the operating system looks after the instruction pointer to resume a non-running thread at. The program (whatever language) will usually have some kind of stream state and maybe buffers. It's all a question of where you want to put the abstraction boundaries. Heavy threads? Light threads? Async? Continuations? Or go the other way and have process-per-core with userland drivers and TCP state, where each process is just listening for interrupts and handles everything from there?

zbentley 9y ago
> Or go the other way and have process-per-core with userland drivers and TCP state, where each process is just listening for interrupts and handles everything from there? A kernel. You are describing a kernel, minus workers, plus a web server.
- pjc50 9y ago
  I was thinking of things like https://shader.kaist.edu/mtcp/ https://shader.kaist.edu/mtcp/ , or https://github.com/snabbco/snabb https://github.com/snabbco/snabb or these kind of things: https://jvns.ca/blog/2016/06/30/why-do-we-use-the-linux-kernels-tcp-stack/ https://jvns.ca/blog/2016/06/30/why-do-we-use-the-linux-kern...

bennofs 9y ago

With green threads, I think you can have async io but with threads. When your current thread is making an async call, suspend the green thread (cheap) and yield back to the event loop. Threads are woken up when their async call finishes. This avoids the overhead of real OS threads per connection.