4 ms·

Async is useful for high IO where you may have a lot of down time between the requests. Are you pulling many requests from different servers with different resp

by Grimm1 6y ago

Async is useful for high IO where you may have a lot of down time between the requests. Are you pulling many requests from different servers with different response times, communicating with a db or pulling out large response bodies. Async is probably going to do better since each one of those synchronously represents potentially large idling periods where other requests could have gotten work done. As to the article the comparisons are good but fails to mention resource constraints, like Gunicorn, forking 16 instances is going to be a lot heavier on memory so for a little more RPS you're probably spending a decent chunk of change more to run your work and I don't think that's worth it considering the Async model in python is pretty easy to grok these days and under this benchmark share a similar performance profile. Now that said If I had to guess these numbers are fine for the average API but if you're doing something like high throughput web crawling or need to serve something on the order of 10's of thousands to hundred thousands RPS async will win out on speed and resource use and ultimately cost. Plus at one point they were like "we could only get an 18% speed up with Vibora" haven't used them my self. But 18% performance increase at really any level of load is fantastic. Hand waving that off tells me the work loads for what is "realistic" don't take in to account real high RPS workloads like you might see at major tech companies.

meritt 6y ago
> forking 16 instances is going to be a lot heavier on memory It really depends on how the application is designed. Fork operates through mmap and copy-on-write. It's extremely lightweight by default. A well-designed fork-based application will already have everything necessary to run a given process into memory, not munge any of the existing shared memory, and only allocate and free memory associated with new events/connections/etc. When programmed that way, individual forks are incredibly light on resources. All the workers are sharing the exact same core application code and logic in memory.
- Grimm1 6y ago
  "All the workers are sharing the exact same core application code and logic in memory." Oh interesting, are you saying an intelligent forking implementation is able to share static portions of memory with multiple children? I was perhaps under the naive assumption forking was pretty much just a full memory copy of the parent.
  - meritt 6y ago
    Yep, but that's simply how the linux kernel works [1]. As a programmer you need to essentially load up all your modules/libraries/data/etc that you will need into memory before the fork, and treat the forked() processes as read-only as much as possible from a resource perspective. If you modify anything from the parent, you get your own page as soon as that happens. [2][3] [1] https://www.informit.com/articles/article.aspx?p=368650 https://www.informit.com/articles/article.aspx?p=368650 [2] https://en.wikipedia.org/wiki/Copy-on-write https://en.wikipedia.org/wiki/Copy-on-write [3] https://en.wikipedia.org/wiki/Fork_(system_call) https://en.wikipedia.org/wiki/Fork_(system_call)

by Grimm1 6y ago

meritt 6y ago

> forking 16 instances is going to be a lot heavier on memory It really depends on how the application is designed. Fork operates through mmap and copy-on-write. It's extremely lightweight by default. A well-designed fork-based application will already have everything necessary to run a given process into memory, not munge any of the existing shared memory, and only allocate and free memory associated with new events/connections/etc. When programmed that way, individual forks are incredibly light on resources. All the workers are sharing the exact same core application code and logic in memory.

Grimm1 6y ago
"All the workers are sharing the exact same core application code and logic in memory." Oh interesting, are you saying an intelligent forking implementation is able to share static portions of memory with multiple children? I was perhaps under the naive assumption forking was pretty much just a full memory copy of the parent.
- meritt 6y ago
  Yep, but that's simply how the linux kernel works [1]. As a programmer you need to essentially load up all your modules/libraries/data/etc that you will need into memory before the fork, and treat the forked() processes as read-only as much as possible from a resource perspective. If you modify anything from the parent, you get your own page as soon as that happens. [2][3] [1] https://www.informit.com/articles/article.aspx?p=368650 https://www.informit.com/articles/article.aspx?p=368650 [2] https://en.wikipedia.org/wiki/Copy-on-write https://en.wikipedia.org/wiki/Copy-on-write [3] https://en.wikipedia.org/wiki/Fork_(system_call) https://en.wikipedia.org/wiki/Fork_(system_call)