3 ms·
"efficient Python web servers (like Tornado) use event driven way of handling HTTP requests. The downside of that is that there can only be one HTTP handler fun
by otterley 15y ago
"efficient Python web servers (like Tornado) use event driven way of handling HTTP requests. The downside of that is that there can only be one HTTP handler function executed at a time. If that function takes a lot of time, it blocks all other requests."
This statement needs clarity. If the handler is CPU-intensive, the design of the server (threaded vs. AIO) is irrelevant. Both will perform equally, the only difference being that on an SMP system, the OS could schedule the thread on a different processor. This is why the typical deployment model for AIO servers is to start one server per processor and route incoming requests to a proxy to balance the requests among the available servers.
On the other hand, if the handler is dependent on slow network services (e.g. database or downstream API service), the AIO-based server will easily be able to handle additional incoming requests while waiting for the other tasks to complete, subject to memory. This is the power of non-blocking I/O function calls.
- nitrogen 15y agothe only difference being that on an SMP system, the OS could schedule the thread on a different processor. There's a bit more to it than that. The OS will frequently block and preempt the long-running thread even on a single CPU system, allowing other, shorter requests to be processed in the mean time.
- otterley 15y agoI'm assuming for this example that every request has equal CPU impact. Preempting the running thread in this case won't improve overall request throughput on a single-CPU system. Adding concurrency here will simply increase response latency.