3 ms·

The problem with Django services is the insanely low concurrency level compared to other server frameworks (including node). Django is single request at a time

The problem with Django services is the insanely low concurrency level compared to other server frameworks (including node). Django is single request at a time with no async. The standard fix is gunicorn worker processes, but then you require entire server memory * N memory instead of lightweight thread/request struct * N memory for N requests. I shudder to think that whenever Django server is doing an HTTP request to a different service or running a DB query, it's just doing nothing while other requests are waiting in the gunicorn queue. The difference is if you have an endpoint with 2s+ queries taking 2s for one customer, with Django, it might cause the entire service to stall for everybody, whereas with a decent async server framework other fast endpoints can make progress while the 2s ones are slow.

pdhborges 4y ago
You can configure gunicorn to use multiple threads to recover quite a bit of concurrency in those scenarios and that is enough for many applications.
- srcreigh 4y ago
  What threading/workers configuration do you use? I'm looking at a page now which recommends 9 concurrent. requests for a Django server running on a 4 core computer. Meanwhile node servers can easily handle hundreds of concurrent requests.
  - pdhborges 4y ago
    We use the ncpu * 2 + 1 formula for the number of workers that serve API requests. I don't think in 'handling x concurrent requests' terms because I don't even know what that means. Usually I think around thoughout, latency distributions and number of connections that can be kept open (for servers that deal with web sockets). For example if you have the 4 core computer and you have 4 workers and your requests take around 50ms each you can get to a throughput of 80 requests per second. If the fraction of request time for IO if 50% you can bump your thread count to try to reach 160 request per second. Note that in this case each request consumes 25ms of CPU so you would never be able to get more than 40 requests per second per CPU whether you are using node or python.
manfre 4y ago
Django has async support for everything except the ORM. async db is possible without the ORM or by doing some thread pool/sync to async wrapping. A PR for that was under review last I checked. Either way, high concurrency websites shouldn't have queries that take multiple seconds and it's still possible to block async processes in most languages if you mix in a blocking sync operation.

pdhborges 4y ago

You can configure gunicorn to use multiple threads to recover quite a bit of concurrency in those scenarios and that is enough for many applications.

srcreigh 4y ago
What threading/workers configuration do you use? I'm looking at a page now which recommends 9 concurrent. requests for a Django server running on a 4 core computer. Meanwhile node servers can easily handle hundreds of concurrent requests.
- pdhborges 4y ago
  We use the ncpu * 2 + 1 formula for the number of workers that serve API requests. I don't think in 'handling x concurrent requests' terms because I don't even know what that means. Usually I think around thoughout, latency distributions and number of connections that can be kept open (for servers that deal with web sockets). For example if you have the 4 core computer and you have 4 workers and your requests take around 50ms each you can get to a throughput of 80 requests per second. If the fraction of request time for IO if 50% you can bump your thread count to try to reach 160 request per second. Note that in this case each request consumes 25ms of CPU so you would never be able to get more than 40 requests per second per CPU whether you are using node or python.

manfre 4y ago

Django has async support for everything except the ORM. async db is possible without the ORM or by doing some thread pool/sync to async wrapping. A PR for that was under review last I checked. Either way, high concurrency websites shouldn't have queries that take multiple seconds and it's still possible to block async processes in most languages if you mix in a blocking sync operation.