3 ms·

>A better model already exists - thread per connection For that you will need a language with good support for threads, which Python definitely isn't (at least

>A better model already exists - thread per connection For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here. Native threads are huge improvement over async, but still not as good as green threads, which really are the best option here for 99% of usecases, abstracting whole async thing away. > Thread per connection has gotten a bad name for performance which is pretty false. With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.

Scarbutt 6y ago
With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist Sounds good to me! Can you even find a VM offering that has a 64 core CPU and less than 32GB of RAM?
- fiedzia 6y ago
  I tried to search for it and and... well, google is so ad-infested that's its useless, but doesn't seem so. Anything with large core count comes with 192GB or more.
Something1234 6y ago
>>A better model already exists - thread per connection > For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here The thread per connection is managed by your application server. Why does the language need to have good thread primitives then? In fact I would argue you spawning and managing your own threads make your service less robust.
- fiedzia 6y ago
  > Why does the language need to have good thread primitives then? Because: a) we want application server to be written in the language we are using, so that we can control every aspect of how it works, instead of having some silo that developer will avoid because its so different b) those threads will share cache, connections, some preloaded data you may need and so on, you want a language that allows this to be done reliablyl c) those threads may need to create more threads (make parallel requests to 3 services and resize an image at the same time)
  - empthought 6y ago
    As if Apache/nginx and wsgi isn’t still the norm...
    - fiedzia 6y ago
      Uwagi is used for performance reasons only, once you have a language where its not a problem, it immediately goes away. I haven't seen anyone using it with Go for example.
dagenix 6y ago
> With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads. Are you assuming that each thread requires a dedicated stack on each core? That isn't how it works - each thread has a single stack regardless of the number of CPU cores. So, a 2MB stack requires just 2MB. However, that 2MB is 2MB of virtual memory. So, assuming that the stack isn't allocated using a 2MB huge page, it will probably actually be physically allocated in 4K chunks as required. So, most threads are going to use considerably less than 2MB of actual memory. Outside of 32bit systems, virtual memory is close to an unlimited resource. Green threads also require stacks. How green threads allocate stack space is going to depend on the particular runtime, but it's likely that like with real threads they will also lazily allocate memory as required. A big difference between stack usage with green threads and regular threads is that the runtime _may_ support shrinking the stack used by a green thread if space is only briefly required by a green thread, while a regular thread will hold on to any memory that is physically allocated for its stack until it exits. Reclaiming memory like this isn't free, however - and you run the risk of "stack thrashing" where the point where a stack grows and shrinks is accidentally in the middle of a hot loop and you end up spending basically all your time growing and shrinking the stack as opposed to doing real work. I _think_ that some green thread runtimes may also support stacks smaller than 4K - but I'm not really sure if / how many actually do.
- fiedzia 6y ago
  > Are you assuming that each thread requires a dedicated stack on each core? No, I am assuming we run 1 thread per core.

Scarbutt 6y ago

With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist Sounds good to me! Can you even find a VM offering that has a 64 core CPU and less than 32GB of RAM?

fiedzia 6y ago
I tried to search for it and and... well, google is so ad-infested that's its useless, but doesn't seem so. Anything with large core count comes with 192GB or more.

Something1234 6y ago

>>A better model already exists - thread per connection > For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here The thread per connection is managed by your application server. Why does the language need to have good thread primitives then? In fact I would argue you spawning and managing your own threads make your service less robust.

fiedzia 6y ago
> Why does the language need to have good thread primitives then? Because: a) we want application server to be written in the language we are using, so that we can control every aspect of how it works, instead of having some silo that developer will avoid because its so different b) those threads will share cache, connections, some preloaded data you may need and so on, you want a language that allows this to be done reliablyl c) those threads may need to create more threads (make parallel requests to 3 services and resize an image at the same time)
- empthought 6y ago
  As if Apache/nginx and wsgi isn’t still the norm...
  - fiedzia 6y ago
    Uwagi is used for performance reasons only, once you have a language where its not a problem, it immediately goes away. I haven't seen anyone using it with Go for example.

dagenix 6y ago

> With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads. Are you assuming that each thread requires a dedicated stack on each core? That isn't how it works - each thread has a single stack regardless of the number of CPU cores. So, a 2MB stack requires just 2MB. However, that 2MB is 2MB of virtual memory. So, assuming that the stack isn't allocated using a 2MB huge page, it will probably actually be physically allocated in 4K chunks as required. So, most threads are going to use considerably less than 2MB of actual memory. Outside of 32bit systems, virtual memory is close to an unlimited resource. Green threads also require stacks. How green threads allocate stack space is going to depend on the particular runtime, but it's likely that like with real threads they will also lazily allocate memory as required. A big difference between stack usage with green threads and regular threads is that the runtime _may_ support shrinking the stack used by a green thread if space is only briefly required by a green thread, while a regular thread will hold on to any memory that is physically allocated for its stack until it exits. Reclaiming memory like this isn't free, however - and you run the risk of "stack thrashing" where the point where a stack grows and shrinks is accidentally in the middle of a hot loop and you end up spending basically all your time growing and shrinking the stack as opposed to doing real work. I _think_ that some green thread runtimes may also support stacks smaller than 4K - but I'm not really sure if / how many actually do.

fiedzia 6y ago
> Are you assuming that each thread requires a dedicated stack on each core? No, I am assuming we run 1 thread per core.