3 ms·

I come from outside of the Python ecosystem, so I'm not familiar with the specific frameworks mentioned (I read the original post too). Can someone simplify th

by _bxg1 7y ago

I come from outside of the Python ecosystem, so I'm not familiar with the specific frameworks mentioned (I read the original post too). Can someone simplify the argument that she's making about them? Something to do with maintaining separate, blocking threads in anticipation of requests vs spinning things up as you need them? Or is it a criticism of dynamic languages as HTTP servers in general (it's not clear what language was used in this article)?

hedora 7y ago
If you stand up a python json rpc server (using whatever libraries are mentioned at the top of the article) it will be nowhere near the untuned numbers. I’d guess more than an order of magnitude off, based on experience with similar python stacks. If you tune and use a performance oriented language (on a big multicore machine), you can probably get about 100x better than the untuned not-python throughputs. So, for http hello world, 1000 untuned python servers ~= one beefy server.
- _bxg1 7y ago
  What I don't understand is what the "untuned" case here is. You could stand up a Python JSON RPC server and simply "not tune it". That would be "untuned code". Presumably that isn't what's being described. Is this some C code written by the author? Is it Apache? Something else? Is the point just how much faster native code is than Python?
  - wmf 7y ago
    I'm guessing the "untuned" code here is C or some other statically-typed compiled language and it's not using any "framework". I guess the point is that straight-line Python code might be 10x-20x slower than C but an inefficient architecture can be 100x-1000x slower while being more complex than a good architecture.
  - hedora 7y ago
    The article goes out of its way to avoid describing the “untuned” setup, though the numbers look about right for a single threaded program without piles of frameworks and serializers in the way. Apparently someone else also built an untuned setup from scratch, and got better numbers. I think the point of the article is that if the results are surprising to you, then you should try writing one too.
wmf 7y ago
1. There's various low-hanging performance fruit that hasn't been picked. 2. Python has several different ways to do concurrency (because they evolved over ~25 years) that shouldn't be mixed but it's very easy to accidentally mix them. 3. Binary RPC (e.g. gRPC or Thrift) is more efficient than REST.
- hermitdev 7y ago
  I have a lib that queries quant data from a internal webservice. We have to break up large queries into smaller ones (like 4 days at a time). We currently use multiprocessing to get some parallelism. I prototyped using aiohttp. It was nearly an order of magnitude faster using a single thread. Unfortunately, cant use aiohttp because it doesn't do Negotiate/SSPI auth, which aiohttp doesnt support.
lumost 7y ago
Probably a bit of both, but python and other dynamic languages make doing the smart thing very difficult. While I continue to work professionally in both static and dynamic languages, I've observed a few limitations that prevent writing a fast python webserver - The Global Interpreter lock - you can't avoid forking processes in order to handle concurrent python code. - Threads are cheaper than processes. A default java thread carries 2MB overhead, a python process for a typical app can easily be 2GB without very careful memory consideration. - The pure single-threaded python runtime is 10-100x slower than your typical statically typed language, even when doing everything as carefully as possible you'll be 1-2 orders of magnitude off the best implementation in a statically typed language. Conversely a sloppy implementation in a statically typed language will probably work about as well as the best python implementation. - Foreign Function Interfaces(FFIs) are slow, in Python an int32 consumes 24 bytes of memory whereas in C the int32 consumes 4 bytes. Stringing together 2 C calls that manipulate an int with python would require 4 allocations and 4 casts. Applications that avoid this overhead have to adopt symbolic APIs that inform an underlying C program how to connect multiple function calls and will grind to a halt if there is any python control flow e.g. TensorFlow. Most of these limitations tend to be common to other dynamically typed languages such as Ruby, PHP, and Perl. While one can theoretically "drop down to C", it's often not that straightforward in common development scenarios. For businesses that require high(er) throughput, or low(er) latency the time spent fighting the interpreter or the associated AWS bills may not be worth the productivity gains from dynamic types in 2020 when compilers can perform an incremental build in milliseconds.
- _bxg1 7y ago
  I don't think it's productive to argue that non-dynamic languages are superior across the board. Everybody knows they're faster. For many people, they're still the right decision. I think (or at least assumed) that the post's discussion is more interesting than that. A sibling comment seemed to be zeroing in on the crux of the issue. "Python is an order of magnitude slower but it's web server is two orders of magnitude slower" is a meaningful and fruitful thing to talk about.
  - lumost 7y ago

by _bxg1 7y ago

hedora 7y ago

If you stand up a python json rpc server (using whatever libraries are mentioned at the top of the article) it will be nowhere near the untuned numbers. I’d guess more than an order of magnitude off, based on experience with similar python stacks. If you tune and use a performance oriented language (on a big multicore machine), you can probably get about 100x better than the untuned not-python throughputs. So, for http hello world, 1000 untuned python servers ~= one beefy server.

_bxg1 7y ago
What I don't understand is what the "untuned" case here is. You could stand up a Python JSON RPC server and simply "not tune it". That would be "untuned code". Presumably that isn't what's being described. Is this some C code written by the author? Is it Apache? Something else? Is the point just how much faster native code is than Python?
- wmf 7y ago
  I'm guessing the "untuned" code here is C or some other statically-typed compiled language and it's not using any "framework". I guess the point is that straight-line Python code might be 10x-20x slower than C but an inefficient architecture can be 100x-1000x slower while being more complex than a good architecture.
- hedora 7y ago
  The article goes out of its way to avoid describing the “untuned” setup, though the numbers look about right for a single threaded program without piles of frameworks and serializers in the way. Apparently someone else also built an untuned setup from scratch, and got better numbers. I think the point of the article is that if the results are surprising to you, then you should try writing one too.

wmf 7y ago

1. There's various low-hanging performance fruit that hasn't been picked. 2. Python has several different ways to do concurrency (because they evolved over ~25 years) that shouldn't be mixed but it's very easy to accidentally mix them. 3. Binary RPC (e.g. gRPC or Thrift) is more efficient than REST.

hermitdev 7y ago
I have a lib that queries quant data from a internal webservice. We have to break up large queries into smaller ones (like 4 days at a time). We currently use multiprocessing to get some parallelism. I prototyped using aiohttp. It was nearly an order of magnitude faster using a single thread. Unfortunately, cant use aiohttp because it doesn't do Negotiate/SSPI auth, which aiohttp doesnt support.

lumost 7y ago

Probably a bit of both, but python and other dynamic languages make doing the smart thing very difficult. While I continue to work professionally in both static and dynamic languages, I've observed a few limitations that prevent writing a fast python webserver - The Global Interpreter lock - you can't avoid forking processes in order to handle concurrent python code. - Threads are cheaper than processes. A default java thread carries 2MB overhead, a python process for a typical app can easily be 2GB without very careful memory consideration. - The pure single-threaded python runtime is 10-100x slower than your typical statically typed language, even when doing everything as carefully as possible you'll be 1-2 orders of magnitude off the best implementation in a statically typed language. Conversely a sloppy implementation in a statically typed language will probably work about as well as the best python implementation. - Foreign Function Interfaces(FFIs) are slow, in Python an int32 consumes 24 bytes of memory whereas in C the int32 consumes 4 bytes. Stringing together 2 C calls that manipulate an int with python would require 4 allocations and 4 casts. Applications that avoid this overhead have to adopt symbolic APIs that inform an underlying C program how to connect multiple function calls and will grind to a halt if there is any python control flow e.g. TensorFlow. Most of these limitations tend to be common to other dynamically typed languages such as Ruby, PHP, and Perl. While one can theoretically "drop down to C", it's often not that straightforward in common development scenarios. For businesses that require high(er) throughput, or low(er) latency the time spent fighting the interpreter or the associated AWS bills may not be worth the productivity gains from dynamic types in 2020 when compilers can perform an incremental build in milliseconds.

_bxg1 7y ago
I don't think it's productive to argue that non-dynamic languages are superior across the board. Everybody knows they're faster. For many people, they're still the right decision. I think (or at least assumed) that the post's discussion is more interesting than that. A sibling comment seemed to be zeroing in on the crux of the issue. "Python is an order of magnitude slower but it's web server is two orders of magnitude slower" is a meaningful and fruitful thing to talk about.
- lumost 7y ago