6 ms·

Show HN: FastWSGI, an ultra-fast WSGI server for Python

89 points by james_roberts 5y ago

jedberg 5y ago
Those are some amazing performance metrics! What's the catch? Is there a tradeoff or did you really find a 10x efficiency?
- james_roberts 5y ago
  Thanks! I was quite surprised myself when I saw the numbers for the basic WSGI app. I'm still trying to figure out exactly why my server is so much faster than other servers out there... My guess is that I avoid calling Python as much as possible. Most of the parsing and the `start_response` code is all written in C instead of Python. Flask applications seem to have some bottleneck. I don't see the same performance numbers as I do with the basic WSGI app. *A note: Those tests are just "Hello World" tests on a single worker. They should be taken with a grain of salt...
james_roberts 5y ago
I've been developing a Python extension, written in C, that provides users with an ultra-fast WSGI server that they can run their WSGI applications on (Flask, Django, uWSGI etc). I have also recently managed to get it working on multiple platforms (Linux, MacOS and Windows). If you want to significantly speed up your WSGI based applications, check it out! It is still in early development at the moment. Any feedback would be greatly appreciated! === [Links] === Github: https://github.com/jamesroberts/fastwsgi https://github.com/jamesroberts/fastwsgi Pypi: https://pypi.org/project/fastwsgi/ https://pypi.org/project/fastwsgi/ Performance comparisons against other popular WSGI servers: https://github.com/jamesroberts/fastwsgi/blob/main/performance_benchmarks/PERFORMANCE.md https://github.com/jamesroberts/fastwsgi/blob/main/performan...
- nerpderp82 5y ago
  In real world performance it doesn't appear to be that much faster than Bjoern, which you also surmised that Flask is the bottleneck. Are there other Python WSGI frameworks that can take advantage of FastWSGI? Why is Flask as slow as it is? Since this is C, what security mitigations have you put into place?
  - james_roberts 5y ago
    In theory, any framework that follows the WSGI guidelines should be able to run on top of FastWSGI and take advantage of its speed. There are many frameworks out there. I do intend to test out some more. For now I've only tested the popular Flask framework and a simple bare bones WSGI app. Flask was never developed to be lightning fast. Even still, I am quite surprised at how slow it is now that I've seen what kind of numbers can be achieved. I haven't looked deep into it to see where the issues might be. As for security, that is a work in progress... I definitely wouldn't use FastWSGI in production in the projects current state. It's still early days in terms of development.
    - byteface 5y ago
      Is it to do with the router?. I was reading this article. https://www.slideshare.net/kwatch/how-to-make-the-fastest-router-in-python https://www.slideshare.net/kwatch/how-to-make-the-fastest-ro... which lead me to this code. https://github.com/kwatch/router-sample/blob/master/minikeight.py https://github.com/kwatch/router-sample/blob/master/minikeig... and i was going to consider trying to benchmark different routers. I started a repo here which drops flask and uses some random router off pypi... https://github.com/byteface/fastwsgitest/blob/master/app.py https://github.com/byteface/fastwsgitest/blob/master/app.py and i think i just need to merge that state machine router into it for a test. I believe it's part of the architecture for templating engine called tenjin that predates even jinja? found your benchmark repo but not installed wrk yet to test.
- vangelis 5y ago
  FYI, you have a typo in the graph `Requests served in 60 seconds`. server'd.
  - deleted 5y ago
    [deleted]
  - james_roberts 5y ago
    Thanks, nice catch! I will update that.
- necovek 5y ago
  With Python, benchmarks that matter are what's the maximum performance you can get out of a particular hardware in the most optimal configuration for the tool in use. (This was most obvious with the async framework benchmarks) Perhaps other WSGI frameworks are not achieving 100% CPU load and FastWCGI is, and you could easily run them with multiple worker processes to get the same CPU load and comparable performance? (this is just wild speculation) I don't know, but that's the kind of benchmark I'd like to see: eg. what's the maximum performance you can get out of a, say, 4-core CPU with each of them with whatever configuration stresses the CPU completely, and what are the other metrics you might be seeing (eg. asyncio will basically net you lower memory use, but not any better performance in RPS)? It'd also be good if the benchmark tool is not running on the same CPU though, as long as you've good a sufficiently fast interconnect.
  - necovek 5y ago
    Not to be taken badly, results you post are amazing and point at a greatly optimised request handling! But with these benchmarks you do not demonstrate the practical value over any competing tool (just that your worker threads can do more in the same time, which is more of the "good engineering" pat-on-the-back type of thing :).
    - james_roberts 5y ago
      Thanks for the feedback! Yeah, the benchmarks just highlight higher numbers in very simple tests. They should definitely be taken with a grain of salt. I can try add some more practical highlights.
  - james_roberts 5y ago
    Yeah I don't have an optimal benchmarking setup currently. I have a 16-core CPU that I run everything off (server + benchmarks = not ideal). I've been working on adding multiprocessing to FastWSGI (only works on Linux at the moment). The RPS almost scales linearly with the number of workers. At 4 processes it hits +260k RPS with decent CPU utilization. I can definitely add more details to the benchmarks. Thanks for the feedback!
- barbazoo 5y ago
  > provides users with an ultra-fast WSGI server that they can run their WSGI applications on (Flask, Django, uWSGI etc) I thought this was an equivalent to uWSGI, what's the benefit of running uWSGI on top of FastWSGI?
  - james_roberts 5y ago
    Oops. This is a typo... should be WSGI apps. Not the uWSGI server
- zerkten 5y ago
  It would be good to eventually see some comparisons running some average/unoptimized code. IME benchmarks seem to focus on either the very basics, or select areas where they are faster than other apps. This is important to cover, but having something that's closer to a real app is more convincing, even if the performance margins drop somewhat. To come back to the average code. I may start an app and try to optimize things as much as reasonable. Eventually, I'm only able to focus on the functionality and a slide in performance can happen. As more developers are added, this can happen more quickly in the average organization that focuses primarily on functionality. Of course, we should analyze the perf problems and improve the code, but projects like yours may offer a huge perf boost for teams that are struggling here. I love Python, but I've always enjoyed that bit more scope that the .NET platform provided around performance when code was less optimal. If you can really speed up WSGI this much it'll be a huge boon.
  - james_roberts 5y ago
    Thanks for the feedback! Yeah, some more "real world" benchmarks should be added. And yes, if you have some application that you've tried to optimize somewhat and haven't managed to get the desired performance you would like, or if you simply haven't had the time to re-write components to be faster, ideally you could use FastWSGI as a drop in replacement for your current WSGI server and get the extra perf boost for "free". It's still in early development, but this is ultimately one of the main goals of the project.
- webmobdev 5y ago
  Looks great, thanks. I was going through the Performance Benchmarks and noticed that the mod_wsgi Apache module - https://github.com/GrahamDumpleton/mod_wsgi https://github.com/GrahamDumpleton/mod_wsgi - is missing? Please consider including it in the benchmark too - would love to see how your module matches against it.
  - james_roberts 5y ago
    Will do! Thanks for the feedback
dabeeeenster 5y ago
Would love to see benchmarks comparing this to FastAPI? Thanks for this project - will definitely be keeping my eye on it!
- necovek 5y ago
  FastAPI is an entire web framework using ASGI, whereas this is just a WSGI server. You could compare FastAPI vs FastWSGI+Flask or any other WSGI framework.
- est 5y ago
  FastAPI is a framework, the server underneath FastAPI would be uvicorn, which is also based on libuv
  - fastball 5y ago
    Actually you can use any ASGI-compatible web server[1] with FastAPI, uvicorn is generally preferred though because it is apparently the fastest. [1] https://asgi.readthedocs.io/en/latest/implementations.html https://asgi.readthedocs.io/en/latest/implementations.html
- ospider 5y ago
  I doubt that, the project name is FastWSGI, not FastASGI.
leetrout 5y ago
Would love to see uwsgi added to the benchmarks. It used to be much faster than gunicorn.
- james_roberts 5y ago
  Thanks for the feedback! Will add it shortly
  - leetrout 5y ago
    Thank you for adding it so quick! Not as big a difference as I'd expected.
fred123 5y ago
Bjoern author here. The code looks very nice, looks like it’s inspired by bjoern but much cleaner. I’ve wanted to clean up the mess that the bjoern code is for years but never came around to actually doing it. That being said I don’t think it provides a lot of value in practice. Same with bjoern. With a reasonably fast server implementation around 99% of time is spent in your Python application, not the server. So it doesn’t actually provide much value in practice to optimize the server. But it’s a nice project to learn about how to write a HTTP and WSGI server :)
- necovek 5y ago
  While you are completely right, I applaud both James and you for working to potentially remove one bottleneck. If wins are significant enough and people are not blocked by request processing anymore, we'll see them work on frameworks which are better optimized too.
- james_roberts 5y ago
  Hey, thanks! Great job on Bjoern by the way! Yeah, I did get some inspiration from Bjoern for this project. Was also a learning project for me. I used this project to pick up C and learn a bit more about HTTP! Hoping to add a few more features to the server side of things, still not finished, but I agree, there is only so much you can do to optimize things when you're making calls to Python anyway.
- whalesalad 5y ago
  This was my first thought - Python itself is always the bottleneck.
  - jacob019 5y ago
    I'm sick of everyone hating on python speed. IO is the bottleneck. By my estimate, 95% of applications are spending <5% of their response time executing python. Maybe your application is compute heavy and is better written in rust, but python gets shit done with minimal effort and the extra cycles are rarely an issue in most workloads.
Boxxed 5y ago
Cool! Looks pretty lightweight. Skimming the code, I see some uses of strtok that look unsafe...I think a URL consisting of entirely `?` will crash. Yep: Server listening at http://0.0.0.0:5000 Parse error: HPE_INVALID_URL Unexpected start char in url free(): double free detected in tcache 2 [1] 70550 IOT instruction (core dumped) (Opened an issue on github)
- james_roberts 5y ago
  Thanks so much for reporting a bug! You created the first issue for the project! I'll definitely have a look into this and push a fix
ospider 5y ago
Good choice for using llhttp, I was always wondering why everyone in the python world keeps ignoring llhttp since it's so much faster than any others. I hope to see some http client lib using llhttp.
- james_roberts 5y ago
  Thanks! Yeah, I stumbled upon it while looking for ways to parse http requests. Turns out its awesome. I believe NodeJS uses it under the hood as well. Could be mistaken...
alvarlagerlof 5y ago
Should explain what WSGI is.
- james_roberts 5y ago
  Web Server Gateway Interface. Simply put, it is just a convention for forwarding requests to Python web applications. More info here: python.org/dev/peps/pep-0333/
kartoolOz 5y ago
Tried fastwsgi + falcon cythonized. Running 1m test @ http://localhost:5000 8 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.41ms 109.36us 3.85ms 76.27% Req/Sec 8.53k 625.45 25.69k 79.77% Latency Distribution 50% 1.43ms 75% 1.48ms 90% 1.53ms 99% 1.62ms 4072168 requests in 1.00m, 454.37MB read Requests/sec: 67816.66 Transfer/sec: 7.57MB Pure fastwsgi on same machine Running 1m test @ http://localhost:5000 8 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 645.52us 30.50us 1.33ms 82.06% Req/Sec 18.67k 795.08 36.56k 86.86% Latency Distribution 50% 648.00us 75% 658.00us 90% 668.00us 99% 736.00us 8919901 requests in 1.00m, 867.68MB read Requests/sec: 148419.82 Transfer/sec: 14.44MB Damn nice.
- james_roberts 5y ago
  Awesome! That performs really well!
- james_roberts 5y ago
  The pure fastwsgi results there are insane! Are you running a single process? That's double the RPS that I've been getting on my machine.
  - kartoolOz 5y ago
    AMD 5900x, yup running a single process

89 points by james_roberts 5y ago

Those are some amazing performance metrics! What's the catch? Is there a tradeoff or did you really find a 10x efficiency?

james_roberts 5y ago
Thanks! I was quite surprised myself when I saw the numbers for the basic WSGI app. I'm still trying to figure out exactly why my server is so much faster than other servers out there... My guess is that I avoid calling Python as much as possible. Most of the parsing and the `start_response` code is all written in C instead of Python. Flask applications seem to have some bottleneck. I don't see the same performance numbers as I do with the basic WSGI app. *A note: Those tests are just "Hello World" tests on a single worker. They should be taken with a grain of salt...

james_roberts 5y ago

I've been developing a Python extension, written in C, that provides users with an ultra-fast WSGI server that they can run their WSGI applications on (Flask, Django, uWSGI etc). I have also recently managed to get it working on multiple platforms (Linux, MacOS and Windows). If you want to significantly speed up your WSGI based applications, check it out! It is still in early development at the moment. Any feedback would be greatly appreciated! === [Links] === Github: https://github.com/jamesroberts/fastwsgi https://github.com/jamesroberts/fastwsgi Pypi: https://pypi.org/project/fastwsgi/ https://pypi.org/project/fastwsgi/ Performance comparisons against other popular WSGI servers: https://github.com/jamesroberts/fastwsgi/blob/main/performance_benchmarks/PERFORMANCE.md https://github.com/jamesroberts/fastwsgi/blob/main/performan...

nerpderp82 5y ago
In real world performance it doesn't appear to be that much faster than Bjoern, which you also surmised that Flask is the bottleneck. Are there other Python WSGI frameworks that can take advantage of FastWSGI? Why is Flask as slow as it is? Since this is C, what security mitigations have you put into place?
- james_roberts 5y ago
  In theory, any framework that follows the WSGI guidelines should be able to run on top of FastWSGI and take advantage of its speed. There are many frameworks out there. I do intend to test out some more. For now I've only tested the popular Flask framework and a simple bare bones WSGI app. Flask was never developed to be lightning fast. Even still, I am quite surprised at how slow it is now that I've seen what kind of numbers can be achieved. I haven't looked deep into it to see where the issues might be. As for security, that is a work in progress... I definitely wouldn't use FastWSGI in production in the projects current state. It's still early days in terms of development.
  - byteface 5y ago
    Is it to do with the router?. I was reading this article. https://www.slideshare.net/kwatch/how-to-make-the-fastest-router-in-python https://www.slideshare.net/kwatch/how-to-make-the-fastest-ro... which lead me to this code. https://github.com/kwatch/router-sample/blob/master/minikeight.py https://github.com/kwatch/router-sample/blob/master/minikeig... and i was going to consider trying to benchmark different routers. I started a repo here which drops flask and uses some random router off pypi... https://github.com/byteface/fastwsgitest/blob/master/app.py https://github.com/byteface/fastwsgitest/blob/master/app.py and i think i just need to merge that state machine router into it for a test. I believe it's part of the architecture for templating engine called tenjin that predates even jinja? found your benchmark repo but not installed wrk yet to test.
vangelis 5y ago
FYI, you have a typo in the graph `Requests served in 60 seconds`. server'd.
- deleted 5y ago
  [deleted]
- james_roberts 5y ago
  Thanks, nice catch! I will update that.
necovek 5y ago
With Python, benchmarks that matter are what's the maximum performance you can get out of a particular hardware in the most optimal configuration for the tool in use. (This was most obvious with the async framework benchmarks) Perhaps other WSGI frameworks are not achieving 100% CPU load and FastWCGI is, and you could easily run them with multiple worker processes to get the same CPU load and comparable performance? (this is just wild speculation) I don't know, but that's the kind of benchmark I'd like to see: eg. what's the maximum performance you can get out of a, say, 4-core CPU with each of them with whatever configuration stresses the CPU completely, and what are the other metrics you might be seeing (eg. asyncio will basically net you lower memory use, but not any better performance in RPS)? It'd also be good if the benchmark tool is not running on the same CPU though, as long as you've good a sufficiently fast interconnect.
- necovek 5y ago
  Not to be taken badly, results you post are amazing and point at a greatly optimised request handling! But with these benchmarks you do not demonstrate the practical value over any competing tool (just that your worker threads can do more in the same time, which is more of the "good engineering" pat-on-the-back type of thing :).
  - james_roberts 5y ago
    Thanks for the feedback! Yeah, the benchmarks just highlight higher numbers in very simple tests. They should definitely be taken with a grain of salt. I can try add some more practical highlights.
- james_roberts 5y ago
  Yeah I don't have an optimal benchmarking setup currently. I have a 16-core CPU that I run everything off (server + benchmarks = not ideal). I've been working on adding multiprocessing to FastWSGI (only works on Linux at the moment). The RPS almost scales linearly with the number of workers. At 4 processes it hits +260k RPS with decent CPU utilization. I can definitely add more details to the benchmarks. Thanks for the feedback!
barbazoo 5y ago
> provides users with an ultra-fast WSGI server that they can run their WSGI applications on (Flask, Django, uWSGI etc) I thought this was an equivalent to uWSGI, what's the benefit of running uWSGI on top of FastWSGI?
- james_roberts 5y ago
  Oops. This is a typo... should be WSGI apps. Not the uWSGI server
zerkten 5y ago
It would be good to eventually see some comparisons running some average/unoptimized code. IME benchmarks seem to focus on either the very basics, or select areas where they are faster than other apps. This is important to cover, but having something that's closer to a real app is more convincing, even if the performance margins drop somewhat. To come back to the average code. I may start an app and try to optimize things as much as reasonable. Eventually, I'm only able to focus on the functionality and a slide in performance can happen. As more developers are added, this can happen more quickly in the average organization that focuses primarily on functionality. Of course, we should analyze the perf problems and improve the code, but projects like yours may offer a huge perf boost for teams that are struggling here. I love Python, but I've always enjoyed that bit more scope that the .NET platform provided around performance when code was less optimal. If you can really speed up WSGI this much it'll be a huge boon.
- james_roberts 5y ago
  Thanks for the feedback! Yeah, some more "real world" benchmarks should be added. And yes, if you have some application that you've tried to optimize somewhat and haven't managed to get the desired performance you would like, or if you simply haven't had the time to re-write components to be faster, ideally you could use FastWSGI as a drop in replacement for your current WSGI server and get the extra perf boost for "free". It's still in early development, but this is ultimately one of the main goals of the project.
webmobdev 5y ago
Looks great, thanks. I was going through the Performance Benchmarks and noticed that the mod_wsgi Apache module - https://github.com/GrahamDumpleton/mod_wsgi https://github.com/GrahamDumpleton/mod_wsgi - is missing? Please consider including it in the benchmark too - would love to see how your module matches against it.
- james_roberts 5y ago
  Will do! Thanks for the feedback

dabeeeenster 5y ago

Would love to see benchmarks comparing this to FastAPI? Thanks for this project - will definitely be keeping my eye on it!

necovek 5y ago
FastAPI is an entire web framework using ASGI, whereas this is just a WSGI server. You could compare FastAPI vs FastWSGI+Flask or any other WSGI framework.
est 5y ago
FastAPI is a framework, the server underneath FastAPI would be uvicorn, which is also based on libuv
- fastball 5y ago
  Actually you can use any ASGI-compatible web server[1] with FastAPI, uvicorn is generally preferred though because it is apparently the fastest. [1] https://asgi.readthedocs.io/en/latest/implementations.html https://asgi.readthedocs.io/en/latest/implementations.html
ospider 5y ago
I doubt that, the project name is FastWSGI, not FastASGI.

leetrout 5y ago

Would love to see uwsgi added to the benchmarks. It used to be much faster than gunicorn.

james_roberts 5y ago
Thanks for the feedback! Will add it shortly
- leetrout 5y ago
  Thank you for adding it so quick! Not as big a difference as I'd expected.

fred123 5y ago

Bjoern author here. The code looks very nice, looks like it’s inspired by bjoern but much cleaner. I’ve wanted to clean up the mess that the bjoern code is for years but never came around to actually doing it. That being said I don’t think it provides a lot of value in practice. Same with bjoern. With a reasonably fast server implementation around 99% of time is spent in your Python application, not the server. So it doesn’t actually provide much value in practice to optimize the server. But it’s a nice project to learn about how to write a HTTP and WSGI server :)

necovek 5y ago
While you are completely right, I applaud both James and you for working to potentially remove one bottleneck. If wins are significant enough and people are not blocked by request processing anymore, we'll see them work on frameworks which are better optimized too.
james_roberts 5y ago
Hey, thanks! Great job on Bjoern by the way! Yeah, I did get some inspiration from Bjoern for this project. Was also a learning project for me. I used this project to pick up C and learn a bit more about HTTP! Hoping to add a few more features to the server side of things, still not finished, but I agree, there is only so much you can do to optimize things when you're making calls to Python anyway.
whalesalad 5y ago
This was my first thought - Python itself is always the bottleneck.
- jacob019 5y ago
  I'm sick of everyone hating on python speed. IO is the bottleneck. By my estimate, 95% of applications are spending <5% of their response time executing python. Maybe your application is compute heavy and is better written in rust, but python gets shit done with minimal effort and the extra cycles are rarely an issue in most workloads.

Boxxed 5y ago

Cool! Looks pretty lightweight. Skimming the code, I see some uses of strtok that look unsafe...I think a URL consisting of entirely `?` will crash. Yep: Server listening at http://0.0.0.0:5000 Parse error: HPE_INVALID_URL Unexpected start char in url free(): double free detected in tcache 2 [1] 70550 IOT instruction (core dumped) (Opened an issue on github)

james_roberts 5y ago
Thanks so much for reporting a bug! You created the first issue for the project! I'll definitely have a look into this and push a fix

ospider 5y ago

Good choice for using llhttp, I was always wondering why everyone in the python world keeps ignoring llhttp since it's so much faster than any others. I hope to see some http client lib using llhttp.

james_roberts 5y ago
Thanks! Yeah, I stumbled upon it while looking for ways to parse http requests. Turns out its awesome. I believe NodeJS uses it under the hood as well. Could be mistaken...

alvarlagerlof 5y ago

Should explain what WSGI is.

james_roberts 5y ago
Web Server Gateway Interface. Simply put, it is just a convention for forwarding requests to Python web applications. More info here: python.org/dev/peps/pep-0333/

kartoolOz 5y ago

Tried fastwsgi + falcon cythonized. Running 1m test @ http://localhost:5000 8 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.41ms 109.36us 3.85ms 76.27% Req/Sec 8.53k 625.45 25.69k 79.77% Latency Distribution 50% 1.43ms 75% 1.48ms 90% 1.53ms 99% 1.62ms 4072168 requests in 1.00m, 454.37MB read Requests/sec: 67816.66 Transfer/sec: 7.57MB Pure fastwsgi on same machine Running 1m test @ http://localhost:5000 8 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 645.52us 30.50us 1.33ms 82.06% Req/Sec 18.67k 795.08 36.56k 86.86% Latency Distribution 50% 648.00us 75% 658.00us 90% 668.00us 99% 736.00us 8919901 requests in 1.00m, 867.68MB read Requests/sec: 148419.82 Transfer/sec: 14.44MB Damn nice.

james_roberts 5y ago
Awesome! That performs really well!
james_roberts 5y ago
The pure fastwsgi results there are insane! Are you running a single process? That's double the RPS that I've been getting on my machine.
- kartoolOz 5y ago
  AMD 5900x, yup running a single process