5 ms·

Bjoern author here. The code looks very nice, looks like it’s inspired by bjoern but much cleaner. I’ve wanted to clean up the mess that the bjoern code is for

Bjoern author here. The code looks very nice, looks like it’s inspired by bjoern but much cleaner. I’ve wanted to clean up the mess that the bjoern code is for years but never came around to actually doing it. That being said I don’t think it provides a lot of value in practice. Same with bjoern. With a reasonably fast server implementation around 99% of time is spent in your Python application, not the server. So it doesn’t actually provide much value in practice to optimize the server. But it’s a nice project to learn about how to write a HTTP and WSGI server :)

necovek 5y ago
While you are completely right, I applaud both James and you for working to potentially remove one bottleneck. If wins are significant enough and people are not blocked by request processing anymore, we'll see them work on frameworks which are better optimized too.
james_roberts 5y ago
Hey, thanks! Great job on Bjoern by the way! Yeah, I did get some inspiration from Bjoern for this project. Was also a learning project for me. I used this project to pick up C and learn a bit more about HTTP! Hoping to add a few more features to the server side of things, still not finished, but I agree, there is only so much you can do to optimize things when you're making calls to Python anyway.
whalesalad 5y ago
This was my first thought - Python itself is always the bottleneck.
- jacob019 5y ago
  I'm sick of everyone hating on python speed. IO is the bottleneck. By my estimate, 95% of applications are spending <5% of their response time executing python. Maybe your application is compute heavy and is better written in rust, but python gets shit done with minimal effort and the extra cycles are rarely an issue in most workloads.
  - deleted 5y ago
    [deleted]
  - simfree 5y ago
    We had a vendor who implemented their stack with Flask and Postgres on Debian. Their API is consistently slow (seconds to tens of seconds) to the point that we wrote our own app in Dotnet Core (running atop Postgres and Debian) that queries the available content once a day (500k rows of data) with minor refreshes hourly. We take tens of milliseconds to query Postgres and generate a rendered HTML page for our clients. Showing this to the vendor's devs we got a very surprised response. Admittedly, we do not operate at their scale, but I am certain this $5 a month droplet will keep running this app for a long time yet even with many users :) Edit: I did write an MVP in Python atop Sqlalchemy wrapping Postgres, but the performance was still not ideal when rendering hundreds or thousands of rows of data, and the primary developer was already using Dotnet Core.
    - cmclaughlin 5y ago
      I’m not familiar with dotnet, but I’m not sure if blaming Python is the problem. A more even comparable rewrite would have been FastApi with an asynchronous library for Postgres (such as SQL Alchemy or TortoiseORM). There are probably ways to achieve similar results with Django or Flask, but it’s pretty easy with FastApi.
      - necovek 5y ago
        To any experienced Python dev, it's obvious from their description of what the problem is. And it's understandable that anyone inexperienced with Python would blame Python. They were returning a large number of rows from Postgres (which, if the DB is properly set up, should take at most tens of ms: of course, depending on the width of the rows too), and most (well, I know of none that don't) Python ORM libraries (SQLAlchemy included) have a huge "serialization" cost (turning raw data from Postgres into objects). I've done a benchmark once, and things like Django-ORM or SQLAlchemy were like 10-50x slower than fetching tuples with psycopg directly. SQLAlchemy-core was fastest when fetching tuples if you wanted to not do raw SQL (IIRC, a performance penalty of at most 100%, translated to a factor, up to 2x slower), but Django's fetch-me-tuples functionality was also a single digit multiple of psycopg. So, the solution to that problem is to fetch tuples, and then pass them in for rendering the page. Of course, this also points at the problem with all the ORM implementations in Python: they are being too "smart" and dynamic for their own good (if all are bad at it, it also means that Python is not doing something good either, so criticism is warranted).
- necovek 5y ago
  That's not what was said. Python application code is the bottleneck. As evident from the benchmarks for FastWSGI, even Flask the framework is a bottleneck: pure Python vs Flask went from 70k RPS to 9k RPS. Python does have a huge performance penalty for basic computation, which is why it has a bunch of C-based libraries that provide bulk-operations that avoid it. If properly used (you rarely need to roll your own with a number of compiled libraries present), Python itself is not a bottleneck. One can argue if that's still Python, but at the very least, it's idiomatic Python development: I hope you don't use Python to prove that pure dynamic languages can outperform compiled languages, but to develop and deliver applications faster using Python's expressiveness. People bring up GIL as well: it will affect your application startup time and memory usage since you can trivially avoid it by running multiple Python processes. But performance of executing code itself will only be minimally affected if you switch to multi-processing (of course, if memory pressure is so high that all those Python libraries loaded multiple times in memory is affecting your app, that can hinder performance, but that's going to be pretty rare).

necovek 5y ago

While you are completely right, I applaud both James and you for working to potentially remove one bottleneck. If wins are significant enough and people are not blocked by request processing anymore, we'll see them work on frameworks which are better optimized too.

james_roberts 5y ago

Hey, thanks! Great job on Bjoern by the way! Yeah, I did get some inspiration from Bjoern for this project. Was also a learning project for me. I used this project to pick up C and learn a bit more about HTTP! Hoping to add a few more features to the server side of things, still not finished, but I agree, there is only so much you can do to optimize things when you're making calls to Python anyway.

whalesalad 5y ago

This was my first thought - Python itself is always the bottleneck.

jacob019 5y ago
I'm sick of everyone hating on python speed. IO is the bottleneck. By my estimate, 95% of applications are spending <5% of their response time executing python. Maybe your application is compute heavy and is better written in rust, but python gets shit done with minimal effort and the extra cycles are rarely an issue in most workloads.
- deleted 5y ago
  [deleted]
- simfree 5y ago
  We had a vendor who implemented their stack with Flask and Postgres on Debian. Their API is consistently slow (seconds to tens of seconds) to the point that we wrote our own app in Dotnet Core (running atop Postgres and Debian) that queries the available content once a day (500k rows of data) with minor refreshes hourly. We take tens of milliseconds to query Postgres and generate a rendered HTML page for our clients. Showing this to the vendor's devs we got a very surprised response. Admittedly, we do not operate at their scale, but I am certain this $5 a month droplet will keep running this app for a long time yet even with many users :) Edit: I did write an MVP in Python atop Sqlalchemy wrapping Postgres, but the performance was still not ideal when rendering hundreds or thousands of rows of data, and the primary developer was already using Dotnet Core.
  - cmclaughlin 5y ago
    I’m not familiar with dotnet, but I’m not sure if blaming Python is the problem. A more even comparable rewrite would have been FastApi with an asynchronous library for Postgres (such as SQL Alchemy or TortoiseORM). There are probably ways to achieve similar results with Django or Flask, but it’s pretty easy with FastApi.
    - necovek 5y ago
      To any experienced Python dev, it's obvious from their description of what the problem is. And it's understandable that anyone inexperienced with Python would blame Python. They were returning a large number of rows from Postgres (which, if the DB is properly set up, should take at most tens of ms: of course, depending on the width of the rows too), and most (well, I know of none that don't) Python ORM libraries (SQLAlchemy included) have a huge "serialization" cost (turning raw data from Postgres into objects). I've done a benchmark once, and things like Django-ORM or SQLAlchemy were like 10-50x slower than fetching tuples with psycopg directly. SQLAlchemy-core was fastest when fetching tuples if you wanted to not do raw SQL (IIRC, a performance penalty of at most 100%, translated to a factor, up to 2x slower), but Django's fetch-me-tuples functionality was also a single digit multiple of psycopg. So, the solution to that problem is to fetch tuples, and then pass them in for rendering the page. Of course, this also points at the problem with all the ORM implementations in Python: they are being too "smart" and dynamic for their own good (if all are bad at it, it also means that Python is not doing something good either, so criticism is warranted).
necovek 5y ago
That's not what was said. Python application code is the bottleneck. As evident from the benchmarks for FastWSGI, even Flask the framework is a bottleneck: pure Python vs Flask went from 70k RPS to 9k RPS. Python does have a huge performance penalty for basic computation, which is why it has a bunch of C-based libraries that provide bulk-operations that avoid it. If properly used (you rarely need to roll your own with a number of compiled libraries present), Python itself is not a bottleneck. One can argue if that's still Python, but at the very least, it's idiomatic Python development: I hope you don't use Python to prove that pure dynamic languages can outperform compiled languages, but to develop and deliver applications faster using Python's expressiveness. People bring up GIL as well: it will affect your application startup time and memory usage since you can trivially avoid it by running multiple Python processes. But performance of executing code itself will only be minimally affected if you switch to multi-processing (of course, if memory pressure is so high that all those Python libraries loaded multiple times in memory is affecting your app, that can hinder performance, but that's going to be pretty rare).