20 ms·

12 requests per second: A realistic look at Python web frameworks

503 points by gilad 6y ago

gchamonlive 6y ago
I have great experiences with Falcon for backend REST APIs, and it is supposed to be great in terms of requests per second. How does it compare to Sanic?
- divbzero 6y ago
  > I have great experiences with Falcon Then keep using Falcon! OP’s core thesis is that developer comfort matters more when choosing an API framework because the performance bottleneck is usually found elsewhere. The API itself should be fast enough with some combination of Gunicorn, gevent, PyPy, and horizontal scaling.
  - gchamonlive 6y ago
    I just wanted to know how it would compare with Sanic, because I never used it.
    - divbzero 6y ago
      The biggest difference is that Falcon is synchronous while Sanic is asynchronous. With Sanic, you would explicitly specify async/await for asynchronous operations and use asynchronous libraries for I/O. Switching to Sanic could also affect how you deploy to production. Both are plenty fast. FastAPI [1] is also worth considering if you’re looking into asynchronous API frameworks. It comes with nice features for specifying API schemas. [1]: https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/
      - gchamonlive 6y ago
        I prefer async big time. We once had to implement async routines while using flask, where the server would return a 200 but keep processing the request, and the actual result would be sent by email. It was hellish and inefficient to make it. In hindsight would have been better to use a queue service and a consumer and decouple the whole process, even if it meant increased infrastructure and maintenance complexity.
- huffmsa 6y ago
  I'm a big Falcon user as well. Got into it because it was easier to get up and running without bells and whistles vs the more "popular" frameworks.
est 6y ago
Might as well refer to TechEmpower benchmarks. https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/
- Jarred 6y ago
  just.js (#9) looks interesting https://github.com/just-js/just https://github.com/just-js/just It seems to be a much tinier JavaScript runtime than Node.js (still using v8), but linux only The benchmark is probably unrealistically optimized code but even so, it implies Node.js itself has a large performance overhead
  - chmod775 6y ago
    If you're not paying attention it would look that way, yes. If you pay attention you'll notice that just.js is using postgres as their DB, while all of the node benchmarks are handicapped by either using mongoose/mongodb or MySQL. There is no node benchmark with postgres, but all of the fastest benchmarks used it. "lithium" is a good example to show how much of an impact switching to postgres has. All of the 4 lithium benchmarks are identical except in what DB they use. The results are: lithium-postgres-batch (#2 - 659850), lithium-postgres-beta (#13 - 398773), lithium-postgres (#14 - 398258), lithium (#45 - 271989). The last result is MySQL.
    - reasonabl_human 6y ago
      What makes postgres special in these cases?
      - chmod775 6y ago
        I would like to know as well. All I know is that for this specific workload there appears to be a strong correlation between the chosen DB and performance. Depending on the benchmark you're looking at on that site, the top 30-60 results are exclusively postgres. Further, for every framework/language that was tested with both MySQL and postgres (there's quite a few of them), the postgres one always ranks higher.
Laminary 6y ago
In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?
- rovr138 6y ago
  Where is SSL the bottleneck? Wondering if terminating earlier and just relying on HTTP after would help.
  - wahern 6y ago
    Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. Session caching can be an important optimization that can significantly reduce those costs, but scaling session caching has its own problems. But IME real-world bottlenecks have more to do with overall architecture. People tend to heavily focus on technical details, such as concurrency architecture--the how. But the biggest opportunities for improved performance usually involve functional aspects of an architecture--the what. (Note that these aren't fixed categories; they're relative positions. A technical detail often becomes a functional model as development progresses.) 12 RPS is a long way from implicating SSL. If you get to the point where SSL is an identifiable bottleneck, you've either made a series of tremendously good decisions or exceptionally poor decisions.
    - toast0 6y ago
      > Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. I don't think that's right. Cloudflare's blog [1] says they can do about 9.5x the handshakes/sec with ECDSA at 256-bits vs RSA at 2048. Verification for ECDSA signatures are somewhat slower, but it's usually an acceptable tradeoff to make clients do a bit more work so that servers do a lot less. I agree though, at 12 RPS, TLS isn't the bottleneck. [1] https://blog.cloudflare.com/ecdsa-the-digital-signature-algorithm-of-a-better-internet/ https://blog.cloudflare.com/ecdsa-the-digital-signature-algo...
ramraj07 6y ago
Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html https://docs.sqlalchemy.org/en/13/faq/performance.html If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute(<raw query>). Makes a huge difference. Next optimization I do is create stored procedures on the database. Only then I start thinking about changing the framework itself. For folks like me who want to get prototypes off the ground in hours, flask and fastapi are godsend, and if that means I have to worry about serving thousands of requests a second soon thats a happy problem for sure.
- nwsm 6y ago
  For me, FastAPI is the sweet spot between performance and quick development.
- vosper 6y ago
  You can also use SQLAlchemy Core, which is an intermediate between the full-blown ORM and running actual strings of SQL. I've had a great experience with Core - I can easily have it output essentially the exact SQL I'd write by hand, but I get many benefits (like the ability to compose queries) that are nicer than dealing with raw SQL.
  - SatvikBeri 6y ago
    Definitely agree, I'm happy to write raw SQL, but SQLAlchemy Core is even better than that because of composability.
  - smoe 6y ago
    This is what I like most about SQLAlchemy, I can use the ORM most of the time, but in cases where it needs optimizations or aggregation queries I can drop down or refactor to core, while still using the same model/table definitions, ability to combine or filter queries dynamically and leverage IDE features like refactoring.
- welder 6y ago
tnash 6y ago
Use of ORMs is often a performance choke point. Raw DB queries are often much, much faster. Almost always, the more you abstract, the worse you perform. It's great as a developer but not so great as a user.
- nmfisher 6y ago
  I haven't touched an ORM in over 6 years, but unless they've improved since then, I honestly can't think of a single reason why anyone would choose to use one. They're clunky monstrosities that act only as guard-rails for inexperienced developers. Far better to invest a few days (which is realistically all you need) to improve their SQL skills and/or code-review practices.
  - morpheuskafka 6y ago
    I am by no means an experienced developer--but the issue I always seem to have without using an ORM is that there are string literals containing different bits of SQL scattered all throughout my code. In addition to being hard to refactor when the database model changes, I think there is a fairly high performance cost to doing so many string concatenations every time the code runs. Is there a better way to manage the SQL command strings when you are sending the queries by hand?
    - branko_d 6y ago
      A lot of times you don't need to change the SQL string at all. You just bind different parameter values before sending the same SQL string to the DB server over and over again. The exceptions are SQL elements that cannot be (easily, or at all) parametrized, such as the column lists in SELECT, ORDER BY, GROUP BY, or changing the WHERE "shape" and so on. In my experience, this tends to be a minority of queries, although an important minority. ---- P.S. If string concatenations are your bottleneck, then your database is screaming fast! The real-life bottlenecks are usually in excessive database round-trips and unoptimized query plans, and are orders-of-magnitude larger.
    - takeda 6y ago
imperio59 6y ago
My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure, the framework matters at very large scale and the benefits from optimizing the framework become large when you're doing millions of requests a second over many thousands of servers because it can help reduce baseline cost of running the service. But I agree with the author's main point which seems to be that framework performance is pretty meaningless when comparing frameworks if you're just starting on a new project. Focus on making a product people wanna actually use first. If you're lucky enough to get to scale you can work about optimizing it then.
- Izkata 6y ago
  > (usually people writing dumb code without realizing it's expensive) Some years ago, one morning I gave a co-worker a recommendation on how to improve a loop that was unnecessarily hitting database through the Django ORM. He committed the fix that afternoon. Barely an hour later I accidentally reintroduced the exact same slowdown in the exact same loop when adding a different piece of data to it. Soooo yeah, ORMs can be so simplistic it's too easy to do by accident even if you know exactly what's going on under the hood.
  - solox3 6y ago
    Something about select_related? Please do share.
    - DangitBobby 6y ago
      My guess is accessing a related field within a loop causing a database request per iteration, e.g. ``` [book.author.name for book in Book.objects.all()] ```
      - 6y ago
maxpert 6y ago
The fact that you are using offset of 50000 and complaining it slows everything down says a lot about the benchmarks. Top it all with ORM query with prefetch all, GIL, and shared CPU (I am guessing) that you used to run benchmark on. You see where this is headed?
mlthoughts2018 6y ago
I think it’s been bog standard practice to run flask via uwsgi or gunicorn with async workers and use multiple process based workers per deployed server unit (eg per pod in Kubernetes). What matters is that the cumulative latency & throughput solve your problem, not how fast you can make one singular async worker thread. I figure most people running complex web services in production would just do an eye roll at this post. Nobody's going to switch to PyPy for any of this. My team at work runs several complex ML workloads, and we use the exact same container pattern for every service running gunicorn to spawn X async workers per pod and then scale pods per service to meet throughput requirements. Sometimes we also just post complex image processing workloads to a queue and batch them to GPU processor workers. In all these use cases, super low effort “just toss it in gunicorn running flask” has worked without issue for services supporting up to peak load of thousands to hundreds of thousands of requests per second.
- 7kmph 6y ago
  Could you share your company’s website?
  - mlthoughts2018 6y ago
    No, I can’t speak on their behalf on Hacker News, so it is important to me to stay disconnected from my employer.
jaimex2 6y ago
When you start hitting bottlenecks in your python web framework its probably time to switch to a faster language, not another framework in python. You're probably done rapid prototyping by this point anyway.
- speedgoose 6y ago
  I don't think there is much gains to rewrite everything in a faster language. Unless they are a very very successful company with billions of customers, it's often cheaper to scale horizontally.
  - seanparsons 6y ago
    Depends on if you identify it early enough really and even then halving your app tier costs could be sizable for any company. If you can get some productivity benefits at the same time from say static typing then there's even more reason to switch.
- square_usual 6y ago
  There's still a good reason to pick fast frameworks in a slow language: you can delay the inevitable for a bit, probably enough time for you to work on a rewrite or whatever.
- antihero 6y ago
  Well if you convert everything from ORM to raw SQL, it will then be easier to extract all of that SQL and use it in a different web framework once you've measured and confirmed that your bottleneck is servicing requests in Python.
forgotmypw17 6y ago
I have to question the value of text written by someone who sets white-on-white text in their website...
- Jonnax 6y ago
  The background is blue?
  - forgotmypw17 6y ago
    The background image is blue, the background itself is white. Without loading the image, the text is the same color.
    - throwawayffffas 6y ago
      I question the value of the comment of someone who loads css but not background images.
sriku 6y ago
A humble request to folks making benchmark or other graphs - please understand that thin coloured lines are not easy to visually parse .. even for folks like me who aren't totally colour blind but have partial red-green colour blindness. At least, the lines can be made thicker so it is easier to make out the colours. Even better, label the lines with an arrow and what they represent.
- TickleSteve 6y ago
  Absolutely! I have mild red/green issues and could barely tell those lines apart.
ximm 6y ago
Maybe I am missing something, but why wasn't sanic tested with pypy? I expect that this combination would outperform everything else.
- erdewit 6y ago
  Was wondering the same thing. PyPy can do wonders for other asyncio-based frameworks.
sandGorgon 6y ago
TLDR - pypy is awesome. Dont use frameworks. Use pypy. Please donate. Pypy needs funds - https://opencollective.com/pypy https://opencollective.com/pypy Pypy doesnt get a fraction of the funding that python does.
cryptos 6y ago
Why Python at all? About 10 years ago I liked Python a lot (and still like it in principle) and felt very productive compared to, say, Java. Java was full of inconvenience, XML, bloated frameworks and all that. But today you can use Kotlin, that is in my opinion even nicer than Python, with performant frameworks (e. g. Quarkus or Ktor) on the super fast JVM. I don't want to start a language war, but maybe Python is not the first choice for their requirements.
- galacticdessert 6y ago
  I feel like when you need to create value and make a product, better to do it in a language you know. Product value is not directly proportional to product performance or raw HTTP request serving time. I for instance know a bit of F#/C# and Java, but would probably pick Python to make a new product just to remove that mental barrier of not having my lack of language knowledge in the way of things
- kavalg 6y ago
  I can code comfortably in Python, Java, JavaScript and to some extent C/C++. In the last 4 years I have been using mostly Python for various reasons (Machine Learning, OS automation, web scraping ...). Compared to the other languages, Python feels lighter and faster to write to the extent that it rarely interrupts my flow of thoughts. Now and then I have to code in JavaScript (frontend), C/C++ (embedded / low level optimizations) and Java (maintenance) and my feelings are: - JavaScript still feels messy - C/C++ is complex, but it is often offset by the complexity/needs of the project (e.g. in embedded) - Java has become kind of bloated with all the new stuff So as others have already mentioned, development productivity is in many cases far more important than speed of code. In my ~20y long career I have rarely seen a project failure due to runtime performance. Most of them failed due to speed/agility of development iterations and also project/product management issues (bad fit, unrealistic project plan, lack of focus and customer feedback). That said, if I need to look for another language due to performance, that would probably be Rust.
  - huffmsa 6y ago
    I also notice I spend less time thinking "how do I do X in this language" with Python and just do it. Maybe not the best implementation of X, but that doesn't really matter because I implemented X, Y and Z while someone else got bogged down in their "better" language and barely got X across the finish line.
hgretg3443 6y ago
C#/ASP.NET is the fastest web framework now: https://www.techempower.com/benchmarks/#section=test&runid=8ca46892-e46c-4088-9443-05722ad6f7fb&hw=ph&test=plaintext https://www.techempower.com/benchmarks/#section=test&runid=8... 7.000.000 requests per second Even GO can only achieve 4.500.000 million requests per secnod being a low-level language, in opposite to high-level C#.
- sgtnoodle 6y ago
  The #1 project on that list appears to be implemented in rust? What does GO being "low-level" have to do with performance of serving requests? I'd imagine its bottlenecks would be due to something fairly arbitrary, like its garbage collection or how it represents strings or something.
- nodamage 6y ago
  That 7 million requests per second is achieved by writing a hard coded plain text HTTP response string directly to the client... it is so far disconnected from any real world use case that the number is basically meaningless.
  - arcturus17 6y ago
    It's still 8th in the composite benchmark. And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? [1] https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=composite https://www.techempower.com/benchmarks/#section=data-r20&hw=...
    - BiteCode_dev 6y ago
      Yes but getting a good ranking at a useless benchmark if not very meaningful. It's cool, but I wouldn't use it to make any serious decision.
    - nodamage 6y ago
      > And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? Indeed. The problem is that many of the scores in the top 100 are really misleading because no one building a web app would implement things that way. There is still some value in the lower down benchmarks but you have to basically read the underlying source to determine if the implementation is remotely realistic or not. For starters I would ignore anything classified as "platform" which is described as: Platform, meaning a raw server (not actually a framework at all). Good luck! You're going to need it. For C# in particular I would only consider the mvc variants as realistic. Edit: I looked into the "asp.net core" composite score a bit more, it looks like those benchmarks are based on the aforementioned "platform" implementations for each test. I actually think this score is even more misleading than the individual benchmarks. At least the individual benchmarks show you the difference between "aspcore" (platform), "aspcore-mw" (middleware-only), and "aspcore-mvc" (full framework with routing). Here are recalculated composite scores based on the more realistic implementations (aspcore-mvc, aspcore-mvc-ado-pg, aspcore-mvc-dap-pg, aspcore-mvc-ef-pg): ASP.NET Core MVC with ADO.NET (raw SQL): 3029 ASP.NET Core MVC with Dapper: 2591 ASP.NET Core MVC with Entity Framework: 2195 Compared to Flask's 468 or Django's 280 it's still significantly faster, but not to the same extreme you might think at first glance at the chart.
KaiserPro 6y ago
We did an evaluation for our API. The API accepts an image upload, passes it onto the backend for processing and returns a ~2k json lump in return. Long story short, fastapi was much much faster than anything else for us. It also felt a bit like flask. The integration with pydantic for validating dataclasses on the fly was also great.
helsinkiandrew 6y ago
Since I've been a developer there have been two changes that I feel have given major performance improvements and made backend framework improvements much less significant (atleast in the apps I develop): CDNs and client side rendering (that means the more, smaller requests for data which are more suited to be served via a CDN) Using (for example) AWS Cloudfront was a gamechanger in how I design webapps and view performance. Being able to 'slice and dice' what requests get SSL terminated at the CDN, cached fairly locally, served from an Amazon managed webserver, or sent to our app server, increased our performance 10 fold. That approach isn't always practical, but I find that it's now much easier to choose the backend for developer performance and doubling the server CPU/memory is quicker and cheaper when needed.
qeternity 6y ago
As a Django shop, we’ve always hoped PyPy would one day be suitable for our production deployments but in the end with various issues we were never able to make the switch. And then Pyston was re-released...and changed everything. It was drop in compatible for us and we saw a 50% drop in latencies. Source availability aside, I suggest anyone running CPython in prod take a look.
- twsted 6y ago
  Can you tell us more about that? I remember pyston v1 from Dropbox. You are speaking about v2, which is a binary package (closed-source at the moment)?
  - qeternity 6y ago
    Yeah it's a closed source binary. We're very happy with it. Great compatibility, no horrendous warm up times, and very meaningful speedups. Not much more to say. It's the same, just a bit faster.
robertlagrant 6y ago
Note: SQLAlchemy 1.4 is async. https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#asynchronous-io-support-for-core-and-orm https://docs.sqlalchemy.org/en/14/changelog/migration_14.htm...
lrossi 6y ago
This benchmark was run on a laptop, which has a very small number of cores compared to the servers that usually run such apps. The author doesn’t mention any attempt to tweak the number of workers, which would make sense in this case. Given that they did notice at some point that CPU usage is lower than expected, I am surprised that they did not try it.
throwdbaaway 6y ago
Good article, but I can't help but notice a gaping hole in the benchmark -- why was there no attempt to run gunicorn in multi-threaded mode? The article has a link to https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a..., but failed to mention the key takeaway from the article: > threaded code got the job done much faster than asyncio in every case
- driverdan 6y ago
  The article explicitly says they were testing single threaded use cases.
  - throwdbaaway 6y ago
    But why? The results would be a lot more useful if they include a single-process multi-threaded setup in the benchmark.
hendry 6y ago
I always assumed Python could scale because of Reddit: https://github.com/reddit-archive/reddit https://github.com/reddit-archive/reddit Not quite sure if their current site's code is opensource... anyone know?
- andsko 6y ago
  And because of Instagram: https://instagram-engineering.com/python-at-scale-strict-modules-c0bb9245c834 https://instagram-engineering.com/python-at-scale-strict-mod...
- segmondy 6y ago
  Any language can scale. Pretty much most languages there can handle scale. It's usually bad algorithms or external services be it API & DB used poorly that impacts performance.
- pas 6y ago
  Facebook uses/used PHP, that scales too. If you make it stateless and use it only as a glue. Dropbox uses Python too. Eve Online used Stackless Python. Of course Reddit/FB/EVE/Dropbox shards everything, there's no global state to manage via Python. The state lives in the data store layer. And for that there are these monstrous/elegant things like Vitess, that YouTube used (uses?): https://vitess.io/docs/overview/architecture/ https://vitess.io/docs/overview/architecture/ which is basically a sharding/routing layer on top of independent MySQL instances.
sdevonoes 6y ago
Interesting. I never heard of Japronto before. For the people working with Python: why Flask instead of Japronto?
- BiteCode_dev 6y ago
  The ecosystem. You got loads of libraries built around flask, and a lot of doc and tutorials, not to mention how much battle tested it is. Those can matter much more than hypothetical perfs in a synthetic benchmark.
p5v 6y ago
The TL;DR you should be looking for: > all of this emphasises the fact that unless you have some super-niche use-case in mind, it's actually a better idea to choose your framework based upon ergonomics and features, rather than speed.
dfgdghdf 6y ago
I would question choosing Python for large server projects because the performance ceiling is so low. At least with the "middle tier" performance languages such as Java / C# you are unlikely to require a complete language switch as the project scales.
- musingsole 6y ago
  With cloud and infinite horizontal scaling...hitting that performance ceiling is an indication of a poor design.
  - dfgdghdf 6y ago
    This is false, since it is not always easy (or even possible) to horizontally scale a work-load.
    - musingsole 6y ago
      If you've made it impossible or just difficult to scale horizontally in the server world, yeah, you've got a bad design.
      - dfgdghdf 6y ago
        Sorry to keep coming back, I don't want to start a flame war here. Difficulty of horizontal scaling is a property of two things: 1. how your solution is designed (your point, and I agree this is often done poorly) 2. the problem / work-load you are trying to scale (my point). It might be that your problem does not "shard" very easily. You cannot fix this with solution architecture, at least not easily. Horizontal scaling of a relational database is very difficult, for example. Edit: Another example. Can you rewrite NGINX in a slower language and use horizontal scaling to fix it? Of course not, because that horizontal scaling would itself leverage NGINX (or something like it)!
        mixmastamyk 6y ago
        There are lots of ways to speed python if need be, and you can recover much of the work already done. Cython is another.
polyrand 6y ago
Related to ORMs/queries/performance, I have found the following combination really good: * aiosql[0] to write raw SQL queries and having them available as python functions (discussed in [1]) * asyncpg[2] if you are using Postgres * Map asyncpg/aiosql results to Pydantic[3] models * FastAPI[4] Pydantic models become the "source of truth" inside the app, they are designed as a copy of the DB schema, then functions receive and return Pydantic models in most cases. This stack also makes me think better about my queries and the DB design. I try to make sure each endpoint makes only a couple of queries. Each query may have multiple CTEs, but it's still only a single round-trip. That also makes you think about what to prefetch or not, maybe I want to also get the data to return if the request is OK and avoid another query. [0] https://github.com/nackjicholson/aiosql https://github.com/nackjicholson/aiosql [1] https://news.ycombinator.com/item?id=24130712 https://news.ycombinator.com/item?id=24130712 [2] https://github.com/MagicStack/asyncpg https://github.com/MagicStack/asyncpg [3] https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/ [4] https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/
- BiteCode_dev 6y ago
  I always though it would be a nice alternative to an ORM: having a tool that take a marshmallow/pydantic/whatever model, optionally passing it additional db specific options, then it generates a bunch of sql files you can call with aiosql. The whole things would then let you optionally get the result wrapped in a model if you need to, with ORM like helpers for common CRUD things. That would have the benefit of the standardized api of an ORM and the flexibility of SQL, without the coupling.
  - polyrand 6y ago
    Absolutely! I think it should be doable, FastAPI and other libraries do heavy Pydantic model inspection, and some do automatic code generation. I should explore this a bit more.
    - BiteCode_dev 6y ago
      It also has some nice properties: - because you have all your queries written in the same place, when you create a db migration, you know where to look for code to update, and can even perform some automatically. - sql queries could be turned into a stored procedure with a simple marking, without changing any code. Of course the downside is that dynamically exploring data is not as easy as with an ORM.
ctvo 6y ago
It's a bit of a step back in time reading things like this. This is stateless HTTP requests hitting a relational database. How is this dead horse still being beaten? The patterns for load balancing, horizontal scalability, caching in this space well documented. What are we gaining still profiling Django, Flask and Ruby on Rails in 2021.
- jtdev 6y ago
  I suppose every app you work on runs the same query repeatedly? Yes, load balancing makes sense, but the author is specifically looking at requests per thread, i.e., wouldn't it be great (and more cost effective) to get as much throughput from a single thread as possible?
- fouc 6y ago
  Does this mean you work with a stateful websockets setup? What stack?
Ambix 6y ago
Why is it so? I've got 100K requests per sec with PHP easily [1] [1] https://github.com/gotzmann/comet https://github.com/gotzmann/comet
- hu3 6y ago
  "Blazing fast with 100K HTTP requests per second and ~0.1 ms latency on commodity cloud hardware" Ok what's the secret sauce and downsides? Because as far as I know there's no way to cheat with PHP like C# did here: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/PlatformBenchmarks/BenchmarkApplication.Fortunes.cs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
  - Ambix 6y ago
    The most important part is Workerman and its efficient network arhitecture based on libevent. The other part is efficient DB drivers. Thats why Comet ranks much higher than Go / NodeJS / Python frameworks in DB higloaded test: https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=query https://www.techempower.com/benchmarks/#section=data-r20&hw=...
crad 6y ago
I get much higher request throughput in my Tornado applications, with very low response latencies, strange.
luord 6y ago
And here I was living under the assumption that psycopg2 was the only option, and probably the biggest reason I was not using pypy. Gotta take a look at pg8000. In general, I've always liked the idea of pypy, so I'll try to use it more, and not just for performance. Will also donate when I can.
Terretta 6y ago
Not that it matters any more, but a colleague mentions Flask was originally a joke of what not to do: https://lucumr.pocoo.org/2010/4/3/april-1st-post-mortem/ https://lucumr.pocoo.org/2010/4/3/april-1st-post-mortem/ Flask author reflects on that here: http://mitsuhiko.pocoo.org/flask-pycon-2011.pdf http://mitsuhiko.pocoo.org/flask-pycon-2011.pdf Quite relevant to the conclusion in the article.
ancount 6y ago
I inherited a flask queue worker, and it suffers from some major problems (like 12 req/second when it's not discarding items from the queue). I am primarily a javascript programmer so I'm a little bit out of my element. I am tempted to refactor the worker to use async features, and that would require factoring out uWSGI, which is fine, I only added it last week. The article states that Vibora is a drop in replacement for flask, but I guess I'm a bit skeptical, as I can't find much information outside of Vibora having a similar api. For a web service with basically one endpoint, I could refactor to another implementation fairly easily, I'm just looking for the right direction. I thought maybe I should refactor the arch to either batch requests to the worker, or to use async. Anyone have a feeling where I should go? I am just getting started researching this, but any advice would be appreciated. Edit: at least quart has a migration page.. probably will just try it out, what can I lose? https://pgjones.gitlab.io/quart/how_to_guides/flask_migration.html https://pgjones.gitlab.io/quart/how_to_guides/flask_migratio... Second edit: Also might try out polyrand's stack in the comments.
tubbyjr 6y ago
My god, the CSS and styling on that page is absolutely abysmal
AtlasBarfed 6y ago
Where is japronto on the techempower? It's not even on there.
oliwarner 6y ago
The important thing to remember is that unless you're running a massive service, requests per second is less important than seconds per request. Getting an API hit from 300ms to 70ms, and proper frontend caching is far more valuable than concurrency (if you can afford to throw servers at it) because it actually affects user performance.

503 points by gilad 6y ago

I have great experiences with Falcon for backend REST APIs, and it is supposed to be great in terms of requests per second. How does it compare to Sanic?

divbzero 6y ago
> I have great experiences with Falcon Then keep using Falcon! OP’s core thesis is that developer comfort matters more when choosing an API framework because the performance bottleneck is usually found elsewhere. The API itself should be fast enough with some combination of Gunicorn, gevent, PyPy, and horizontal scaling.
- gchamonlive 6y ago
  I just wanted to know how it would compare with Sanic, because I never used it.
  - divbzero 6y ago
    The biggest difference is that Falcon is synchronous while Sanic is asynchronous. With Sanic, you would explicitly specify async/await for asynchronous operations and use asynchronous libraries for I/O. Switching to Sanic could also affect how you deploy to production. Both are plenty fast. FastAPI [1] is also worth considering if you’re looking into asynchronous API frameworks. It comes with nice features for specifying API schemas. [1]: https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/
    - gchamonlive 6y ago
      I prefer async big time. We once had to implement async routines while using flask, where the server would return a 200 but keep processing the request, and the actual result would be sent by email. It was hellish and inefficient to make it. In hindsight would have been better to use a queue service and a consumer and decouple the whole process, even if it meant increased infrastructure and maintenance complexity.
huffmsa 6y ago
I'm a big Falcon user as well. Got into it because it was easier to get up and running without bells and whistles vs the more "popular" frameworks.

est 6y ago

Might as well refer to TechEmpower benchmarks. https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/

Jarred 6y ago
just.js (#9) looks interesting https://github.com/just-js/just https://github.com/just-js/just It seems to be a much tinier JavaScript runtime than Node.js (still using v8), but linux only The benchmark is probably unrealistically optimized code but even so, it implies Node.js itself has a large performance overhead
- chmod775 6y ago
  If you're not paying attention it would look that way, yes. If you pay attention you'll notice that just.js is using postgres as their DB, while all of the node benchmarks are handicapped by either using mongoose/mongodb or MySQL. There is no node benchmark with postgres, but all of the fastest benchmarks used it. "lithium" is a good example to show how much of an impact switching to postgres has. All of the 4 lithium benchmarks are identical except in what DB they use. The results are: lithium-postgres-batch (#2 - 659850), lithium-postgres-beta (#13 - 398773), lithium-postgres (#14 - 398258), lithium (#45 - 271989). The last result is MySQL.
  - reasonabl_human 6y ago
    What makes postgres special in these cases?
    - chmod775 6y ago
      I would like to know as well. All I know is that for this specific workload there appears to be a strong correlation between the chosen DB and performance. Depending on the benchmark you're looking at on that site, the top 30-60 results are exclusively postgres. Further, for every framework/language that was tested with both MySQL and postgres (there's quite a few of them), the postgres one always ranks higher.

Laminary 6y ago

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?

rovr138 6y ago
Where is SSL the bottleneck? Wondering if terminating earlier and just relying on HTTP after would help.
- wahern 6y ago
  Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. Session caching can be an important optimization that can significantly reduce those costs, but scaling session caching has its own problems. But IME real-world bottlenecks have more to do with overall architecture. People tend to heavily focus on technical details, such as concurrency architecture--the how. But the biggest opportunities for improved performance usually involve functional aspects of an architecture--the what. (Note that these aren't fixed categories; they're relative positions. A technical detail often becomes a functional model as development progresses.) 12 RPS is a long way from implicating SSL. If you get to the point where SSL is an identifiable bottleneck, you've either made a series of tremendously good decisions or exceptionally poor decisions.
  - toast0 6y ago
    > Signing, verification, and key exchange are quite expensive. ECC doesn't help server-side; it mostly reduces client-side verification costs. I don't think that's right. Cloudflare's blog [1] says they can do about 9.5x the handshakes/sec with ECDSA at 256-bits vs RSA at 2048. Verification for ECDSA signatures are somewhat slower, but it's usually an acceptable tradeoff to make clients do a bit more work so that servers do a lot less. I agree though, at 12 RPS, TLS isn't the bottleneck. [1] https://blog.cloudflare.com/ecdsa-the-digital-signature-algorithm-of-a-better-internet/ https://blog.cloudflare.com/ecdsa-the-digital-signature-algo...

ramraj07 6y ago

Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html https://docs.sqlalchemy.org/en/13/faq/performance.html If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute(<raw query>). Makes a huge difference. Next optimization I do is create stored procedures on the database. Only then I start thinking about changing the framework itself. For folks like me who want to get prototypes off the ground in hours, flask and fastapi are godsend, and if that means I have to worry about serving thousands of requests a second soon thats a happy problem for sure.

nwsm 6y ago
For me, FastAPI is the sweet spot between performance and quick development.
vosper 6y ago
You can also use SQLAlchemy Core, which is an intermediate between the full-blown ORM and running actual strings of SQL. I've had a great experience with Core - I can easily have it output essentially the exact SQL I'd write by hand, but I get many benefits (like the ability to compose queries) that are nicer than dealing with raw SQL.
- SatvikBeri 6y ago
  Definitely agree, I'm happy to write raw SQL, but SQLAlchemy Core is even better than that because of composability.
- smoe 6y ago
  This is what I like most about SQLAlchemy, I can use the ORM most of the time, but in cases where it needs optimizations or aggregation queries I can drop down or refactor to core, while still using the same model/table definitions, ability to combine or filter queries dynamically and leverage IDE features like refactoring.
welder 6y ago

tnash 6y ago

Use of ORMs is often a performance choke point. Raw DB queries are often much, much faster. Almost always, the more you abstract, the worse you perform. It's great as a developer but not so great as a user.

nmfisher 6y ago
I haven't touched an ORM in over 6 years, but unless they've improved since then, I honestly can't think of a single reason why anyone would choose to use one. They're clunky monstrosities that act only as guard-rails for inexperienced developers. Far better to invest a few days (which is realistically all you need) to improve their SQL skills and/or code-review practices.
- morpheuskafka 6y ago
  I am by no means an experienced developer--but the issue I always seem to have without using an ORM is that there are string literals containing different bits of SQL scattered all throughout my code. In addition to being hard to refactor when the database model changes, I think there is a fairly high performance cost to doing so many string concatenations every time the code runs. Is there a better way to manage the SQL command strings when you are sending the queries by hand?
  - branko_d 6y ago
    A lot of times you don't need to change the SQL string at all. You just bind different parameter values before sending the same SQL string to the DB server over and over again. The exceptions are SQL elements that cannot be (easily, or at all) parametrized, such as the column lists in SELECT, ORDER BY, GROUP BY, or changing the WHERE "shape" and so on. In my experience, this tends to be a minority of queries, although an important minority. ---- P.S. If string concatenations are your bottleneck, then your database is screaming fast! The real-life bottlenecks are usually in excessive database round-trips and unoptimized query plans, and are orders-of-magnitude larger.
  - takeda 6y ago

imperio59 6y ago

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure, the framework matters at very large scale and the benefits from optimizing the framework become large when you're doing millions of requests a second over many thousands of servers because it can help reduce baseline cost of running the service. But I agree with the author's main point which seems to be that framework performance is pretty meaningless when comparing frameworks if you're just starting on a new project. Focus on making a product people wanna actually use first. If you're lucky enough to get to scale you can work about optimizing it then.

Izkata 6y ago
> (usually people writing dumb code without realizing it's expensive) Some years ago, one morning I gave a co-worker a recommendation on how to improve a loop that was unnecessarily hitting database through the Django ORM. He committed the fix that afternoon. Barely an hour later I accidentally reintroduced the exact same slowdown in the exact same loop when adding a different piece of data to it. Soooo yeah, ORMs can be so simplistic it's too easy to do by accident even if you know exactly what's going on under the hood.
- solox3 6y ago
  Something about select_related? Please do share.
  - DangitBobby 6y ago
    My guess is accessing a related field within a loop causing a database request per iteration, e.g. ``` [book.author.name for book in Book.objects.all()] ```
    - 6y ago

maxpert 6y ago

The fact that you are using offset of 50000 and complaining it slows everything down says a lot about the benchmarks. Top it all with ORM query with prefetch all, GIL, and shared CPU (I am guessing) that you used to run benchmark on. You see where this is headed?

mlthoughts2018 6y ago

I think it’s been bog standard practice to run flask via uwsgi or gunicorn with async workers and use multiple process based workers per deployed server unit (eg per pod in Kubernetes). What matters is that the cumulative latency & throughput solve your problem, not how fast you can make one singular async worker thread. I figure most people running complex web services in production would just do an eye roll at this post. Nobody's going to switch to PyPy for any of this. My team at work runs several complex ML workloads, and we use the exact same container pattern for every service running gunicorn to spawn X async workers per pod and then scale pods per service to meet throughput requirements. Sometimes we also just post complex image processing workloads to a queue and batch them to GPU processor workers. In all these use cases, super low effort “just toss it in gunicorn running flask” has worked without issue for services supporting up to peak load of thousands to hundreds of thousands of requests per second.

7kmph 6y ago
Could you share your company’s website?
- mlthoughts2018 6y ago
  No, I can’t speak on their behalf on Hacker News, so it is important to me to stay disconnected from my employer.

jaimex2 6y ago

When you start hitting bottlenecks in your python web framework its probably time to switch to a faster language, not another framework in python. You're probably done rapid prototyping by this point anyway.

speedgoose 6y ago
I don't think there is much gains to rewrite everything in a faster language. Unless they are a very very successful company with billions of customers, it's often cheaper to scale horizontally.
- seanparsons 6y ago
  Depends on if you identify it early enough really and even then halving your app tier costs could be sizable for any company. If you can get some productivity benefits at the same time from say static typing then there's even more reason to switch.
square_usual 6y ago
There's still a good reason to pick fast frameworks in a slow language: you can delay the inevitable for a bit, probably enough time for you to work on a rewrite or whatever.
antihero 6y ago
Well if you convert everything from ORM to raw SQL, it will then be easier to extract all of that SQL and use it in a different web framework once you've measured and confirmed that your bottleneck is servicing requests in Python.

forgotmypw17 6y ago

I have to question the value of text written by someone who sets white-on-white text in their website...

Jonnax 6y ago
The background is blue?
- forgotmypw17 6y ago
  The background image is blue, the background itself is white. Without loading the image, the text is the same color.
  - throwawayffffas 6y ago
    I question the value of the comment of someone who loads css but not background images.

sriku 6y ago

A humble request to folks making benchmark or other graphs - please understand that thin coloured lines are not easy to visually parse .. even for folks like me who aren't totally colour blind but have partial red-green colour blindness. At least, the lines can be made thicker so it is easier to make out the colours. Even better, label the lines with an arrow and what they represent.

TickleSteve 6y ago
Absolutely! I have mild red/green issues and could barely tell those lines apart.

ximm 6y ago

Maybe I am missing something, but why wasn't sanic tested with pypy? I expect that this combination would outperform everything else.

erdewit 6y ago
Was wondering the same thing. PyPy can do wonders for other asyncio-based frameworks.

sandGorgon 6y ago

TLDR - pypy is awesome. Dont use frameworks. Use pypy. Please donate. Pypy needs funds - https://opencollective.com/pypy https://opencollective.com/pypy Pypy doesnt get a fraction of the funding that python does.

cryptos 6y ago

Why Python at all? About 10 years ago I liked Python a lot (and still like it in principle) and felt very productive compared to, say, Java. Java was full of inconvenience, XML, bloated frameworks and all that. But today you can use Kotlin, that is in my opinion even nicer than Python, with performant frameworks (e. g. Quarkus or Ktor) on the super fast JVM. I don't want to start a language war, but maybe Python is not the first choice for their requirements.

galacticdessert 6y ago
I feel like when you need to create value and make a product, better to do it in a language you know. Product value is not directly proportional to product performance or raw HTTP request serving time. I for instance know a bit of F#/C# and Java, but would probably pick Python to make a new product just to remove that mental barrier of not having my lack of language knowledge in the way of things
kavalg 6y ago
I can code comfortably in Python, Java, JavaScript and to some extent C/C++. In the last 4 years I have been using mostly Python for various reasons (Machine Learning, OS automation, web scraping ...). Compared to the other languages, Python feels lighter and faster to write to the extent that it rarely interrupts my flow of thoughts. Now and then I have to code in JavaScript (frontend), C/C++ (embedded / low level optimizations) and Java (maintenance) and my feelings are: - JavaScript still feels messy - C/C++ is complex, but it is often offset by the complexity/needs of the project (e.g. in embedded) - Java has become kind of bloated with all the new stuff So as others have already mentioned, development productivity is in many cases far more important than speed of code. In my ~20y long career I have rarely seen a project failure due to runtime performance. Most of them failed due to speed/agility of development iterations and also project/product management issues (bad fit, unrealistic project plan, lack of focus and customer feedback). That said, if I need to look for another language due to performance, that would probably be Rust.
- huffmsa 6y ago
  I also notice I spend less time thinking "how do I do X in this language" with Python and just do it. Maybe not the best implementation of X, but that doesn't really matter because I implemented X, Y and Z while someone else got bogged down in their "better" language and barely got X across the finish line.

hgretg3443 6y ago

C#/ASP.NET is the fastest web framework now: https://www.techempower.com/benchmarks/#section=test&runid=8ca46892-e46c-4088-9443-05722ad6f7fb&hw=ph&test=plaintext https://www.techempower.com/benchmarks/#section=test&runid=8... 7.000.000 requests per second Even GO can only achieve 4.500.000 million requests per secnod being a low-level language, in opposite to high-level C#.

sgtnoodle 6y ago
The #1 project on that list appears to be implemented in rust? What does GO being "low-level" have to do with performance of serving requests? I'd imagine its bottlenecks would be due to something fairly arbitrary, like its garbage collection or how it represents strings or something.
nodamage 6y ago
That 7 million requests per second is achieved by writing a hard coded plain text HTTP response string directly to the client... it is so far disconnected from any real world use case that the number is basically meaningless.
- arcturus17 6y ago
  It's still 8th in the composite benchmark. And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? [1] https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=composite https://www.techempower.com/benchmarks/#section=data-r20&hw=...
  - BiteCode_dev 6y ago
    Yes but getting a good ranking at a useless benchmark if not very meaningful. It's cool, but I wouldn't use it to make any serious decision.
  - nodamage 6y ago
    > And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? Indeed. The problem is that many of the scores in the top 100 are really misleading because no one building a web app would implement things that way. There is still some value in the lower down benchmarks but you have to basically read the underlying source to determine if the implementation is remotely realistic or not. For starters I would ignore anything classified as "platform" which is described as: Platform, meaning a raw server (not actually a framework at all). Good luck! You're going to need it. For C# in particular I would only consider the mvc variants as realistic. Edit: I looked into the "asp.net core" composite score a bit more, it looks like those benchmarks are based on the aforementioned "platform" implementations for each test. I actually think this score is even more misleading than the individual benchmarks. At least the individual benchmarks show you the difference between "aspcore" (platform), "aspcore-mw" (middleware-only), and "aspcore-mvc" (full framework with routing). Here are recalculated composite scores based on the more realistic implementations (aspcore-mvc, aspcore-mvc-ado-pg, aspcore-mvc-dap-pg, aspcore-mvc-ef-pg): ASP.NET Core MVC with ADO.NET (raw SQL): 3029 ASP.NET Core MVC with Dapper: 2591 ASP.NET Core MVC with Entity Framework: 2195 Compared to Flask's 468 or Django's 280 it's still significantly faster, but not to the same extreme you might think at first glance at the chart.

KaiserPro 6y ago

We did an evaluation for our API. The API accepts an image upload, passes it onto the backend for processing and returns a ~2k json lump in return. Long story short, fastapi was much much faster than anything else for us. It also felt a bit like flask. The integration with pydantic for validating dataclasses on the fly was also great.

helsinkiandrew 6y ago

Since I've been a developer there have been two changes that I feel have given major performance improvements and made backend framework improvements much less significant (atleast in the apps I develop): CDNs and client side rendering (that means the more, smaller requests for data which are more suited to be served via a CDN) Using (for example) AWS Cloudfront was a gamechanger in how I design webapps and view performance. Being able to 'slice and dice' what requests get SSL terminated at the CDN, cached fairly locally, served from an Amazon managed webserver, or sent to our app server, increased our performance 10 fold. That approach isn't always practical, but I find that it's now much easier to choose the backend for developer performance and doubling the server CPU/memory is quicker and cheaper when needed.

qeternity 6y ago

As a Django shop, we’ve always hoped PyPy would one day be suitable for our production deployments but in the end with various issues we were never able to make the switch. And then Pyston was re-released...and changed everything. It was drop in compatible for us and we saw a 50% drop in latencies. Source availability aside, I suggest anyone running CPython in prod take a look.

twsted 6y ago
Can you tell us more about that? I remember pyston v1 from Dropbox. You are speaking about v2, which is a binary package (closed-source at the moment)?
- qeternity 6y ago
  Yeah it's a closed source binary. We're very happy with it. Great compatibility, no horrendous warm up times, and very meaningful speedups. Not much more to say. It's the same, just a bit faster.

robertlagrant 6y ago

Note: SQLAlchemy 1.4 is async. https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#asynchronous-io-support-for-core-and-orm https://docs.sqlalchemy.org/en/14/changelog/migration_14.htm...

lrossi 6y ago

This benchmark was run on a laptop, which has a very small number of cores compared to the servers that usually run such apps. The author doesn’t mention any attempt to tweak the number of workers, which would make sense in this case. Given that they did notice at some point that CPU usage is lower than expected, I am surprised that they did not try it.

throwdbaaway 6y ago

Good article, but I can't help but notice a gaping hole in the benchmark -- why was there no attempt to run gunicorn in multi-threaded mode? The article has a link to https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a..., but failed to mention the key takeaway from the article: > threaded code got the job done much faster than asyncio in every case

driverdan 6y ago
The article explicitly says they were testing single threaded use cases.
- throwdbaaway 6y ago
  But why? The results would be a lot more useful if they include a single-process multi-threaded setup in the benchmark.

hendry 6y ago

I always assumed Python could scale because of Reddit: https://github.com/reddit-archive/reddit https://github.com/reddit-archive/reddit Not quite sure if their current site's code is opensource... anyone know?

andsko 6y ago
And because of Instagram: https://instagram-engineering.com/python-at-scale-strict-modules-c0bb9245c834 https://instagram-engineering.com/python-at-scale-strict-mod...
segmondy 6y ago
Any language can scale. Pretty much most languages there can handle scale. It's usually bad algorithms or external services be it API & DB used poorly that impacts performance.
pas 6y ago
Facebook uses/used PHP, that scales too. If you make it stateless and use it only as a glue. Dropbox uses Python too. Eve Online used Stackless Python. Of course Reddit/FB/EVE/Dropbox shards everything, there's no global state to manage via Python. The state lives in the data store layer. And for that there are these monstrous/elegant things like Vitess, that YouTube used (uses?): https://vitess.io/docs/overview/architecture/ https://vitess.io/docs/overview/architecture/ which is basically a sharding/routing layer on top of independent MySQL instances.

sdevonoes 6y ago

Interesting. I never heard of Japronto before. For the people working with Python: why Flask instead of Japronto?

BiteCode_dev 6y ago
The ecosystem. You got loads of libraries built around flask, and a lot of doc and tutorials, not to mention how much battle tested it is. Those can matter much more than hypothetical perfs in a synthetic benchmark.

p5v 6y ago

The TL;DR you should be looking for: > all of this emphasises the fact that unless you have some super-niche use-case in mind, it's actually a better idea to choose your framework based upon ergonomics and features, rather than speed.

dfgdghdf 6y ago

I would question choosing Python for large server projects because the performance ceiling is so low. At least with the "middle tier" performance languages such as Java / C# you are unlikely to require a complete language switch as the project scales.

musingsole 6y ago
With cloud and infinite horizontal scaling...hitting that performance ceiling is an indication of a poor design.
- dfgdghdf 6y ago
  This is false, since it is not always easy (or even possible) to horizontally scale a work-load.
  - musingsole 6y ago
    If you've made it impossible or just difficult to scale horizontally in the server world, yeah, you've got a bad design.
    - dfgdghdf 6y ago
      Sorry to keep coming back, I don't want to start a flame war here. Difficulty of horizontal scaling is a property of two things: 1. how your solution is designed (your point, and I agree this is often done poorly) 2. the problem / work-load you are trying to scale (my point). It might be that your problem does not "shard" very easily. You cannot fix this with solution architecture, at least not easily. Horizontal scaling of a relational database is very difficult, for example. Edit: Another example. Can you rewrite NGINX in a slower language and use horizontal scaling to fix it? Of course not, because that horizontal scaling would itself leverage NGINX (or something like it)!
      - mixmastamyk 6y ago
        There are lots of ways to speed python if need be, and you can recover much of the work already done. Cython is another.

polyrand 6y ago

Related to ORMs/queries/performance, I have found the following combination really good: * aiosql[0] to write raw SQL queries and having them available as python functions (discussed in [1]) * asyncpg[2] if you are using Postgres * Map asyncpg/aiosql results to Pydantic[3] models * FastAPI[4] Pydantic models become the "source of truth" inside the app, they are designed as a copy of the DB schema, then functions receive and return Pydantic models in most cases. This stack also makes me think better about my queries and the DB design. I try to make sure each endpoint makes only a couple of queries. Each query may have multiple CTEs, but it's still only a single round-trip. That also makes you think about what to prefetch or not, maybe I want to also get the data to return if the request is OK and avoid another query. [0] https://github.com/nackjicholson/aiosql https://github.com/nackjicholson/aiosql [1] https://news.ycombinator.com/item?id=24130712 https://news.ycombinator.com/item?id=24130712 [2] https://github.com/MagicStack/asyncpg https://github.com/MagicStack/asyncpg [3] https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/ [4] https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/

BiteCode_dev 6y ago
I always though it would be a nice alternative to an ORM: having a tool that take a marshmallow/pydantic/whatever model, optionally passing it additional db specific options, then it generates a bunch of sql files you can call with aiosql. The whole things would then let you optionally get the result wrapped in a model if you need to, with ORM like helpers for common CRUD things. That would have the benefit of the standardized api of an ORM and the flexibility of SQL, without the coupling.
- polyrand 6y ago
  Absolutely! I think it should be doable, FastAPI and other libraries do heavy Pydantic model inspection, and some do automatic code generation. I should explore this a bit more.
  - BiteCode_dev 6y ago
    It also has some nice properties: - because you have all your queries written in the same place, when you create a db migration, you know where to look for code to update, and can even perform some automatically. - sql queries could be turned into a stored procedure with a simple marking, without changing any code. Of course the downside is that dynamically exploring data is not as easy as with an ORM.

ctvo 6y ago

It's a bit of a step back in time reading things like this. This is stateless HTTP requests hitting a relational database. How is this dead horse still being beaten? The patterns for load balancing, horizontal scalability, caching in this space well documented. What are we gaining still profiling Django, Flask and Ruby on Rails in 2021.

jtdev 6y ago
I suppose every app you work on runs the same query repeatedly? Yes, load balancing makes sense, but the author is specifically looking at requests per thread, i.e., wouldn't it be great (and more cost effective) to get as much throughput from a single thread as possible?
fouc 6y ago
Does this mean you work with a stateful websockets setup? What stack?

Ambix 6y ago

Why is it so? I've got 100K requests per sec with PHP easily [1] [1] https://github.com/gotzmann/comet https://github.com/gotzmann/comet

hu3 6y ago
"Blazing fast with 100K HTTP requests per second and ~0.1 ms latency on commodity cloud hardware" Ok what's the secret sauce and downsides? Because as far as I know there's no way to cheat with PHP like C# did here: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/PlatformBenchmarks/BenchmarkApplication.Fortunes.cs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
- Ambix 6y ago
  The most important part is Workerman and its efficient network arhitecture based on libevent. The other part is efficient DB drivers. Thats why Comet ranks much higher than Go / NodeJS / Python frameworks in DB higloaded test: https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=query https://www.techempower.com/benchmarks/#section=data-r20&hw=...

crad 6y ago

I get much higher request throughput in my Tornado applications, with very low response latencies, strange.

luord 6y ago

And here I was living under the assumption that psycopg2 was the only option, and probably the biggest reason I was not using pypy. Gotta take a look at pg8000. In general, I've always liked the idea of pypy, so I'll try to use it more, and not just for performance. Will also donate when I can.

Terretta 6y ago

Not that it matters any more, but a colleague mentions Flask was originally a joke of what not to do: https://lucumr.pocoo.org/2010/4/3/april-1st-post-mortem/ https://lucumr.pocoo.org/2010/4/3/april-1st-post-mortem/ Flask author reflects on that here: http://mitsuhiko.pocoo.org/flask-pycon-2011.pdf http://mitsuhiko.pocoo.org/flask-pycon-2011.pdf Quite relevant to the conclusion in the article.

ancount 6y ago

I inherited a flask queue worker, and it suffers from some major problems (like 12 req/second when it's not discarding items from the queue). I am primarily a javascript programmer so I'm a little bit out of my element. I am tempted to refactor the worker to use async features, and that would require factoring out uWSGI, which is fine, I only added it last week. The article states that Vibora is a drop in replacement for flask, but I guess I'm a bit skeptical, as I can't find much information outside of Vibora having a similar api. For a web service with basically one endpoint, I could refactor to another implementation fairly easily, I'm just looking for the right direction. I thought maybe I should refactor the arch to either batch requests to the worker, or to use async. Anyone have a feeling where I should go? I am just getting started researching this, but any advice would be appreciated. Edit: at least quart has a migration page.. probably will just try it out, what can I lose? https://pgjones.gitlab.io/quart/how_to_guides/flask_migration.html https://pgjones.gitlab.io/quart/how_to_guides/flask_migratio... Second edit: Also might try out polyrand's stack in the comments.

tubbyjr 6y ago

My god, the CSS and styling on that page is absolutely abysmal

AtlasBarfed 6y ago

Where is japronto on the techempower? It's not even on there.

oliwarner 6y ago

The important thing to remember is that unless you're running a massive service, requests per second is less important than seconds per request. Getting an API hit from 300ms to 70ms, and proper frontend caching is far more valuable than concurrency (if you can afford to throw servers at it) because it actually affects user performance.