3 ms·

You can certainly use it, but whether you see any benefit is going to strongly depend on your workload. If you're doing significant calculations in the API then

by rpep 4y ago

You can certainly use it, but whether you see any benefit is going to strongly depend on your workload. If you're doing significant calculations in the API then it might be considerably more performant, but if your API is primarily retrieving things from the database and transforming it to JSON then you're going to be limited mostly by the database latency and so I wouldn't expect major improvements.

doix 4y ago
If you are fetching lots (not even 'big data', but a few thousand rows) of data using the Django ORM, you will see a performance difference when using pypy, or at least I did a few years ago. The database can happily return a few thousand rows very quickly, especially if you take care to optimize your queries and have good indexes. Converting a few thousand rows to python/django objects takes _time_. I can't quantify anything, because it's been too long, but I remember it being fairly significant. When I profiled it, the majority of the time was spent calling __setattr__ a few million times. Like you said, it depends on your use case. If your queries are slow, then optimize your database queries. But if your queries are fast and your responses are still slow, then investigating pypy is definitely worth it. You can also play around with .values_list or something in Django, so that you get 'raw values' instead of objects (but there's still a cost to building them up).
coldtea 4y ago
And yet the same supposedly "io bound" workloads (like "parse request, fetch something from DB, return it as JSON") still have widely difference performance characteristics in some languages vs others, with 10x to 100x requests handled per second...
- acdha 4y ago
  2x, yes. If you’re seeing 100x you’re comparing different things like creating and serializing complex objects versus simple types or using a JSON parser which loads an entire document into objects versus one which only retrieved specific values.
  - coldtea 4y ago
    Well, perhaps not 100x in rps, but 10x sure: Overall top performing frameworks (JS, Java, and Rust) at 650K rps. That's 7x over the top Python based framework at: 86K rps. And another very popular Python framework (flask) gets just to 2K rps. That's 325 times worse to the best. And that's the "single DB query" benchmark: https://www.techempower.com/benchmarks/#section=data-r21&test=db https://www.techempower.com/benchmarks/#section=data-r21&tes... https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#single-database-query https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj...

by rpep 4y ago

doix 4y ago

If you are fetching lots (not even 'big data', but a few thousand rows) of data using the Django ORM, you will see a performance difference when using pypy, or at least I did a few years ago. The database can happily return a few thousand rows very quickly, especially if you take care to optimize your queries and have good indexes. Converting a few thousand rows to python/django objects takes _time_. I can't quantify anything, because it's been too long, but I remember it being fairly significant. When I profiled it, the majority of the time was spent calling __setattr__ a few million times. Like you said, it depends on your use case. If your queries are slow, then optimize your database queries. But if your queries are fast and your responses are still slow, then investigating pypy is definitely worth it. You can also play around with .values_list or something in Django, so that you get 'raw values' instead of objects (but there's still a cost to building them up).

coldtea 4y ago

And yet the same supposedly "io bound" workloads (like "parse request, fetch something from DB, return it as JSON") still have widely difference performance characteristics in some languages vs others, with 10x to 100x requests handled per second...

acdha 4y ago
2x, yes. If you’re seeing 100x you’re comparing different things like creating and serializing complex objects versus simple types or using a JSON parser which loads an entire document into objects versus one which only retrieved specific values.
- coldtea 4y ago
  Well, perhaps not 100x in rps, but 10x sure: Overall top performing frameworks (JS, Java, and Rust) at 650K rps. That's 7x over the top Python based framework at: 86K rps. And another very popular Python framework (flask) gets just to 2K rps. That's 325 times worse to the best. And that's the "single DB query" benchmark: https://www.techempower.com/benchmarks/#section=data-r21&test=db https://www.techempower.com/benchmarks/#section=data-r21&tes... https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#single-database-query https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj...