2 ms·

That sounds like a good solution, and is a good data point to know, thank you. Did you try Sync FastAPI? I'm wondering how its performance compares with async

fastball 6y ago
When you say sync do you just mean having sync endpoints? If so, then yeah, it's required since we're using SQLAlchemy ORM. Otherwise the calls to SQLA ORM would block the main event loop. As it is, FastAPI (well really Starlette) creates threads for sync endpoints to prevent blocking the main thread/event loop. So yeah, still seeing good speedups in our own benchmarks even though most of our endpoints are sync. What was arguably more important though was how much switching to ASGI helped with handling WebSockets. We're using SocketIO, and trying to get a fundamentally async protocol working within sync (Flask) land was a massive pain. We had repeated reliability and deployment issues that were very hard to debug. Switching to FastAPI made that much easier.
- StavrosK 6y ago
  Oh, I can imagine, ASGI must be immeasurably easier. Where do the async speedup gains come from, though, if your database is still sync? Wouldn't threadpools provide comparable performance before?
  - fastball 6y ago
    Well so actually we have both now. For WebSockets, all of the code is async, so I'm already using `asyncpg` for any database stuff that is happening there. With regards to why are the sync endpoints faster, I think it is a number of things, some of which are userland changes that could've been made under Flask, but all of which are somewhat related to the switch. With regards to things that FastAPI itself has changed, I think using a (de)serialization lib like Pydantic and serializing to JSON by default (which is what we were doing under Flask anyway, though with Marshmallow) makes a lot of the code paths in the underlying lib a bit faster, because with Flask there was more "magic" going on behind the scenes. For userland stuff, I think partly because there is less magic going in the background (I really like FastAPIs dependency injection system), it's made it easier to identify the bottlenecks and optimize hot code paths.
    - StavrosK 6y ago
      That makes perfect sense, thank you. I love FastAPI just for the code clarity and ease of working with better type objects (the Pydantic classes) alone, though the speed benefit is nice to have too.

fastball 6y ago

When you say sync do you just mean having sync endpoints? If so, then yeah, it's required since we're using SQLAlchemy ORM. Otherwise the calls to SQLA ORM would block the main event loop. As it is, FastAPI (well really Starlette) creates threads for sync endpoints to prevent blocking the main thread/event loop. So yeah, still seeing good speedups in our own benchmarks even though most of our endpoints are sync. What was arguably more important though was how much switching to ASGI helped with handling WebSockets. We're using SocketIO, and trying to get a fundamentally async protocol working within sync (Flask) land was a massive pain. We had repeated reliability and deployment issues that were very hard to debug. Switching to FastAPI made that much easier.

StavrosK 6y ago
Oh, I can imagine, ASGI must be immeasurably easier. Where do the async speedup gains come from, though, if your database is still sync? Wouldn't threadpools provide comparable performance before?
- fastball 6y ago
  Well so actually we have both now. For WebSockets, all of the code is async, so I'm already using `asyncpg` for any database stuff that is happening there. With regards to why are the sync endpoints faster, I think it is a number of things, some of which are userland changes that could've been made under Flask, but all of which are somewhat related to the switch. With regards to things that FastAPI itself has changed, I think using a (de)serialization lib like Pydantic and serializing to JSON by default (which is what we were doing under Flask anyway, though with Marshmallow) makes a lot of the code paths in the underlying lib a bit faster, because with Flask there was more "magic" going on behind the scenes. For userland stuff, I think partly because there is less magic going in the background (I really like FastAPIs dependency injection system), it's made it easier to identify the bottlenecks and optimize hot code paths.
  - StavrosK 6y ago
    That makes perfect sense, thank you. I love FastAPI just for the code clarity and ease of working with better type objects (the Pydantic classes) alone, though the speed benefit is nice to have too.