4 ms·
It's definitely not a blanket statement. For us, it is. Considering most requests aren't just 1 simple query either. If you even do 5 simple queries, each at 20
by mattrobenolt 13y ago
It's definitely not a blanket statement. For us, it is. Considering most requests aren't just 1 simple query either. If you even do 5 simple queries, each at 20ms each, that's 100ms spend just waiting for the db. Plus other systems in place.
And honestly for us, we'd be happy if all queries were ~20ms. :) Lots are unfortunately slower than that. Especially if all 45k/s were doing the queries.
- bliti 13y agoHow many database servers do you have? How do you have them handle the queue?
- mattrobenolt 13y ago1 primary master, with a few read slaves and a few shards.
- bliti 13y agoThanks for answering my questions. (:
- beambot 13y agoWith asynchronous (go-like) channels becoming so popular, I'd love to see some non-blocking Django ORM database calling features. That way, you could fire off a few non-blocking db requests and keep executing on the app server (eg. rendering?) until you hit a point where you needed the database data. I know you could roll your own implementation... but building this into the ORM would still be really slick.
- mattrobenolt 13y agoThere are frameworks that work like this, iirc. I think some in Scala, but I'm not sure. IMO, the added complexity wouldn't be worth it with things like Varnish. I think there are better solutions and this is attacking the wrong problem. But, YRMV. It depends on the situation at hand always.
- yummyfajitas 13y agoBasically anything living in the akka+spray world is written this way. But akka+spray is not an framework in the same sense that django is.
- lmm 13y agoAre there async drivers for e.g. mysql yet? Last time I did that I had a very nice async spray frontend, but it was all backed by a pool of regular blocking threads because that was the only way to use the database driver.
- yummyfajitas 13y agoThere is this, but it's not particularly mature: https://github.com/mauricio/postgresql-async https://github.com/mauricio/postgresql-async I also use https://github.com/etaty/rediscala https://github.com/etaty/rediscala for accessing redis. But even without async drivers for MySQL, you can still do: val sqlResult: Future[StuffFromSql] = future { val stmt = connection.createStatement("...") ... } This will tie up a thread, but you can still run other actions in parallel with it.
- saryant 13y agoIf you do this, execute the future on a different dispatcher so you aren't blocking the default dispatcher. This can cause problems if you tie up the default dispatcher with blocking SQL queries while using clustering: the cluster gossip won't be able to fire and if your failure detection settings are low, that node can be marked unreachable by the rest of the cluster.
- nostrademons 13y agoAppEngine NDB does this for their datastore. Even works with Django, though not with the ORM (you have to use AppEngine's datastore API).