3 ms·

> Rust offers perfomance improvements in CPU and memory usage of the core application logic, and it's almost never the bottleneck for the typical web apps. Has

> Rust offers perfomance improvements in CPU and memory usage of the core application logic, and it's almost never the bottleneck for the typical web apps. Has anyone actually managed to find that this is the case? Just because your app is stalling on db queries doesn't mean it's "IO bound". Even the most trivial service that's just routing messages between two other services is likely to be radically faster when implemented in Rust vs JS.

jashmatthews 6y ago
Almost every web app. Exceptions are stuff like dynamic map tile generation. Even WhatsApp scale runs on a “slow” dynamic language.
- staticassertion 6y ago
  I'm not sure what you mean. > Almost every web app. No? My point is exactly that almost every web app is not io bound. They're just... slow. They write bad queries, they make bad usage of resources and block on requests, etc. > Even WhatsApp scale runs on a “slow” dynamic language. But that has nothing to do with being IO bound. Whatsapp, via Erlang, works by having tons of connections on a single box. And you can watch plenty of talks by them about how they've had to work with BEAM to get Erlang to actually be IO bound. "IO bound" is a meme. Most people just mean "my queries are slow". Unless you're saturating your NIC, you're not io bound with regards to throughput, and unless your queries execute in ~<100ns you probably aren't IO bound on latency either.
  - btrask 6y ago
    You're right that most web apps aren't actually "IO bound" per se. They aren't making optimal or maximal use of either network or disk IO. However the important point is that they aren't CPU bound either. Most are "user acquisition bound" (like tens of requests per second peak). Some are more like "developer time bound" (cutting the server farm in half is not worth the cost in dev time). Either way, using a slow development framework (including language, DB, etc.) that the developers like is a reasonable choice for what they're trying to do. (I personally wish most software was faster, but they have no reason to care what I think.)
    - staticassertion 6y ago
      I'm in full agreement that you can prioritize other things over efficiency, and that a framework in a slower language can be the right call.
      - btrask 6y ago
        Actually, I thought about it some more. I think a server that spends 99% of its time blocked on IO is "IO bound" even if it's because no one is sending it requests. It's a very different condition than saturating the NIC, but it's still meaningfully waiting on IO. Like, the process has to be spending its time somewhere, and it doesn't know "why" it's blocked. Maybe the terminology around this could be developed some more.
        staticassertion 6y ago
        I think we can at least agree that the terminology is fairly broken here, people very often say IO bound when they mean something totally different, and it is extremely common to hear "X doesn't need to be fast, it's IO bound anyway".
  - jashmatthews 6y ago
    When people say IO bound in this sense, they mean that's the lower bound on what your users experience: 4G latency 50ms AWS inter AZ latency 1ms Thread context switch 2us The response times from things like search APIs even at FANG companies with vast resources and highly skilled engineers are often in the hundreds of milliseconds.
golergka 6y ago
> Just because your app is stalling on db queries doesn't mean it's "IO bound". OK, "IO bound" may not be the best description in this case. Because you're bound not by your ethernet card or operating system drivers, but literally by the work that's happening on a separate, database server's CPU. I should work on my terminology, thanks for the correction. In my experience, it is usually waiting for DB. Regardless of whether your web app is well or badly written, it would get, at my estimate, would get no more than 5-10% request time improvement even if you made the app CPU infinitely fast, as 90-95% of the time is taken waiting for that database query. But if you spend the same developer time rewriting all your ORM-generated n + 1 select statements to a single SQL query, you would still spend the same proportion of time waiting for the database, but your overall latency would get 10 times less. I've had some situations where I could get 10-20 time performance improvements by not touching application code at all, just tweaking the database indexes. So, rewriting it all in a new language where you can, at most, 10% improvement, while you probably have areas where you can get 200-500% improvements seems unwise.
- eska 6y ago
  If your database queries are so bad that you can get such big improvements by changing the queries and indexes, then obviously focus on that, not the client programming language. But once you've done that your programming language is responsible for a larger ratio of the total time, since it runs in series to the database query, not in parallel. The benefit, in my experience, is much larger than 10% then.

jashmatthews 6y ago

Almost every web app. Exceptions are stuff like dynamic map tile generation. Even WhatsApp scale runs on a “slow” dynamic language.

staticassertion 6y ago
I'm not sure what you mean. > Almost every web app. No? My point is exactly that almost every web app is not io bound. They're just... slow. They write bad queries, they make bad usage of resources and block on requests, etc. > Even WhatsApp scale runs on a “slow” dynamic language. But that has nothing to do with being IO bound. Whatsapp, via Erlang, works by having tons of connections on a single box. And you can watch plenty of talks by them about how they've had to work with BEAM to get Erlang to actually be IO bound. "IO bound" is a meme. Most people just mean "my queries are slow". Unless you're saturating your NIC, you're not io bound with regards to throughput, and unless your queries execute in ~<100ns you probably aren't IO bound on latency either.
- btrask 6y ago
  You're right that most web apps aren't actually "IO bound" per se. They aren't making optimal or maximal use of either network or disk IO. However the important point is that they aren't CPU bound either. Most are "user acquisition bound" (like tens of requests per second peak). Some are more like "developer time bound" (cutting the server farm in half is not worth the cost in dev time). Either way, using a slow development framework (including language, DB, etc.) that the developers like is a reasonable choice for what they're trying to do. (I personally wish most software was faster, but they have no reason to care what I think.)
  - staticassertion 6y ago
    I'm in full agreement that you can prioritize other things over efficiency, and that a framework in a slower language can be the right call.
    - btrask 6y ago
      Actually, I thought about it some more. I think a server that spends 99% of its time blocked on IO is "IO bound" even if it's because no one is sending it requests. It's a very different condition than saturating the NIC, but it's still meaningfully waiting on IO. Like, the process has to be spending its time somewhere, and it doesn't know "why" it's blocked. Maybe the terminology around this could be developed some more.
      - staticassertion 6y ago
        I think we can at least agree that the terminology is fairly broken here, people very often say IO bound when they mean something totally different, and it is extremely common to hear "X doesn't need to be fast, it's IO bound anyway".
- jashmatthews 6y ago
  When people say IO bound in this sense, they mean that's the lower bound on what your users experience: 4G latency 50ms AWS inter AZ latency 1ms Thread context switch 2us The response times from things like search APIs even at FANG companies with vast resources and highly skilled engineers are often in the hundreds of milliseconds.

golergka 6y ago

> Just because your app is stalling on db queries doesn't mean it's "IO bound". OK, "IO bound" may not be the best description in this case. Because you're bound not by your ethernet card or operating system drivers, but literally by the work that's happening on a separate, database server's CPU. I should work on my terminology, thanks for the correction. In my experience, it is usually waiting for DB. Regardless of whether your web app is well or badly written, it would get, at my estimate, would get no more than 5-10% request time improvement even if you made the app CPU infinitely fast, as 90-95% of the time is taken waiting for that database query. But if you spend the same developer time rewriting all your ORM-generated n + 1 select statements to a single SQL query, you would still spend the same proportion of time waiting for the database, but your overall latency would get 10 times less. I've had some situations where I could get 10-20 time performance improvements by not touching application code at all, just tweaking the database indexes. So, rewriting it all in a new language where you can, at most, 10% improvement, while you probably have areas where you can get 200-500% improvements seems unwise.

eska 6y ago
If your database queries are so bad that you can get such big improvements by changing the queries and indexes, then obviously focus on that, not the client programming language. But once you've done that your programming language is responsible for a larger ratio of the total time, since it runs in series to the database query, not in parallel. The benefit, in my experience, is much larger than 10% then.