4 ms·

> Which is all well and good, until we start hitting some limits. Like, maybe we have SO MANY CONNECTIONS that we run out of memory, because each of these threa

> Which is all well and good, until we start hitting some limits. Like, maybe we have SO MANY CONNECTIONS that we run out of memory, because each of these threads has its own stack, and that's not free. It's not a lot, but a lot of "not a lot" quickly becomes a lot, as we all learn sooner or later. I understand this was more of a side point as an introduction, but it's a very quick dismissal of blocking I/O that I've been seeing a lot. Modern threads + blocking I/O is much more performant than most people realize, often yielding better throughput than async due to reduced syscalls and other factors. Async is important when you are in an extremely latency constrained environment and/or need to prioritize tasks, but in other places, not so much. How much does increased stack memory usage really matter to a real server?

fasterthanlime 5y ago
Both my current and previous day jobs involve building an edge network, which involves doing TLS termination: that means dealing with a lot of crap that comes your way - much more traffic than is valid/legitimate traffic, because on the public internet, anything goes. So I may be biased here. The situation for a purely backend service (behind an existing reverse proxy) might very well be different — but as others have mentioned, the Rust http ecosystem has solidified around async, so even if you could get away with 1 connection = 1 thread, you might not want to, just because of where the ecosystem is. One thing I didn't mention is that epoll isn't even state of the art: there's a lot of work going on around io-uring and thread-per-core runtimes nowadays, which I'm following rather closely because again, at my day job it does matter!
- zozbot234 5y ago
  Rust 'async' frameworks allow you to issue blocking calls, either by auto-spawning a separate thread or on via a trivial executor that blocks on the current thread.
- ibraheemdev 5y ago
  > One thing I didn't mention is that epoll isn't even state of the art: there's a lot of work going on around io-uring and thread-per-core runtimes nowadays, which I'm following rather closely because again, at my day job it does matter! io-uring is definitely a game changer, but it isn't async specific!
MrBuddyCasino 5y ago
I have done lots big corp projects, and came to accept the fact that using async I/O or reactive streams or whatever technology it is that is currently en vogue is only very loosely related to actual requirements, because lets be honest those are pointless for an internal backend with at most 10 concurrent connections. This is usually due to two factors: - people lack understanding of fundamentals, thus buying into the hype, and are bad at weighing the trade-offs because they often can't see the downsides, especially regarding complexity - engineers are curious and want to play with $latest_tech, and then construct a post-hoc justification for it to which they are sometimes unaware themselves Or put differently, its either incompetence or mismatched incentives (principal-agent problem). There is some justification for playing with new tech as in "it helps motivation and retains talent", but sometimes the resulting complexity is far out of proportion. That being said, tech ecosystems have different cultures, and I can understand why Rust is the way it is. It attracts idealists that tolerate complexity in the pursuit of the optimal solution. They have already produced more useful software than eg Haskell, and this is ultimately the yardstick of success. Lets see how Zig will do in that regard.
- zozbot234 5y ago
  I'm not sure that tolerance for accidental complexity is all that idiomatic in Rust. Rust async is a lot less complex than what Go does under the hood. Rust as a whole is even less complex than C++, with essentially no loss in useful featureset.
  - MrBuddyCasino 5y ago
    The lang team is very careful in designing new features, and I agree that they don't introduce accidental complexity lightly. Rust is an impressive achievement and a step forward over C++.
- ibraheemdev 5y ago
  That's a good overview similar to my experience. > those are pointless for an internal backend with at most 10 concurrent connections I'm also arguing that the benefit of async over threads at 10 _thousand_ concurrent connections is still unclear. Yes memory usage will be less, but memory is cheap. Context switching overhead may be less, but a lot of it is still there because of readiness-based I/O. We don't see new, high-scale systems built on blocking I/O perhaps not because they can't be, but because async has become the norm, at the cost of _a lot_ of complexity.

fasterthanlime 5y ago

Both my current and previous day jobs involve building an edge network, which involves doing TLS termination: that means dealing with a lot of crap that comes your way - much more traffic than is valid/legitimate traffic, because on the public internet, anything goes. So I may be biased here. The situation for a purely backend service (behind an existing reverse proxy) might very well be different — but as others have mentioned, the Rust http ecosystem has solidified around async, so even if you could get away with 1 connection = 1 thread, you might not want to, just because of where the ecosystem is. One thing I didn't mention is that epoll isn't even state of the art: there's a lot of work going on around io-uring and thread-per-core runtimes nowadays, which I'm following rather closely because again, at my day job it does matter!

zozbot234 5y ago
Rust 'async' frameworks allow you to issue blocking calls, either by auto-spawning a separate thread or on via a trivial executor that blocks on the current thread.
ibraheemdev 5y ago
> One thing I didn't mention is that epoll isn't even state of the art: there's a lot of work going on around io-uring and thread-per-core runtimes nowadays, which I'm following rather closely because again, at my day job it does matter! io-uring is definitely a game changer, but it isn't async specific!

MrBuddyCasino 5y ago

I have done lots big corp projects, and came to accept the fact that using async I/O or reactive streams or whatever technology it is that is currently en vogue is only very loosely related to actual requirements, because lets be honest those are pointless for an internal backend with at most 10 concurrent connections. This is usually due to two factors: - people lack understanding of fundamentals, thus buying into the hype, and are bad at weighing the trade-offs because they often can't see the downsides, especially regarding complexity - engineers are curious and want to play with $latest_tech, and then construct a post-hoc justification for it to which they are sometimes unaware themselves Or put differently, its either incompetence or mismatched incentives (principal-agent problem). There is some justification for playing with new tech as in "it helps motivation and retains talent", but sometimes the resulting complexity is far out of proportion. That being said, tech ecosystems have different cultures, and I can understand why Rust is the way it is. It attracts idealists that tolerate complexity in the pursuit of the optimal solution. They have already produced more useful software than eg Haskell, and this is ultimately the yardstick of success. Lets see how Zig will do in that regard.

zozbot234 5y ago
I'm not sure that tolerance for accidental complexity is all that idiomatic in Rust. Rust async is a lot less complex than what Go does under the hood. Rust as a whole is even less complex than C++, with essentially no loss in useful featureset.
- MrBuddyCasino 5y ago
  The lang team is very careful in designing new features, and I agree that they don't introduce accidental complexity lightly. Rust is an impressive achievement and a step forward over C++.
ibraheemdev 5y ago
That's a good overview similar to my experience. > those are pointless for an internal backend with at most 10 concurrent connections I'm also arguing that the benefit of async over threads at 10 _thousand_ concurrent connections is still unclear. Yes memory usage will be less, but memory is cheap. Context switching overhead may be less, but a lot of it is still there because of readiness-based I/O. We don't see new, high-scale systems built on blocking I/O perhaps not because they can't be, but because async has become the norm, at the cost of _a lot_ of complexity.