3 ms·
How many threads can you spawn before the system grinds to a halt? If you're processing thousands of requests per second and each request gets its own thread th
by efaref 6y ago
How many threads can you spawn before the system grinds to a halt? If you're processing thousands of requests per second and each request gets its own thread then you will start to queue on thread spawning. Don't forget that each thread gets its own stack taking up megabytes of memory.
The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controllers to network protocol implementations and telephony software. An engineer I respect once told me that really it's the only way to write services at scale, and anything else is just a step on the road until you reinvent it. He was probably exaggerating, but it is very important, and nearly ubiquitous.
Having used custom frameworks for async code in C and C++, it's really refreshing to have it baked into the language and well supported. It's yet another arrow in Rust's fantastic quiver.
- sanxiyn 6y agorouille (my Rust web framework of choice) spawns threads each request, and it handles thousands of requests per second just fine. Computers are fast, and Rust doesn't slow down your computer. If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading. I guess it was different in the past when computers were slow. I can believee that.
- secondcoming 6y ago> it handles thousands of requests per second just fine It probably doesn't. Spawning threads per request is a lazy pattern. We use Apache to handle billions of requests per day, and even its thread pool can be an issue.
- pvorb 6y agoAnd once starting and stopping threads adds to much delay to your request processing, there could be a thread pool that grows as needed and which will reuse threads that haven't been closed yet. This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server.
- nicoburns 6y ago> This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server. True, but there's a reason that Apache usage is declining at the rate that it is.
- josephg 6y agoHow does it handle slow http attacks? If I open 10k TCP connections to your server and drip feed http requests 1 byte at a time on each connection, what happens? You used to be able to easily DOS apache servers this way, because you just needed enough concurrent connections to exhaust its thread pool and then it wouldn't be able to handle any more requests. And then you need a bit rate on each connection just high enough not to trip apache's connection timeout. (So like, 20 TCP connections each sending 1 byte every 20 seconds would do it. Not sure about today but Apache used to be brought to its knees with 1 bps of bandwidth.) You could probably mitigate this by putting nginx in front of your server, but this works because nginx uses async internally to handle requests. And that won't work if you ever do proxy passthrough (for SSE, websockets, etc).
- ibraheemdev 6y agoAre you aware that rouille and tiny_http (it's underlying HTTP implementation) have been unmaintained for a while now?