5 ms·
"What they mean by IO bound is actually that their system doesn’t use enough work to saturate a single core when written in Rust: if that’s the case, of course
by asd4 3y ago
"What they mean by IO bound is actually that their system doesn’t use enough work to saturate a single core when written in Rust: if that’s the case, of course write a single threaded system."
Many of the applications I write are like this, a daemon sitting in the background reacting to events. Making them single threaded means I can get rid of all the Arc and Mutex overhead (which is mostly syntactic at that point, but makes debugging and maintenance easier). Being able to do this is one of the things I love about Rust: only pay for what you need.
The article that this one is responding to calls out tokio and other async libraries for making it harder to get back to a simple single threaded architecture. Sure there is some hyperbole but I generally agree with the criticism.
Making everything more complex by default because its better for high throughput applications seems to be opposite of Rust's ideals.
- Kinrany 3y agoI was going to comment on the same quote. The problem is that one may still want concurrency even when a single thread on a single CPU is enough.
- amluto 3y agoI’ve written services like this, and I would never have called them IO bound. They’re not throughput-bound at all. They mostly sit idle, then they do work and try to get it done quickly to minimize use of system resources. Unless they sometimes get huge bursts of work and something else cares quite a lot about latency during those bursts, using more than one thread adds complexity and overhead for no gain.
- __alexs 3y agoIn an era of 10Gb NICs in every server very few things are really IO bound.
- kosolam 3y ago10gb nics and their respective connections are quite expensive. Not many servers have these at all.
- reacharavindh 3y agoAs a person with a sysadmin + HPc background having built several clusters recently, this is not true(anymore). 10G NICs are almost as common as Gigabit NICs(both in availability and cost). To give you an idea, we commonly use 10G NICs on all compute nodes, and they connect to a 10G top of the rack switch which connects to services like file servers via 100G connections. The 10G connections are all 10GBase-T simple Ethernet connections. The 100G connections are DACs that are more expensive but not prohibitively so. What cloud providers give you for VMs is not the norm in the datacenters anymore.
- kosolam 3y agoEverything is relative. If you are a cloud provider it’s one thing. I’m speaking from the perspective of the small medium business that rents these physical or virtual servers.
- packetslave 3y agomy $700 Mac Mini has a 10gb NIC. 2.5gb and 5gb NICs are very common on modern PC motherboards. Modern servers from Dell and HP are shipping with 25gb or even 100gb NICs.
- andrewprock 3y agoThe cost of 10g is much higher than a single computer. The entire networking stack must be upgraded to 10g. At the very least the Internet device, and possibly the Internet connection as well. It will be cheaper in the cloud than on site.
- packetslave 3y agoWell, it depends on what your use case for "10g" is. If all you care about is fast file transfers between your PC and your NAS, you can get a small 5-8 port 10gb switch for under $300 that will easily handle line-rate traffic (at least for large packet sizes) If you want 10g line-rate bandwidth between hundreds or thousands of servers? Yeah, I used to help build those fabrics at Google. It's not cheap or easy. 10g to the internet is more about aggregate bandwidth for a bunch of clients than throughput to any single client. Except for very specialized use cases you're going to have a hard time pushing anywhere close to 10g over the internet with a single client.
- tuetuopay 3y agoThe NIC does not really have a lot to do with being IO bound. IO bound means you spend most of your time waiting on an IO operation to complete. Usually writes are bound by the hardware (how fast your NIC is, how fast your storage is, ...), but reads are bounds by the hardware, but mostly by the "thing" that sends the data. So it's great you have a 10Gbps NIC, but if your database takes 10ms to run your query, you'll still be sitting for 10ms on your arse to read 1KB of data.
- withoutboats3 3y agoIn this context, we're talking about things for which the throughput is IO-bound. You're talking about the latency of an individual request. Throughput being IO-bound is indeed about the hardware, and the truth is that at the high end it's increasingly uncommon for things to be IO-bound, because our NICs and disks continue to improve while our CPU cycles have stagnated.
- raggi 3y agoIn purely practical terms the old system interfaces are sufficiently problematic that for any workload with necessarily smaller buffers than tens of kb, most implementations will get stuck being syscall bound first. Spectre really didn’t help here either.
- vacuity 3y agoI think this is where we have to really move towards the io_uring/FlexSC approach.
- elteto 3y agoThe speed of your NIC doesn't matter when you are waiting for an INSERT on a DB with a bad schema. Heck, your DB could be on localhost and you are not even hitting the NIC card. Still the same.
- hjl22 3y agoThere are plenty of applications that do not run on servers. Lots of IO bound stuff in mobile or desktop apps - waiting for network responses, reading data files on startup, etc.
- PaulDavisThe1st 3y agoAlthough NVMe/SSD drives have changed things a lot, any media creation software is still IO bound in the sense that: a. you cannot plan to read data from disk on demand, because it will take too long (still!), and it will almost certainly block b. you cannot plan to write data to disk on demand, because it will take too long (still!) and it will almost certainly block c. the bandwidth is still a limit on the amount of data that can be handled. It is much higher than it was with spinners, but there is still a limit.
- riku_iki 3y ago> In an era of 10Gb NICs in every server very few things are really IO bound. for my data crunching project, one core processes about 500MB/s = 4Gb/s, and I have 64 cores..
- withoutboats3 3y agoA lot of people on the internet are confused about what "IO bound" means, and use it in this incorrect way.
- surajrmal 3y agoYes exactly. Not everything seeking concurrency is a web server. In an OS, every single system service must concurrently serve IPC requests, but the vast majority of them do so single threaded to reduce overall CPU consumption. Making dozens of services thread per core on a four core device would be a waste of CPU and RAM.
- marcosdumay 3y ago> Not everything seeking concurrency is a web server. Web servers should be overwhelmingly synchronous. They are the one easiest kind of application to just launch a lot more. Even on different machines. There are some limits on how many you can achieve but they aren't anything near low. (And when you finally reach them, you are much better rearchitecting your system than squeezing a marginal improvement due with asynchronous code.) There's a lot to gain from non-blocking IO, so you can serve lots and lots of idle clients. But not much from asynchronous code. Honestly, I feel like the world has gone crazy.
- basro 3y agoInstead of Arc and Mutex you'd be using Rc and RefCell. Wouldn't it be just as complex and verbose code-wise? I understand that it is less efficient but in the case you describe wouldn't paying for a few extra atomics be negligible anyway?
- monocasa 3y agoI've found that practically I'm more likely to simply use Box, Vec, and just regular data on the stack rather than Rc and RefCell when I esque Arc and Mutex by using a single context. The data modeling is different enough that you generally don't have to share multiple references to the same data in the first place. That's where the real efficiencies come to play.
- withoutboats3 3y agotokio supports a single threaded executor when you really need it, and its not even hard. It's called a LocalSet in tokio's API: https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html#awaiting-a-localset https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html...
- basro 3y agoThis is true but the rest of the ecosystem is not built for it. If you try to use axum in this way you'd still need to use send and sync all over the place.