3 ms·
"When we have modern libraries and frameworks" I would question the assumption of having "modern" libraries, or at least rephrase it in terms of "performance"
by jorangreef 8y ago
"When we have modern libraries and frameworks"
I would question the assumption of having "modern" libraries, or at least rephrase it in terms of "performance" and "reliability":
Do we really have performant libraries (do they at least saturate the hardware)?
When it comes to networking, I think the answer is going to be more and more, no, we do not have performant libraries and software is being outpaced by hardware. Hardware is sitting idle thanks to our libraries.
Do we really have reliable libraries?
When it comes to storage, again I think the answer is no. For example, Linux RAID has long suffered from a plethora of faulty logic (pick a random disk sector as the canonical sector, split-brain etc.) and of course Linux RAID is not at the right layer to even have a chance of getting this right. And yet even ZFS lacks algorithms to detect or prevent split-brain. It's possible to detect (with 2 disks) and possible to prevent (with 3 or more disks), by taking quorum from the longest partially ordered set in the topology, but even ZFS (which is still awesome) is not reliable in this regard.
When it comes to networking, even TCP is not that reliable but has huge issues with bufferbloat. For example, a single person uploading a Gmail attachment on a 10mbps line will break the Internet for a second or two for everyone at your office (try it sometime - get everyone running "ping" and then upload an attachment). This is why Apple's background software update uses LEDBAT. Most libraries are not using LEDBAT or similar but are trusting TCP's congestion algorithms to do the right thing.