6 ms·
I think the du case was an outlier. Normal workloads shouldn't be so heavily affected. I am expecting a few percent loss on most programs though. It's basica
by jandrese 9y ago
I think the du case was an outlier. Normal workloads shouldn't be so heavily affected. I am expecting a few percent loss on most programs though. It's basically a larger penalty for making a syscall, which was already a fairly slow operation so performance minded people avoid them in tight loops. It will be bad for people who need to do lots of fast I/O I suspect.
- mtanski 9y agoIt sounds like databases on very fast storage will be updated. Tons of syscalls made for disk io and network io.
- lmm 9y agoDatabases are already written to minimise syscalls, like any other heavily performance-tuned system.
- amadvance 9y agoApplications that already put an effort to minimize syscalls are the ones that most likely depend on syscalls performance.
- lmm 9y agoUnlikely. It's relatively easy to get to the point where syscalls aren't the bottleneck by a long margin, so the only apps where syscalls are the bottleneck will be those that haven't put in any optimisation effort.
- tankenmate 9y agoSome portable / embedded databases come to mind. Also "normal" databases doing replication initiation, and replication re-sync. And lastly backup, restore, tar, etc with small files. For files under a handful of pages long mmap() isn't a big gain. Another syscall I think that might cause issues is gettimeofday(), that particular call has been optimised to the nth degree, and lots of user programs spam the crap out of it (mostly necessarily), especially networking and streaming programs. It would be interesting to see how much of an overhead percentagewise page table isolation will cost, and its effects on low end media devices, et al.
- amluto 9y agovDSO-based timing is unaffected.
- pm215 9y agoLinux gettimeofday these days is implemented in the 'vdso', which is code provided by the kernel that runs solely in userspace. So it's not a syscall in the 'privilege level switch by executing insn that takes an exception' sense and shouldn't be affected by the syscall-entry/exit path becoming more expensive.
- tankenmate 9y agoDoes this also apply to vsyscall's being emulated? Will this mean that older static binaries will no longer run? or just suffer a penalty as well?
- wolf550e 9y agoLinux kernel has a compatibility guarantee to the user space visible API, so static binaries will continue to run. If the static binary is so old it does not know about vdso and uses regular syscall to query time, it will be slowed down.
- tankenmate 9y agoThis topic got me to look at the vDSO for my own machine so I wrote a short Perl program to dump your kernel's vDSO. https://pastebin.com/UnQX5U1f https://pastebin.com/UnQX5U1f
- tankenmate 9y agoI added a quick repo to github if you want to clone or send pull requests for architectures apart from 64bit LE. https://github.com/mattkeenan/dump-vdso https://github.com/mattkeenan/dump-vdso
- amluto 9y agoEmulated vsyscalls were already very slow and will be even slower on a patched kernel. They'll still work, though.
- contrarian_ 9y agoSounds like servers handling lots of small UDP packets would be hit pretty hard.
- revelation 9y agoApplications like this where the syscall overhead (and latency) starts to be a significant factor in processing time and latency have moved to userland drivers anyway: DPDK for 10-100 Gbps networking: https://dpdk.org/ https://dpdk.org/ SPDK for NVMe storage: http://www.spdk.io/ http://www.spdk.io/ The queuing and balancing stuff the kernel does makes sense for spinning rust harddisks and residential networking, but when the underlying hardware is so fast that nothing is ever queued, really what are you doing. At 100 Gbps line speed, a 1518 byte packet takes all of ~ 120ns to transmit, or about 360 clock cycles for a 3 GHz processor.
- emj 9y agoTaking control over network cards in user space seems doable nowdays. There was a talk about doing such drivers with IOMMU/DMA at CCC: https://media.ccc.de/v/34c3-9159-demystifying_network_cards https://media.ccc.de/v/34c3-9159-demystifying_network_cards
- arghwhat 9y agoUser-space drivers have been doable for a while, and dpdk[1] is definitely worth a check. There's also some manufacturers[2] that only does user-space drivers for their high-performance cards (e.g. 4x10Gb/s, 2x40Gb/s, 2x100Gb/s cards). Being designed with this in mind helps performance a lot. 1: https://dpdk.org/ https://dpdk.org/ 2: http://www.napatech.com/ http://www.napatech.com/
- DSMan195276 9y ago> Applications like this where the syscall overhead (and latency) starts to be a significant factor in processing time and latency have moved to userland drivers anyway: I would personally think that is worse, though please correct me if I'm wrong. The userland driver will run with an isolated PT like any other userland process won't it? If so, it will suffer the same slowdown that every other process now has every-time it has to communicate with the kernel, which I would think would be a lot for a driver.
- ambrop7 9y agoI think syscalls are not as slow as many people imagine they are, especially with modern CPUs and kernels (there are special instruction for syscalls that are faster than the old "interrupt" approach). See here: http://pzemtsov.github.io/2017/07/23/the-slow-currenttimemillis.html http://pzemtsov.github.io/2017/07/23/the-slow-currenttimemil... ("Off-topic: A system call"). But they will be slow with this mitigation.
- mehrdadn 9y agoIsn't every (edit: contended) mutex/etc. wait operation a syscall? That's gotta hurt for any program that waits for frequent events that don't take too long to process.
- mortehu 9y agoThey are only system calls when you need to wait or wake up processes. https://en.m.wikipedia.org/wiki/Futex https://en.m.wikipedia.org/wiki/Futex
- mehrdadn 9y agoI was assuming contention but I guess I wasn't clear, sorry. I updated the post. But saying this "only" occurs when there is contention is very misleading since it makes it seem like the scenario of lock contention is a negligible concern. It's not.
- kllrnohj 9y agoThread-suspending contended mutexes are already extremely slow. If you have a heavily-contended mutex you already have a major performance bug. If this is the kick in the pants you need to go fix it that's arguably a good thing ;) Note that mutex contention does not itself mean immediately falling back to futex - commonly you'll spinloop first and hope that resolves your contention (fast), then fall back to futex (slow)
- mehrdadn 9y ago> If you have a heavily-contended mutex you already have a major performance bug. I can't really devote time to countering the unfounded assertion that every contended mutex must be a bug. It certainly isn't consistent with my experience, but if every problem you've solved could have been parallelized infinitely without increasing lock contention, more power to you.
- asveikau 9y ago
- brazzledazzle 9y agoThis may end up doing some significant hurt to some build and testing pipelines.
- dannyw 9y agoPostgres and Redis are looking at a 20-25% performance hit. https://www.phoronix.com/scan.php?page=article&item=linux-415-x86pti&num=1 https://www.phoronix.com/scan.php?page=article&item=linux-41...