5 ms·
Now, consider that there's work going on to enable Linux to be compiled with profile guided optimization with clang[0], the DAMON patchset that enables proactiv
by marcodiego 5y ago
Now, consider that there's work going on to enable Linux to be compiled with profile guided optimization with clang[0], the DAMON patchset that enables proactive reclamation with "32% memory saving with only 1.91% runtime overhead"[1] and performance improvements achievable with futex2 system call[2].
Linux future seems bright with regard to performance.
[0] https://lkml.org/lkml/2021/1/11/98 https://lkml.org/lkml/2021/1/11/98
[1] https://lore.kernel.org/lkml/20210608115254.11930-1-sj38.park@gmail.com/ https://lore.kernel.org/lkml/20210608115254.11930-1-sj38.par...
[2] https://lkml.org/lkml/2021/4/27/1208 https://lkml.org/lkml/2021/4/27/1208
- londons_explore 5y agoLinux needs a standard set of benchmarks that are vaguely representative of things users use Linux for. Phone, laptop and a few server use cases. It then needs someone to do a big parameter tuning to select optimal settings. Too many decent algorithms don't make it into the kernel because there are too many tunables, and the ones that do typically arent well tuned for anyone's use case. Even big projects like Ubuntu typically don't change many tunables in the kernel.
- voldacar 5y agoI wonder if someday you could make a differentiable kernel and do gradient descent in kernel-parameter-space
- astrange 5y agoTuning is highly overrated in software performance, eg compiler optimizations don't help that much and searching their params helps even less. I think it comes from people wishing they could change their software without actually needing to learn how to change it.
- jeffbee 5y agoThat's quite a bad take. You can bank on PGO+LTO of a C++ server being good for 20% throughput easily, and not compiling for k8-generic when you have a modern machine also makes a big difference. I would agree that just flipping compiler flags might not get you very far, but there are key inputs that you shouldn't leave on the table.
- astrange 5y agoBy "not that much" I mean orders of magnitude. 20% is reasonable, especially for C++ that needs a lot of inlining. But a more modern language would benefit from defining some optimizations like inlining as mandatory, the same way tail calls are (and of course GCC has this.) Then there isn't a chance of deploying a build with extra-slow behavior.
- josephg 5y agoA performance improvement of 20% would be a massive win for something as large and widely used and as the linux kernel. That would average out into a few percentage points of performance improvement in every application across the globe which runs on linux / android. Merging a patch like that in linux would be like quietly reaching into every device across the planet and giving them a small, free CPU improvement and drop in power usage. Not game changing for any individual user, but huge in aggregate.
- arp242 5y agoAnd if you run a lot of servers it means you need fewer servers. These kind of changes can literally save millions.
- mhh__ 5y agoOne of the things that struck me recently is that working out what a computer is actually spending it's time doing is now almost a Master's thesis to do properly.
- adrianN 5y agoDepending on the level of analysis you can even get a PhD. See for example https://core.ac.uk/download/pdf/18462805.pdf https://core.ac.uk/download/pdf/18462805.pdf
- mhh__ 5y agoNot surprising at all, sadly.
- bayindirh 5y agoI've found it the hard way when I was trying to profile my own code. If you're using a few optimized libraries and designed your code for high-speed parallel execution, the compiled thing becomes almost impossible to trace in a practical manner.
- rwmj 5y agoDidn't perf work? What difficulties did you encounter?
- bayindirh 5y agoUsed perf record for getting CPU efficiency numbers, and it worked very well, however to be able to reliably trace the code, I had to compile it in debug mode with no optimizations, and run with a single thread. That combination increased the execution time considerably. Some of the Matrix and solver code coming from Eigen is heavily optimized for SIMD operations and using it with -O0 is just painful. Moreover, I had to verify its memory sanity and used Valgrind for that. Using a full size problem meant it had to churn for days to finish execution. Having deadlines doesn't always help in this stuff.
- 5y ago
- viraptor 5y ago> Linux needs a standard set of benchmarks that are vaguely representative of things users use Linux for. Have you seen phoronix framework + benchmarks? They use common tools, but did some good work on making the tests repeatable and accessible to anyone else. https://github.com/phoronix-test-suite/phoronix-test-suite/ https://github.com/phoronix-test-suite/phoronix-test-suite/
- fgeiger 5y agoLinaro is also doing a lot in that space (at least for ARM): https://www.linaro.org/os-build-and-test/ https://www.linaro.org/os-build-and-test/
- beebeepka 5y agoJust minor nitpick. Phoronix is one guy
- acer4666 5y agoWithout knowledge of preferred pronouns, "they" seems appropriate.
- UbrtrbNchDneRle 5y agoviraptor 1 : 0 the west.
- fgeiger 5y agoAs I read it, the point was not about the wrong pronoun but about the fact that the suite is developed by one person and not a large team and that the burden of benchmarking Linux performance should maybe not rest on one person alone.
- rat9988 5y agoYou missed his point. There was no wrong pronoun, this is what he is claiming.
- 5y ago
- diegocg 5y agoThere are already plenty of benchmarks
- api 5y agoA few server use cases? It's used for like 99% of all server use cases.
- guenthert 5y agoRight, problem solved. Now we just need to identify the few representative use cases. Is it people using a phone, as, er, phone with actual voice communication? or video conferencing? Or those who play games casually or not-quite-so-casually? Or video streaming? Or just for texting and would rather prioritize security and battery runtime? And that's just the phone. I could come up with many more questions regarding desktop and server usage as I'm more familiar with those. I'm afraid, there is no one optimal set of configuration options. Not even two or three.
- MaxBarraclough 5y agoAs someone with a limited knowledge of kernel stuff, does this mean Linux is likely to significantly outperform other comparable kernels like FreeBSD and OpenSolaris? Or are those other kernels keeping pace?
- marcodiego 5y agoConsidering the absolute dominance on HPC for a few years now, I don't think there is competition against Linux at the moment in terms of performance. Last time a non Linux system appeared in the top500 list was in June 2017.
- damagednoob 5y agoIsn't cost also a factor at play here?
- hawk_ 5y agoI suspect that's more due to "popularity" or richness of the ecosystem than performance necessarily.