7 ms·
Threadripper 3960X Compiles Linux Kernel in Under 30 Seconds
- pella 7y agodiscussion: https://news.ycombinator.com/item?id=21628149 https://news.ycombinator.com/item?id=21628149 Linux # Threadripper : https://news.ycombinator.com/item?id=21628482 https://news.ycombinator.com/item?id=21628482
- philliphaydon 7y agoI clicked to see discussion on the topic of the thread but you’re linking to threadripper discussion. Disappointed in lack of linux kernel compile time discussion.
- ghostpepper 7y agoNot trying to be rude, but genuinely curious: what else is there to discuss? Is this a new record, or just a record for the price point? What was the previous record? Will this have a large effect on how the kernel is developed? For what it's worth, I understand how hard it is to focus on a project when compile times get more than about thirty seconds.
- philliphaydon 7y agoWas thinking people would talk about compile times with their hardware or experiences with intel vs amd etc. It’s just this title is very specific to compile time compared to the other being very general discussion.
- linsomniac 7y agoIt is cool to see this level of parallelism coming to commodity hardware. 5-ish years ago a friend of mine had access to an experimental box one of the server vendors had made. Never went to production AFAIK. My memory is that it had 128 cores, 256 threads. One of the things we did on it, of course, was compile the kernel. I don't remember the exact numbers, but recall it being in the single digits number of seconds. This was done with just a ton of Xeon sockets.
- hrgiger 7y agoi think this the script they use: https://openbenchmarking.org/innhold/9a7355bdb73d85c9e044d020939c0ae9bcc1774e https://openbenchmarking.org/innhold/9a7355bdb73d85c9e044d02... on vanilla head branch myn "time make -s -j32" took "0m56.226s" with 1950x
- corysama 7y agoWe need to move on from the Linux Kernel and start using “compile Unreal Engine” as a CPU benchmark. https://gpuopen.com/threadripper-for-gamedev-ue4/ https://gpuopen.com/threadripper-for-gamedev-ue4/
- dragontamer 7y agoThat's probably a good point: the Linux Kernel is written using the C language. But C++ is a far more complicated language to parse and compile. Unreal Engine, Firefox, Chrome... large C++ programs easily take an hour to compile on normal hardware. With that being said: compiling for an hour is a huge hassle for reviewers. I think reviewers prefer benchmarks that complete in under a minute, rather than over an hour.
- michaellarabel 7y agoThat's why I also use the LLVM compilation test (C++) and other workloads besides just the Linux kernel... But it seems the Linux kernel results are always what generates the most interest.
- Matthias247 7y agoI would bet it's just familiarity. Most people that are interested in PCs have heard of Linux, and know that it has a Kernel. Or they even have already compiled kernels themselves. Whereas LLVM is more of a thing that is known to a subset of software developers. I'm personally happy to see the LLVM tests, since it gives a good impression on what benefits software developer which work on big native code-bases can expect.
- cesarb 7y agoI think it's path dependence. The kernel has a large number of compile-time configuration options; back in the 90s, it was common to compile your own kernel, after tuning these options to your own particular hardware. It was also common to get new kernel releases (as a source code .tar.gz, or as a patch to the previous source code .tar.gz) directly from kernel.org, instead of waiting for the next release of the distribution you were using. So the time it took to compile a new version of the kernel for your machine was something many Linux users had experience with, and it was clear when a machine was faster (perhaps as a consequence of having previously compiled and installed a newer kernel!) because it took less time to compile the kernel. To turn that into a benchmark was just a matter of standardizing on a kernel release and a set of configuration options.
- KirinDave 7y agoI don't really understand why this is newsworthy, so maybe someone could help me understand. It's a big, power hungry desktop CPU with incremental performance gains over the last generation. Why is this important? Is there some kind of architectural breakthrough these CPUs are using? Are these CPUs recovering performance lost to Spectre/Meltdown mitigation?
- barkingcat 7y agoThis is a big power hungry desktop CPU with incremental performance gains with the x86/x64 instruction set from someone not Intel and forcing intel to compete on pricing and performance. Just that alone is newsworthy.
- KirinDave 7y agoI think that's it: it's sort of an intel vs AMD interest piece. It's newsworthy in that it's a market signal, not a technology signal. The desktop CPU market segment is seeing a lot of transformation as secondary purpose built processors and SBCs become more and more of the market, so this is just sort of a fun puff piece. Thanks for helping me answer the question.
- Karliss 7y ago3970x almost halves the LLVM compilation time compared to 2990wx using the same core count. I would call that more than incremental performance gain.
- KirinDave 7y agoWhy? Isn't that within a modest error margin for the general arc of these? I'm looking at this graph over time and it seems like a linear projection wouldn't be broken by this. "The next generation of processors is not quite twice as fast as the current generation" just seems like a pretty normal statement to me.
- unethical_ban 7y ago
- abridgett 7y agoReminds me when they built a (much older, smaller) kernel on a pSeries (POWER architecture) in under 5 seconds: http://es.tldp.org/Presentaciones/200211hispalinux/blanchard/talk_2.html http://es.tldp.org/Presentaciones/200211hispalinux/blanchard... Gosh, 18 years ago, now I feel really old :D
- deleted 7y ago[deleted]
- l33tman 7y agoI was compiling the kernel for my boxes for the most part of the 90's and 00's and the funny thing is a sub-60-second compile time was always possible with the "latest gear". Like a 100 MHz pentium. Of course the number of modules you want/need to compile has steadily increased but still funny to see someone impressed with a 30 second compile time 25 years later and with a 100x+ increase in CPU power :)
- cesarb 7y agoThe kernel has gotten bloated. Back in the day, it could easily run in 8 megabytes of RAM, with plenty left over for the userspace and cache. Nowadays, just the kernel code (before loading any modules) is already bigger than that.
- l33tman 7y agoI was one of the first who worked on embedded Linux, and I put the kernel and a user-space program in a device with 512kb RAM once. It was too tight to do anything useful with, but just barely. It was no problem doing useful stuff on a 2 MB device (this was in 1999) :) Though to Linux defence, it is not nearly as bloated over this period of time as, say, Windows, by orders of magnitude. And, you aren't required to compile in everything. It's just very tedious (and frankly a bit pointless) to trim it down..
- xenospn 7y agoEverybody knows the REAL test is a FreeBSD 'make world' anyway :)
- basementcat 7y agoI used to think 'make world' on BSD was some big deal until I started to synthesize HDL code to bitfiles or GDS.
- Scipio_Afri 7y agoWhat is GDS in this context?
- ilaksh 7y agoNice. How long to build Chromium?
- fortran77 7y agoIf you want to keep these compile times, better not rewrite the kernel in Rust.
- Havoc 7y agoWas recently googling what the Kernel guys were using out of curiosity. GKH mentioned that he had access to 32 core AWS instances for it....7 years ago. Still...exciting times for consumers.
- kstenerud 7y agoWow... I still remember recompiling the NetBSD kernel on my Amiga 3000 to tweak the timings on my Retina Z3. Took 14 hours to compile the kernel... So if processors are now >1500x faster than they were in 1991, why is it that my Amiga 3000 has a faster, more responsive UI? Why is it that an ssh session takes 8 seconds to establish? Why is it that browsing a network share is 10x slower than viewing a file list was on a BBS?
- fooker 7y agoBecause not everything is CPU bound.
- yellowapple 7y agoBut all the other bounds have improved massively, too: - CPUs are multiple orders of magnitude faster - GPUs are multiple orders of magnitude faster - RAM is multiple orders of magnitude faster and more abundant - Disks are multiple orders of magnitude faster and bigger - Network connections are multiple orders of magnitude faster Why is it that the end result feels the same? The answer, of course, is that software complexity has kept pace: software, like a gas, expands until it fills its container.
- lordlimecat 7y agoNo, the answer is your ssh configuration. You're hanging on timeouts, either doing DNS resolution or trying multiple auth mechanisms. Use -vvv to find the hangup.
- yellowapple 7y agoI wasn't the author of the original comment. SSH ain't a problem for me. I was talking more about system responsiveness in general.
- vbezhenar 7y agoSomething's wrong with your DNS setup, I guess. SSH session takes less than one second to establish for me.
- ddevault 7y agoLLVM in under 2 minutes is way more impressive. That's a crazy CPU.
- beagle3 7y agoIt's a beast, but ... that's not a very impressive measure. Bellard strikes again - From [0], in 2004: TCCBOOT is a boot loader able to compile and boot a Linux kernel directly from its source code. It is only 138 KB big (uncompressed code) and it can compile and run a typical Linux kernel in less than 15 seconds on a 2.4 GHz Pentium 4. [0] https://bellard.org/tcc/tccboot.html https://bellard.org/tcc/tccboot.html [2004]
- anticensor 7y agoI would prefer gccboot (with gcc -march=native -mtune=native) though, because tccboot cannot optimise the output.