4 ms·
ThinLTO: Scalable and Incremental Link-Time Optimization for LLVM
- cm3 10y agoRelevant Rust discussion: https://www.reddit.com/r/rust/comments/4p4l5k/ https://www.reddit.com/r/rust/comments/4p4l5k/
- tmd83 10y agoI didn't understand fully what it took to achieve this but simply looking at the numbers it is so clear how impressive the work is. Near or better performance with enormously fast build time, significant improvement in incremental compilation and enormous reduction in memory usage. Its unusual to see an optimization attempt succeed from so many different angle.
- aseipp 10y agoIt's been in the works for quite a while (I remember reading some of the initial RFC postings), although I haven't seen people talk much of it until now. To be fair, I believe this work was primarily done by Google, who also have contributed a lot code to LTO/IPO-based systems in GCC before (such as "Lightweight IPO" or LIPO in GCC) as well as obvious, ongoing Clang/LLVM work. So I imagine they probably had a very good idea of what was necessary to achieve all that... But yes, the results are really impressive! Here are some slides from EuroLLVM last year with some pretty graphs and comparisons to traditional LTO/LIPO: http://llvm.org/devmtg/2015-04/slides/ThinLTO_EuroLLVM2015.pdf http://llvm.org/devmtg/2015-04/slides/ThinLTO_EuroLLVM2015.p...
- DannyBee 10y ago"Its unusual to see an optimization attempt succeed from so many different angle." David, Teresa, and Mehdi are really amazing engineers. (also, this is the third or so LTO system some of them have built :P)
- double051 10y agoThere's a session from WWDC 2016 that talks about this - https://developer.apple.com/videos/play/wwdc2016/405/ https://developer.apple.com/videos/play/wwdc2016/405/ The LTO part runs from 14:00 to 22:00. The streaming seems to only work in Safari (for Macs), but you can download the mp4s in HD (1.2 GB) or SD (300 MB). They do a good job of describing how it works and why this is kind of a big deal.
- corysama 10y agoFor non-Safari-users, the download is hiding in the grayed-out "Resources" tab.
- DannyBee 10y agoInteresting. Google built it because it's pretty necessary to scale to the size application we build. It looks like Apple started using it just because it's faster anyway ;)
- acqq 10y agoThe link to the PDF about the Incremental LTO is here, thanks to aseipp: https://news.ycombinator.com/item?id=11962571 https://news.ycombinator.com/item?id=11962571
- anarazel 10y agoSee also https://news.ycombinator.com/item?id=11950631 https://news.ycombinator.com/item?id=11950631
- jussaskin 10y agoIs all this functionality encoded within .o files or does it create a separate cache file or directory?
- jussaskin 10y agoIs it compatible with ccache?
- fefe23 10y agoThat's all well and good, but where can I get the code? Where can I play with this? How can Apple be shipping it (?) when LLVM is an open source project and it's not in the SVN head yet? Is there a special branch you need to use? A different repository perhaps?
- Sanddancer 10y agoIt's in TRUNK in with the rest of the llvm-lto stuff.
- ashitlerferad 10y agoIs LLVM still using SVN??
- gribbly 10y ago>How can Apple be shipping it (?) when LLVM is an open source project and it's not in the SVN head yet? Apple ships their own proprietary forked version of Clang+LLVM, this is for example how they implemented Swift, which was made open source long after it was introduced as part of Apple's proprietary toolchain.
- zump 10y agoNice work, mainly because the linker seems to be the most mysterious aspect of computer architecture.