9 ms·
> I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship,
by woodruffw 6y ago
> I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products.
As someone who works within LLVM professionally, I don't think this is particularly likely -- compilers are massive and complicated beasts, and the "moat" for proprietary improvements is small: they're either niche and therefore not of interest to the majority of programmers, or they're sufficiently general and easiest to maintain by releasing back upstream (where Apple, Google, and Microsoft will pay a small army of compiler engineers to keep them working).
Your concern is the one that kept GCC from stabilizing its various intermediate representations for decades, which is why virtually all program analysis research happens in LLVM these days.
Edit: To elaborate on the above: neither Apple, nor Google, nor Microsoft wants to individually maintain LLVM. Microsoft appears (to this outsider) to be actively looking to replace (parts of) MSVC/cl with the LLVM ecosystem, because they're tired on maintaining their own optimizing compiler.
- sroussey 6y agoBut Intel put proprietary work in ICC and their math libs, so there is something on periphery that’s going to keep the worry alive even if it doesn’t have a base.
- woodruffw 6y agoWell, to be clear, both ICC and MKL are fully proprietary. They do indeed justify the worry, but they're not in the same "open source but may be susceptible to proprietary creep" category.
- Manishearth 6y agoI agree, as someone who works on rustc I think it's a significant endeavor to maintain a substantial fork of either Rust or LLVM. (Rust does maintain a fork of LLVM, but it's just a couple minor patches that typically get upstreamed eventually)
- boris 6y ago> they're sufficiently general and easiest to maintain by releasing back upstream (where Apple, Google, and Microsoft will pay a small army of compiler engineers to keep them working). ...keep them working for their use-cases.
- woodruffw 6y ago> keep them working for their use-cases Are you under the impression that C and C++ code within Apple, Microsoft, and Google is fundamentally different from C and C++ code elsewhere? Because it isn’t. Google’s engineers maintain LLVM’s ASan for their purposes, but those purposes happen to be everybody else’s as well.
- geofft 6y agoRight, the reality is that if you want to be a profitable tech company today, you have to leverage open source. "Their use cases" includes, like, everything in a Linux distro. (And yes, this applies to Apple and Microsoft as well as Google.) The necessity of open source for industry has both negative and positive implications for those of us who care about open source / free software as an ideal and not simply a tool of capitalism. The negative one (which GCC's leadership failed to really internalize) is that the number of engineer-hours at the command of for-profit companies is much higher than the number of engineer-hours at the command of community-driven projects. If you deliberately build a worse product to prevent corporations from using it for profit, given enough time, the corporations will replace it. The positive one, though, is that those engineer-hours are generally more valuable when pointed at some common cross-company codebase unless it is the specific thing that makes you money, and FOSS as an ideal provides a well-accepted model under which they can organize cross-company work (doubly so when they employ idealists like us as implementors). A compiler makes very few people money directly. It's generally a tool that you want to work well, and it's helpful to have other people run into the majority of problems and fix them before they cause trouble for the things that do make money. So it's not surprising that LLVM is catching up with GCC, nor is it surprising that LLVM is and remains open source. If you are concerned about the LLVM monoculture, build a competitor such that it is cheaper / more profitable for companies to work on your competitor than to either work on LLVM or build their own compiler. Figure out what will make companies want to contribute and encourage it. (GCC did not do this, but it is perhaps slowly relaxing here.) If you are concerned about LLVM becoming proprietary, make it so that is cheaper / more profitable for companies to release their changes instead of holding onto them; that is, figure out what will make companies feel like they must contribute and encourage it. (One common strategy, used by Linux itself, is to maintain a high level of internal API churn coupled with genuinely good improvements in that churn and a policy that people must update in-tree callers; at that point, the more API surface you use in your private fork, the farther behind you get, and you'll watch competing companies outpace you.)
- fractalb 6y agoWhat if those bigcorps don't merge some improvements for getting an edge over the competitors? And, what if every one of those bigcorps do the same? How would that impact the project in the future?
- zelly 6y agoAlready the case for Apple LLVM which has optimizations for Apple Silicon not upstreamed and never will be upstreamed.
- bluGill 6y agoThey pay the costs of maintaining their fork. Thus there will be constant internal chatter of is the non-merged stuff really valuable enough considering the cost of maintenance. Sometimes yes, but sometimes no as well. So overall that is a long term win.
- pjmlp 6y agoI don't consider the improvements that ARM, Apple, Sony, CodePlay, NVidia, AMD, among others, don't upstream due to IP considerations or revealing of hardware secrets, niche.
- _ph_ 6y agoBut if they have IP considerations or trade secrets to protect, they wouldn't contribute them to any other open source project either. This way, they can at least upstream everything, which doesn't fall under these restrictions.
- pjmlp 6y agoThey don't upstream everything, while reducing their development costs, that is the point. Personally I don't care, but I bet many FOSS anti-GPL advocates will eventually care.
- _ph_ 6y agoThey don't upstream what they never would be up streamed. But they probably they upstream a lot which they wouldn't have, if they couldn't use the open source tool at all. Also, a lot of companies reduce their development costs by using open source software, gpl or other, without ever upstreaming. So "saving development cost" doesn't sound like an argument to me.
- pjmlp 6y agoWith GPL there is a legal tool to force contribution, though. As mentioned I don't care, after university my main UNIX platforms were HP-UX, Aix and Solaris with their respective system compilers anyway. Linux is already getting replacement candidates in IoT space via Zephyr, NuttX, mbed, RTOS, Azure RTOS, and who knows if Fuchsia will eventually get out of the lab, so be it.
- _ph_ 6y agoWith GPL there is a legal tool to force contribution, though. No, for practical purposes, there is not. You can only enforce delivery of the source when a product based on GPL software gets delivered. But of course companies, which create software, know about this. Any usage of GPL software for delivered products only happens after the decision to publish the created software has been made. In doubt, companies tend to not use GPL software as part of deliveries.
- dehrmann 6y ago> or they're sufficiently general and easiest to maintain by releasing back upstream Exactly; maintaining a fork is a big pain. The ongoing cost of keeping it up-to-date is a pretty big incentive to merge it, even if there's not a legal requirement.
- afiori 6y agomany argue that a legal requirement is detrimental in practice, as in that case the company cannot independently decide how much to upstream. with GPL you need a legal team to decide and plan how much to use, with BSD/MIT it can be an afterthought.
- stefan_ 6y agoThat is a weird way to say "your concerns were entirely correct". Every single proprietary mobile graphics driver uses LLVM for its shader compiler and the lack of copyleft has severely curtailed progress on open-source drivers. Of course none of it goes back to LLVM because updating your production copy of LLVM is just as messy and broken as the non-stable GCC IR you complain about and the fact that OSS development is still fundamentally incompatible with 1-year industry release schedules that shift all the time.
- Narishma 6y ago> Every single proprietary mobile graphics driver uses LLVM for its shader compiler and the lack of copyleft has severely curtailed progress on open-source drivers. And you think if they didn't have the option of using LLVM they would have released an open source driver instead? That makes no sense to me.
- bryanlarsen 6y agoYes. Developing a state of the art optimizing compiler is hard. If their only choices are developing their own compiler or basing off of gcc and releasing their work, they'd choose gcc.
- jasode 6y ago> If their only choices are developing their own compiler or basing off of gcc and releasing their work, they'd choose gcc. But that's the exact choice Apple faced in 2005 and they did not choose gcc. They paid Chris Lattner and his team to develop an alternative compiler. Excerpt from wikipedia[1]: >"Finally, GCC is licensed under the terms of GNU General Public License (GPL) version 3, which requires developers who distribute extensions for, or modified versions of, GCC to make their source code available, whereas LLVM has a BSD-like license which does not have such a requirement. >Apple chose to develop a new compiler front end from scratch, supporting C, Objective-C and C++. This "clang" project was open-sourced in July 2007." For some reason, the enthusiastic focus on the benefits of GPL principles seems to ignore the actual game theoretic behavior of actors in the real world to not choose GPL at all. [1] https://en.wikipedia.org/wiki/Clang#Background https://en.wikipedia.org/wiki/Clang#Background
- R0b0t1 6y agoOnce GCC is no longer the state of the art it is unlikely MIT/BSD licensed alternatives will have their contributions publicly released.