9 ms·

Bringing Clang to Windows

142 points by intrepidsoldier 11y ago

twoodfin 11y ago
Where does C2 come in on the stack? Is it compiling/translating LLVM IR?
- intrepidsoldier 11y ago
  The latter.
Phlow 11y ago
What's the situation with SEH these days in Clang?
- spongo2 11y ago
  http://clang.llvm.org/docs/MSVCCompatibility.html http://clang.llvm.org/docs/MSVCCompatibility.html says minimal. Note that this is about cross-plat conformant c++ code. SEH is windows specific.
  - cokernel_hacker 11y ago
    That document is out of date. SEH works OK for 64-bit X86, 32-bit X86 is ongoing. Catching C++ exceptions has been implemented for 64-bit X86 but isn't ready for prime time.
revelation 11y ago
I'm a bit confused about how this will work exactly. If I have Clang and VC++ compiled code trying to interop with each other, what happens if Clang decided to use a different memory layout for std::string (or any other type really) than VC++? What compiler is doing which task in the pipeline here exactly? I mean, the reason everyone does interop strictly in C is that there are zero standards for how a compiler implements the C++ features exactly down-low.
- ComputerGuru 11y ago
  As I understand it, it's not bringing Clang/LLVM's libc++ to Windows. The memory layout of, for example, std::string isn't decided by the compiler - it's decided by the library/header files. Both Clang and MSVC would be using the same MS C++ header files, so the only thing up to the compiler would anything that isn't specified in the header files, behind the scenes. We're already using Clang with msbuild on Windows for native C11 support (come on, Microsoft!), courtesy of pre-built Clang for Windows binaries [1] - using the Microsoft C/C++ headers and linking against the Microsoft C/C++ runtimes. There's already a plugin for MSVC that makes the Clang toolset available. VS 2013 (or maybe it was 2012?) decoupled the MSVC build chain/toolset from the IDE, making it possible to swap out which toolchain you're using from the project properties. Actually, it may have even been VS 2010, because I was using it to compile with ICC, but then again, I remember having a hell of a hard time so perhaps it was VS2012/2013. 1: http://llvm.org/docs/GettingStartedVS.html http://llvm.org/docs/GettingStartedVS.html
  - spongo2 11y ago
    correct, this will use MS libraries. If you watch Jim Radigan's Build talk (he owns C2), he talks a little bit about how the ABI is really a part of the platform and it might clarify some of this. http://channel9.msdn.com/Events/Build/2015/3-714 http://channel9.msdn.com/Events/Build/2015/3-714 about 23 minutes in. Thanks, Steve (vc dev mgr)
    - DannyBee 11y ago
      Clang was already planning on using MS libraries/headers. IE this is literally not different from what was already being done, without MS :)
      - spongo2 11y ago
        yes, we are well aware that clang and llvm is working on targeting windows. this is clang+c2 which we will fully support.
  - 11y ago
mwcampbell 11y ago
If the Windows team at MS thinks of c2 and the MS linker as part of the Windows ABI, then I guess the MinGW toolchain (and a hypothetical clang toolchain where LLVM generates the native code) will always be second class, and therefore best avoided by developers who want to target Windows, even for classic Win32 desktop apps.
- coliveira 11y ago
  It all depends on how cutting edge you want to be in the windows platform. If you want access to the latest technology, MS tools will always be the necessary choice -- Borland learned this lesson after a lot of hard work. But if you want to create products using the stable version of the windows API, gcc is already a very good choice.
  - spiralpolitik 11y ago
    I kind of disagree here. Compilers reached commodity status a long time ago. Nobody makes money off them compared to how it was in the 90s and the list of ex-compiler makers is as long as your arm (Watcom, Metrowerks etc). 'clang' is probably approaching the point that its good enough so the question for Microsoft is do they still see any value in continuing the develop their own compiler or should they just switch to 'clang' ? I think this is the big shift we are seeing in MS this year. Instead trying to owning everything, they seem to be shifting their focus to owning the parts that have value and commoditizing the rest. The strategy was successful for Apple so its unsurprising that Microsoft is looking to a similar strategy as part of their salvation.
    - coliveira 11y ago
      I think they want compatibility with other compilers, but unless they move alway from Windows, there is a lot of their APIs that require close compiler/library support: COM, DCOM+, .NET, etc. They will probably need at least to maintain VC++ for a long time.
      - intrepidsoldier 11y ago
davidgerard 11y ago
> and is fast becoming the preferred C++ compiler for non-Windows platforms. They wish. Microsoft still hates GNU.
- ComputerGuru 11y ago
  It is, though. And Microsoft isn't alone in hating GNU - FreeBSD no longer ships with or uses GCC; the entire FreeBSD kernel and userland is built with Clang these days. And, of course, Apple's the one that started this whole stampede away from GCC in the first place. LLVM/Clang really and truly have hit GCC hard - and definitely for the best. Look at the improvements to GCC spurred by newer features in the Clang build toolchain. EDIT: In reply to davidgerard's comment below: > I will not believe this falsifiable claim without numbers. FreeBSD has almost no users compared to Linux. Non-Windows does not mean Linux. And what falsifiable claim? They didn't say Clang is the most popular - they said it's becoming. But if that's really the game you want to play, then it's Linux that has almost no users compared to the number of iPhones/iPads/iWatches/MacBooks out there - Clang/LLVM-powered, each and every one of them.
  - davidgerard 11y ago
    I will not believe this falsifiable claim without numbers. FreeBSD has almost no users compared to Linux.
    - CurtHagenlocher 11y ago
      Unless you think of iOS and OS X as flavors of FreeBSD.
      - davidgerard 11y ago
        Mac OS X is separately accountable, and that's <i>entirely</i> because Apple hates GPLv3.
        tzs 11y ago
istvan__ 11y ago
Hell yeah! Microsoft just became one of the viable platforms this week in the cheap segment with reasonable licensing.
- yarrel 11y ago
  Still not in the free as in speech and beer segment though.
malkia 11y ago
Wow! Microsoft! Wow! You should've had Satya Nadella as CEO ages ago! Congrats!
phkahler 11y ago
I'm starting to see a strategy here. I've been wondering how Microsoft can make money from all.this open source activity. Their platform is becoming obsolete, but is being replaced by OSX, iOS, and android. That makes developers unhappy because of the multiple OSes they need to support. But MS can step in and bring their own platform as an abstraction over these others and make developers happy. Build once, target everything. I'm betting key pieces will not become open source, they want to have control over the new platform. That is the midterm goal - regain control (even a little bit) of a dominant platform.
- iamcreasy 11y ago
  (sigh)...
- wvenable 11y ago
  I disagree because bringing Objective-C, Java/Android, and CLang compilation to Visual Studio doesn't provide any sort of abstraction. The support for Objective-C, Clang, etc is, in my opinion, merely about bringing iOS and Android software to Windows (especially for phones and tablets). Microsoft knows they need current high quality apps on their platform to get users. Users which will buy their devices and operating systems. They don't really want developers to run iOS software unmodified on Windows but they do want to make it really easy for them to attempt a port.
- PeteJacobs23 11y ago
  Windows is not becoming obsolete. Quite the opposite. In the enterprise Windows will stay for a long time. I'm working at company > 500000 employees and our IT will be very happy when they can replace the iOS devices with Windows equivalents. And certainly the Desktops aren't going to be replaced by OSX. Even more with Windows 10 around. MS strategy is much more straight forward. Simply offer reasons and make it easier for customers to use their products. Mainly the cloud and mobile services. They're just smart enough to understand where Google and Facebook make their money with - not by selling OS.
  - Demiurge 11y ago
DannyBee 11y ago
I agree with folks who think this is a bit confusing. This blog post is announcing: 1. Using clang as a frontend. 2. Using C2 (MS compiler) as a backend. What the LLVM community (IE not MS so far, but participation is of course, welcome)) has been building: 1. Using clang as a frontend 2. Using llvm as a backend and even more specifically 1. Using MS Headers + MS Libraries 2. Using clang as a frontend 3. Using llvm as a backend. (see blog.llvm.org/2014/07/clangllvm-on-windows-update.html for a 9 month old view) Some of the mingw32 folks have been working on replacing #1 with non-ms stuff, but let's put that aside for the sake of this description ) Using clang as a frontend is where almost all of the ABI compatibility happens anyway (and where various developers have been focusing), so you end up with VC++ ABI compatible binaries whether you use C2 or not, if you/they use the pre-existing clang stuff. Whether they are doing that (and then lowering llvm ir to c2 ir) , or if they've built an alternative IR generation with clang (IE generating c2 ir directly from clang), that they haven't released, is not clear. If they've built something that generates C2 IR from clang, they would have had to write their own ABI compat, which would mean it may be different than the binaries clang would build. (exception handling is also codegen'd by llvm, so exception handling could end up different, though of course, compatibility is the goal)
- clord 11y ago
  We went through this at my former employer (not MS) just before i left. In fact the politics surrounding it drove me to seek relative sanity of doing a startup. When I started working on it, there really were two camps: clang emitting house IR instead of LLVM IR, or translating LLVM IR either early (right out of clang) or late (as a target after all optimization). I was working on the prototypes for the latter pair of options before another constraint was added that invalidated all of those paths. The problem was that clang is quite intertwined with LLVM IR. ABI is spread all over the place, both FE and BE. It was proving hard to get something the house optimizers/code generators would be happy with. So a more radical path was chosen — wiring clang's parser + ASTs to another C++ front-end's AST and then using the old IR generator to get all of the idiosyncrasies right for the target. It was wacky, but seemed to be half-way working when I left, such that major open source code was compiling. The whole idea of taking just parts of clang/llvm reeks of a half-measure anyway. It's probably better to either keep the existing compiler stack, or cut it right out and put the secret sauce into LLVM as add-in modules.
__Joker 11y ago
The broader stroke is MS to be relevant in the platforms side of the story. Sometime people will argue that windows is used extensively in enterprises, but obviously the today startup which will be tomorrows enterprises are using almost no MS eco system.
nerraga 11y ago
At approx. 26min he mentions four processors that their code generator has to target: ARM32, x86, x64, and a new one coming out in the fall. What would that be? ARM64? *edit - never mind. Last five seconds they hint at it as arm64. MS is definitely doing some interesting stuff these days.
brentshields 11y ago
"So, any minor divergence in features of Clang and Visual C++ will cause you as a developer to have to worry about these compiler differences or limit you to using the smallest subset of the two compiler features" In other words, the set of vc++ features.
- intrepidsoldier 11y ago
  Yes. We know we are behind on conformance.

twoodfin 11y ago

Where does C2 come in on the stack? Is it compiling/translating LLVM IR?

intrepidsoldier 11y ago
The latter.

Phlow 11y ago

What's the situation with SEH these days in Clang?

spongo2 11y ago
http://clang.llvm.org/docs/MSVCCompatibility.html http://clang.llvm.org/docs/MSVCCompatibility.html says minimal. Note that this is about cross-plat conformant c++ code. SEH is windows specific.
- cokernel_hacker 11y ago
  That document is out of date. SEH works OK for 64-bit X86, 32-bit X86 is ongoing. Catching C++ exceptions has been implemented for 64-bit X86 but isn't ready for prime time.

revelation 11y ago

I'm a bit confused about how this will work exactly. If I have Clang and VC++ compiled code trying to interop with each other, what happens if Clang decided to use a different memory layout for std::string (or any other type really) than VC++? What compiler is doing which task in the pipeline here exactly? I mean, the reason everyone does interop strictly in C is that there are zero standards for how a compiler implements the C++ features exactly down-low.

ComputerGuru 11y ago
As I understand it, it's not bringing Clang/LLVM's libc++ to Windows. The memory layout of, for example, std::string isn't decided by the compiler - it's decided by the library/header files. Both Clang and MSVC would be using the same MS C++ header files, so the only thing up to the compiler would anything that isn't specified in the header files, behind the scenes. We're already using Clang with msbuild on Windows for native C11 support (come on, Microsoft!), courtesy of pre-built Clang for Windows binaries [1] - using the Microsoft C/C++ headers and linking against the Microsoft C/C++ runtimes. There's already a plugin for MSVC that makes the Clang toolset available. VS 2013 (or maybe it was 2012?) decoupled the MSVC build chain/toolset from the IDE, making it possible to swap out which toolchain you're using from the project properties. Actually, it may have even been VS 2010, because I was using it to compile with ICC, but then again, I remember having a hell of a hard time so perhaps it was VS2012/2013. 1: http://llvm.org/docs/GettingStartedVS.html http://llvm.org/docs/GettingStartedVS.html
- spongo2 11y ago
  correct, this will use MS libraries. If you watch Jim Radigan's Build talk (he owns C2), he talks a little bit about how the ABI is really a part of the platform and it might clarify some of this. http://channel9.msdn.com/Events/Build/2015/3-714 http://channel9.msdn.com/Events/Build/2015/3-714 about 23 minutes in. Thanks, Steve (vc dev mgr)
  - DannyBee 11y ago
    Clang was already planning on using MS libraries/headers. IE this is literally not different from what was already being done, without MS :)
    - spongo2 11y ago
      yes, we are well aware that clang and llvm is working on targeting windows. this is clang+c2 which we will fully support.
- 11y ago

mwcampbell 11y ago

If the Windows team at MS thinks of c2 and the MS linker as part of the Windows ABI, then I guess the MinGW toolchain (and a hypothetical clang toolchain where LLVM generates the native code) will always be second class, and therefore best avoided by developers who want to target Windows, even for classic Win32 desktop apps.

coliveira 11y ago
It all depends on how cutting edge you want to be in the windows platform. If you want access to the latest technology, MS tools will always be the necessary choice -- Borland learned this lesson after a lot of hard work. But if you want to create products using the stable version of the windows API, gcc is already a very good choice.
- spiralpolitik 11y ago
  I kind of disagree here. Compilers reached commodity status a long time ago. Nobody makes money off them compared to how it was in the 90s and the list of ex-compiler makers is as long as your arm (Watcom, Metrowerks etc). 'clang' is probably approaching the point that its good enough so the question for Microsoft is do they still see any value in continuing the develop their own compiler or should they just switch to 'clang' ? I think this is the big shift we are seeing in MS this year. Instead trying to owning everything, they seem to be shifting their focus to owning the parts that have value and commoditizing the rest. The strategy was successful for Apple so its unsurprising that Microsoft is looking to a similar strategy as part of their salvation.
  - coliveira 11y ago
    I think they want compatibility with other compilers, but unless they move alway from Windows, there is a lot of their APIs that require close compiler/library support: COM, DCOM+, .NET, etc. They will probably need at least to maintain VC++ for a long time.
    - intrepidsoldier 11y ago

davidgerard 11y ago

> and is fast becoming the preferred C++ compiler for non-Windows platforms. They wish. Microsoft still hates GNU.

ComputerGuru 11y ago
It is, though. And Microsoft isn't alone in hating GNU - FreeBSD no longer ships with or uses GCC; the entire FreeBSD kernel and userland is built with Clang these days. And, of course, Apple's the one that started this whole stampede away from GCC in the first place. LLVM/Clang really and truly have hit GCC hard - and definitely for the best. Look at the improvements to GCC spurred by newer features in the Clang build toolchain. EDIT: In reply to davidgerard's comment below: > I will not believe this falsifiable claim without numbers. FreeBSD has almost no users compared to Linux. Non-Windows does not mean Linux. And what falsifiable claim? They didn't say Clang is the most popular - they said it's becoming. But if that's really the game you want to play, then it's Linux that has almost no users compared to the number of iPhones/iPads/iWatches/MacBooks out there - Clang/LLVM-powered, each and every one of them.
- davidgerard 11y ago
  I will not believe this falsifiable claim without numbers. FreeBSD has almost no users compared to Linux.
  - CurtHagenlocher 11y ago
    Unless you think of iOS and OS X as flavors of FreeBSD.
    - davidgerard 11y ago
      Mac OS X is separately accountable, and that's <i>entirely</i> because Apple hates GPLv3.
      - tzs 11y ago

istvan__ 11y ago

Hell yeah! Microsoft just became one of the viable platforms this week in the cheap segment with reasonable licensing.

yarrel 11y ago
Still not in the free as in speech and beer segment though.

malkia 11y ago

Wow! Microsoft! Wow! You should've had Satya Nadella as CEO ages ago! Congrats!

phkahler 11y ago

I'm starting to see a strategy here. I've been wondering how Microsoft can make money from all.this open source activity. Their platform is becoming obsolete, but is being replaced by OSX, iOS, and android. That makes developers unhappy because of the multiple OSes they need to support. But MS can step in and bring their own platform as an abstraction over these others and make developers happy. Build once, target everything. I'm betting key pieces will not become open source, they want to have control over the new platform. That is the midterm goal - regain control (even a little bit) of a dominant platform.

iamcreasy 11y ago
(sigh)...
wvenable 11y ago
I disagree because bringing Objective-C, Java/Android, and CLang compilation to Visual Studio doesn't provide any sort of abstraction. The support for Objective-C, Clang, etc is, in my opinion, merely about bringing iOS and Android software to Windows (especially for phones and tablets). Microsoft knows they need current high quality apps on their platform to get users. Users which will buy their devices and operating systems. They don't really want developers to run iOS software unmodified on Windows but they do want to make it really easy for them to attempt a port.
PeteJacobs23 11y ago
Windows is not becoming obsolete. Quite the opposite. In the enterprise Windows will stay for a long time. I'm working at company > 500000 employees and our IT will be very happy when they can replace the iOS devices with Windows equivalents. And certainly the Desktops aren't going to be replaced by OSX. Even more with Windows 10 around. MS strategy is much more straight forward. Simply offer reasons and make it easier for customers to use their products. Mainly the cloud and mobile services. They're just smart enough to understand where Google and Facebook make their money with - not by selling OS.
- Demiurge 11y ago

DannyBee 11y ago

I agree with folks who think this is a bit confusing. This blog post is announcing: 1. Using clang as a frontend. 2. Using C2 (MS compiler) as a backend. What the LLVM community (IE not MS so far, but participation is of course, welcome)) has been building: 1. Using clang as a frontend 2. Using llvm as a backend and even more specifically 1. Using MS Headers + MS Libraries 2. Using clang as a frontend 3. Using llvm as a backend. (see blog.llvm.org/2014/07/clangllvm-on-windows-update.html for a 9 month old view) Some of the mingw32 folks have been working on replacing #1 with non-ms stuff, but let's put that aside for the sake of this description ) Using clang as a frontend is where almost all of the ABI compatibility happens anyway (and where various developers have been focusing), so you end up with VC++ ABI compatible binaries whether you use C2 or not, if you/they use the pre-existing clang stuff. Whether they are doing that (and then lowering llvm ir to c2 ir) , or if they've built an alternative IR generation with clang (IE generating c2 ir directly from clang), that they haven't released, is not clear. If they've built something that generates C2 IR from clang, they would have had to write their own ABI compat, which would mean it may be different than the binaries clang would build. (exception handling is also codegen'd by llvm, so exception handling could end up different, though of course, compatibility is the goal)

clord 11y ago
We went through this at my former employer (not MS) just before i left. In fact the politics surrounding it drove me to seek relative sanity of doing a startup. When I started working on it, there really were two camps: clang emitting house IR instead of LLVM IR, or translating LLVM IR either early (right out of clang) or late (as a target after all optimization). I was working on the prototypes for the latter pair of options before another constraint was added that invalidated all of those paths. The problem was that clang is quite intertwined with LLVM IR. ABI is spread all over the place, both FE and BE. It was proving hard to get something the house optimizers/code generators would be happy with. So a more radical path was chosen — wiring clang's parser + ASTs to another C++ front-end's AST and then using the old IR generator to get all of the idiosyncrasies right for the target. It was wacky, but seemed to be half-way working when I left, such that major open source code was compiling. The whole idea of taking just parts of clang/llvm reeks of a half-measure anyway. It's probably better to either keep the existing compiler stack, or cut it right out and put the secret sauce into LLVM as add-in modules.

__Joker 11y ago

The broader stroke is MS to be relevant in the platforms side of the story. Sometime people will argue that windows is used extensively in enterprises, but obviously the today startup which will be tomorrows enterprises are using almost no MS eco system.

nerraga 11y ago

At approx. 26min he mentions four processors that their code generator has to target: ARM32, x86, x64, and a new one coming out in the fall. What would that be? ARM64? *edit - never mind. Last five seconds they hint at it as arm64. MS is definitely doing some interesting stuff these days.

brentshields 11y ago

"So, any minor divergence in features of Clang and Visual C++ will cause you as a developer to have to worry about these compiler differences or limit you to using the smallest subset of the two compiler features" In other words, the set of vc++ features.

intrepidsoldier 11y ago
Yes. We know we are behind on conformance.