7 ms·

Zig: Drop MinGW-w64

105 points by goranmoomin 5y ago

iopq 5y ago
Rust on Windows used to run on Mingw, but has since moved on It made it a lot easier in many ways to just compile a thing and have it work without thinking about mingw and interfacing C from mingw
anonymfus 5y ago
Hmm... https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention https://docs.microsoft.com/en-us/cpp/build/x64-calling-conve... > Return values > A scalar return value that can fit into 64 bits, including the __m64 type, is returned through RAX. Non-scalar types including floats, doubles, and vector types such as __m128, __m128i, __m128d are returned in XMM0. The state of unused bits in the value returned in RAX or XMM0 is undefined. > User-defined types can be returned by value from global functions and static member functions. To return a user-defined type by value in RAX, it must have a length of 1, 2, 4, 8, 16, 32, or 64 bits. It must also have no user-defined constructor, destructor, or copy assignment operator. It can have no private or protected non-static data members, and no non-static data members of reference type. It can't have base classes or virtual functions. And, it can only have data members that also meet these requirements. (This definition is essentially the same as a C++03 POD type. Because the definition has changed in the C++11 standard, we don't recommend using std::is_pod for this test.) Otherwise, the caller must allocate memory for the return value and pass a pointer to it as the first argument. The remaining arguments are then shifted one argument to the right. The same pointer must be returned by the callee in RAX. Isn't this what https://github.com/ziglang/zig/issues/9998#issuecomment-997088467 https://github.com/ziglang/zig/issues/9998#issuecomment-9970... claims to be undocumented?
- anonymfus 5y ago
  Following comments on Github also point this out.
- Andoryuuta 5y ago
  > Otherwise, the caller must allocate memory for the return value and pass a pointer to it as the first argument. The remaining arguments are then shifted one argument to the right. I think the issue is that it isnt _fully_ documented anywhere. The MS docs do partially describe it, but it omits details about how this should be handled in the case of C++ member functions (which are the COM methods in question). C++ member functions always receive the hidden 'this' pointer as the first argument in Microsoft's x64 calling convention, which directly conflicts with this document stating the returned struct would be a "pointer to it as the first argument". You can see this in some of the godbolt links in the linked issue. Ultimately, some COM functions that take no arguments actually end up taking two hidden parameters in rcx and rdx (this pointer, and the return struct pointer respectively)
bla3 5y ago
Windows uses its own calling convention. I'm sure COM win32 headers have no shortage or atrocities, but this post here is just saying that GCC doesn't implement the Windows calling convention for struct returns, while clang does.
jmull 5y ago
It’s interesting to see the process at work.
olvy0 5y ago
As someone who has to deal from time to time with code that is a mix of old COM components and newer mingw dependencies I found this discussion fascinating. I learned to avoid most issues by wrapping everything at the boundaries with a pure C api/abi, and not letting exceptions propagate at all, instead catching them and returning an error code. I wonder why zig can't do the same but I guess they want calls to directx etc to match what's documented instead of going through c like contortions.
chasil 5y ago
I know that Clang runs on Windows, as PuTTY's "about" dialog credits it. Does this offer a reasonable replacement for MinGW? https://wetmelon.github.io/clang-on-windows.html https://wetmelon.github.io/clang-on-windows.html
- malkia 5y ago
  There is CLANG environment support for MSYS2 - https://www.msys2.org/docs/environments/ https://www.msys2.org/docs/environments/
cyber_kinetist 5y ago
My last experience with MinGW-w64 was when I was trying to compile my C++ simulation code in Windows and finding out that AVX instructions were not working because the compiler had misalignment-related bugs. (The issue is still open in https://github.com/msys2/MSYS2-packages/issues/1209 https://github.com/msys2/MSYS2-packages/issues/1209) MinGW/MSYS certainly had appeal to former Linux devs who didn't want to touch the horrors of MSVC, but Zig (with its included Clang compiler/runtime) might end up being a better solution for people trying to compile C/C++ code on Windows in a stable manner.
- colejohnson66 5y ago
  Wow. “mingwandroid” is coming off as pretty rude: > Is it really so difficult to determine the correct place to report issues to?
jbk 5y ago
That’s the opposite of our experience on VLC and Firefox was sharing our opinion: mingw-w64 people are skilled, nice and very clever and think about all use cases. This is how we are able to support configurations that even MS does not support… Also, a contrario from what this threads seems to say, the Windows SDK headers are faaaar from being open source compatible
- badsectoracula 5y ago
  Yeah, i've been using MinGW (and now MinGW-w64) for practically decades now, originally via MSYS and later via MSYS2 and i never remember having and real issues with it - i even did some DirectX programming with it. The only thing i remember being an issue at some point (and perhaps still is) is that it relies on msvcrt.dll that theoretically isn't part of the Windows platform but in practice that DLL has been there since Windows 98 and the chances of this ever being a real problem are zero - so it is really an "issue" about discussions with hairsplitting pedants than a practical one :-P.
  - slimsag 5y ago
    If you squint closely, you'll see my issue is not hairsplitting but rather a practical one about trouble with MinGW headers not being super up-to-date (missing DirectX 12 APIs, etc.) I'm a huge fan of MinGW-w64, though. I would bet most involved in the issue are. We all win by improving the status quo - I'm just a buffoon figuring out the best way to do that :)
    - jbk 5y ago
      DirectX 12 API is a pretty small case to throw all the project to trash. Notably since DirectX headers are always a different case compared to the rest of the Windows SDK
      - slimsag 5y ago
        I'm a massive fan of your work on VLC, but I feel that's a bit of a mischaracterization.. The author of the issue spoke about his troubles with MinGW in four bullet points at the top that are not related to DirectX at all, whether you agree or disagree with those the fact is most people (including Andrew Kelley) rebutted many of his arguments, and Andrew even reached out to MinGW devs on IRC to chat about the topic of contributing upstream. That seems a reasonable approach to me. I chimed in on the issue because DirectX 12 was the problem _I_ faced, explained what I learned, complained I still don't have a super official way to get up-to-date DirectX headers that are compatible with MinGW, and ultimately said I gave up on working on the issue out of frustration. Throughout the issue there are people suggesting throwing out all of MinGW would be reckless, heck even I concluded that would not be possible for multiple reasons in my write-up. I think people are reading too much into the title and trying to make this out to be some sort of attack on MinGW, I really do not think that was anyone's intent - we're just exploring if we can improve the status quo..
jchw 5y ago
I think the MinGW project is very useful, but I simply do not recommend using it. My experience with it was trying to debug why a friend’s build was working on Windows and not Wine. I figured it out: it’s because MinGW implements something called pseudo relocations, which implement symbol-relative relocations inside the entrypoint of your executable. It’s needed because when compiling, you don’t know if an unresolved symbol will be external or not, so it gets compiled as if it’s not. But when linking, and you finally know that the symbol is external, it’s too late to change the code to refer to an import table indirection. On ELF platforms, relocations can refer to symbols in other libraries during runtime linking. So external symbols can be treated very similarly to internal ones, just with linking done at runtime instead of compile-time. On Windows, you have to use `__declspec(dllimport)` or some such to inform the compiler that the symbol will be external. It can then generate code for referring to the IAT. MinGW wanted to support compiling UNIXy software that didn’t use this attribute, so it does not require it. Instead… the ELF-style pseudo-relocations are used. So at the entrypoint, your MinGW program or DLL loops through a table of pointers inside machine code, looks up their real address in the IAT, and performs fixups. It will even temporarily change the protection of the relevant pages to be writable. Check to see if your MinGW program has unexpected imports to kernel32.VirtualProtect! (IIRC this can also happen if you trigger executable stack by using GCC trampolines, another reason to not use GCC…) (edit: Please read the replies below, but according to MinGW developers, this only happens in some edge cases; ordinarily, thunks will be used, which do not have this problem. In addition, MinGW binaries will always link to VritualProtect, because the pseudo-reloc code exists whether or not you use them. I never noticed it before, so that's interesting.) There’s only one problem: this is extremely failure-prone. For example, if you are on AMD64, the typical CALL instruction contains a 32-bit RIP-relative offset. But in a 64-bit process space, two modules can happily be more than 2^32 bytes away in memory. I assume ELF platforms have a linker that’s aware of this, since the linker knows about these special relocations, and thus has to ensure the binaries are in range… but I don’t know much about ELF linking, so don’t take that as fact. So why does this break on Wine and not Windows? Well, as far as I can tell, it doesn’t break on Windows because for some reason ASLR never puts any of the modules too far away. I haven’t figured out why. Wine doesn’t implement ASLR and loads modules at their preferred addresses as long as it’s available, and those addresses wind up being too far away. The pseudo reloc code does not output any errors when the address overflows; would probably be worth a pull request, but I am still puzzling with why Windows doesn’t ever seem to cause the problem. Unless I am missing something crucial, it’s possible that a large amount of Windows x64 binaries compiled with MinGW work almost entirely by accident, and maybe not even every time. Most symbols like this are function calls, so a significantly safer approach would be for MinGW to generate a thunk for those, and then just fail at linking when encountering a data import that isn’t decorated. Unfortunately, I don’t make the calls. :P I don’t mean to disparage any specific persons. I just generally recommend against MinGW even in the face of the fact that it is virtually the only free software option to compile Windows binaries.

105 points by goranmoomin 5y ago

iopq 5y ago

Rust on Windows used to run on Mingw, but has since moved on It made it a lot easier in many ways to just compile a thing and have it work without thinking about mingw and interfacing C from mingw

anonymfus 5y ago

Hmm... https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention https://docs.microsoft.com/en-us/cpp/build/x64-calling-conve... > Return values > A scalar return value that can fit into 64 bits, including the __m64 type, is returned through RAX. Non-scalar types including floats, doubles, and vector types such as __m128, __m128i, __m128d are returned in XMM0. The state of unused bits in the value returned in RAX or XMM0 is undefined. > User-defined types can be returned by value from global functions and static member functions. To return a user-defined type by value in RAX, it must have a length of 1, 2, 4, 8, 16, 32, or 64 bits. It must also have no user-defined constructor, destructor, or copy assignment operator. It can have no private or protected non-static data members, and no non-static data members of reference type. It can't have base classes or virtual functions. And, it can only have data members that also meet these requirements. (This definition is essentially the same as a C++03 POD type. Because the definition has changed in the C++11 standard, we don't recommend using std::is_pod for this test.) Otherwise, the caller must allocate memory for the return value and pass a pointer to it as the first argument. The remaining arguments are then shifted one argument to the right. The same pointer must be returned by the callee in RAX. Isn't this what https://github.com/ziglang/zig/issues/9998#issuecomment-997088467 https://github.com/ziglang/zig/issues/9998#issuecomment-9970... claims to be undocumented?

anonymfus 5y ago
Following comments on Github also point this out.
Andoryuuta 5y ago
> Otherwise, the caller must allocate memory for the return value and pass a pointer to it as the first argument. The remaining arguments are then shifted one argument to the right. I think the issue is that it isnt _fully_ documented anywhere. The MS docs do partially describe it, but it omits details about how this should be handled in the case of C++ member functions (which are the COM methods in question). C++ member functions always receive the hidden 'this' pointer as the first argument in Microsoft's x64 calling convention, which directly conflicts with this document stating the returned struct would be a "pointer to it as the first argument". You can see this in some of the godbolt links in the linked issue. Ultimately, some COM functions that take no arguments actually end up taking two hidden parameters in rcx and rdx (this pointer, and the return struct pointer respectively)

bla3 5y ago

Windows uses its own calling convention. I'm sure COM win32 headers have no shortage or atrocities, but this post here is just saying that GCC doesn't implement the Windows calling convention for struct returns, while clang does.

jmull 5y ago

It’s interesting to see the process at work.

olvy0 5y ago

As someone who has to deal from time to time with code that is a mix of old COM components and newer mingw dependencies I found this discussion fascinating. I learned to avoid most issues by wrapping everything at the boundaries with a pure C api/abi, and not letting exceptions propagate at all, instead catching them and returning an error code. I wonder why zig can't do the same but I guess they want calls to directx etc to match what's documented instead of going through c like contortions.

chasil 5y ago

I know that Clang runs on Windows, as PuTTY's "about" dialog credits it. Does this offer a reasonable replacement for MinGW? https://wetmelon.github.io/clang-on-windows.html https://wetmelon.github.io/clang-on-windows.html

malkia 5y ago
There is CLANG environment support for MSYS2 - https://www.msys2.org/docs/environments/ https://www.msys2.org/docs/environments/

cyber_kinetist 5y ago

My last experience with MinGW-w64 was when I was trying to compile my C++ simulation code in Windows and finding out that AVX instructions were not working because the compiler had misalignment-related bugs. (The issue is still open in https://github.com/msys2/MSYS2-packages/issues/1209 https://github.com/msys2/MSYS2-packages/issues/1209) MinGW/MSYS certainly had appeal to former Linux devs who didn't want to touch the horrors of MSVC, but Zig (with its included Clang compiler/runtime) might end up being a better solution for people trying to compile C/C++ code on Windows in a stable manner.

colejohnson66 5y ago
Wow. “mingwandroid” is coming off as pretty rude: > Is it really so difficult to determine the correct place to report issues to?

jbk 5y ago

That’s the opposite of our experience on VLC and Firefox was sharing our opinion: mingw-w64 people are skilled, nice and very clever and think about all use cases. This is how we are able to support configurations that even MS does not support… Also, a contrario from what this threads seems to say, the Windows SDK headers are faaaar from being open source compatible

badsectoracula 5y ago
Yeah, i've been using MinGW (and now MinGW-w64) for practically decades now, originally via MSYS and later via MSYS2 and i never remember having and real issues with it - i even did some DirectX programming with it. The only thing i remember being an issue at some point (and perhaps still is) is that it relies on msvcrt.dll that theoretically isn't part of the Windows platform but in practice that DLL has been there since Windows 98 and the chances of this ever being a real problem are zero - so it is really an "issue" about discussions with hairsplitting pedants than a practical one :-P.
- slimsag 5y ago
  If you squint closely, you'll see my issue is not hairsplitting but rather a practical one about trouble with MinGW headers not being super up-to-date (missing DirectX 12 APIs, etc.) I'm a huge fan of MinGW-w64, though. I would bet most involved in the issue are. We all win by improving the status quo - I'm just a buffoon figuring out the best way to do that :)
  - jbk 5y ago
    DirectX 12 API is a pretty small case to throw all the project to trash. Notably since DirectX headers are always a different case compared to the rest of the Windows SDK
    - slimsag 5y ago
      I'm a massive fan of your work on VLC, but I feel that's a bit of a mischaracterization.. The author of the issue spoke about his troubles with MinGW in four bullet points at the top that are not related to DirectX at all, whether you agree or disagree with those the fact is most people (including Andrew Kelley) rebutted many of his arguments, and Andrew even reached out to MinGW devs on IRC to chat about the topic of contributing upstream. That seems a reasonable approach to me. I chimed in on the issue because DirectX 12 was the problem _I_ faced, explained what I learned, complained I still don't have a super official way to get up-to-date DirectX headers that are compatible with MinGW, and ultimately said I gave up on working on the issue out of frustration. Throughout the issue there are people suggesting throwing out all of MinGW would be reckless, heck even I concluded that would not be possible for multiple reasons in my write-up. I think people are reading too much into the title and trying to make this out to be some sort of attack on MinGW, I really do not think that was anyone's intent - we're just exploring if we can improve the status quo..

jchw 5y ago

I think the MinGW project is very useful, but I simply do not recommend using it. My experience with it was trying to debug why a friend’s build was working on Windows and not Wine. I figured it out: it’s because MinGW implements something called pseudo relocations, which implement symbol-relative relocations inside the entrypoint of your executable. It’s needed because when compiling, you don’t know if an unresolved symbol will be external or not, so it gets compiled as if it’s not. But when linking, and you finally know that the symbol is external, it’s too late to change the code to refer to an import table indirection. On ELF platforms, relocations can refer to symbols in other libraries during runtime linking. So external symbols can be treated very similarly to internal ones, just with linking done at runtime instead of compile-time. On Windows, you have to use `__declspec(dllimport)` or some such to inform the compiler that the symbol will be external. It can then generate code for referring to the IAT. MinGW wanted to support compiling UNIXy software that didn’t use this attribute, so it does not require it. Instead… the ELF-style pseudo-relocations are used. So at the entrypoint, your MinGW program or DLL loops through a table of pointers inside machine code, looks up their real address in the IAT, and performs fixups. It will even temporarily change the protection of the relevant pages to be writable. Check to see if your MinGW program has unexpected imports to kernel32.VirtualProtect! (IIRC this can also happen if you trigger executable stack by using GCC trampolines, another reason to not use GCC…) (edit: Please read the replies below, but according to MinGW developers, this only happens in some edge cases; ordinarily, thunks will be used, which do not have this problem. In addition, MinGW binaries will always link to VritualProtect, because the pseudo-reloc code exists whether or not you use them. I never noticed it before, so that's interesting.) There’s only one problem: this is extremely failure-prone. For example, if you are on AMD64, the typical CALL instruction contains a 32-bit RIP-relative offset. But in a 64-bit process space, two modules can happily be more than 2^32 bytes away in memory. I assume ELF platforms have a linker that’s aware of this, since the linker knows about these special relocations, and thus has to ensure the binaries are in range… but I don’t know much about ELF linking, so don’t take that as fact. So why does this break on Wine and not Windows? Well, as far as I can tell, it doesn’t break on Windows because for some reason ASLR never puts any of the modules too far away. I haven’t figured out why. Wine doesn’t implement ASLR and loads modules at their preferred addresses as long as it’s available, and those addresses wind up being too far away. The pseudo reloc code does not output any errors when the address overflows; would probably be worth a pull request, but I am still puzzling with why Windows doesn’t ever seem to cause the problem. Unless I am missing something crucial, it’s possible that a large amount of Windows x64 binaries compiled with MinGW work almost entirely by accident, and maybe not even every time. Most symbols like this are function calls, so a significantly safer approach would be for MinGW to generate a thunk for those, and then just fail at linking when encountering a data import that isn’t decorated. Unfortunately, I don’t make the calls. :P I don’t mean to disparage any specific persons. I just generally recommend against MinGW even in the face of the fact that it is virtually the only free software option to compile Windows binaries.