5 ms·

Windows dynamic linking depends on the active code page

149 points by signa11 2y ago

enriquto 2y ago
This is a neat hack, and very well written. I felt a bit of nostalgia looking at the (classical) cpp flex to avoid repeating an identifier #define X(s) #s #define S(s) X(s)
- shawnz 2y ago
  For the unfamiliar, what does this pattern do exactly?
  - ahartmetz 2y ago
    It turns a macro argument into a string of that argument. Let's say you have a mydebug(expression) macro that should print both "expression" and the value of expression.
  - jjnoakes 2y ago
    The macro X turns its argument into a string, but using it on its own means you can't turn other macro values into strings, only macro names. #define F 42 #define X(s) #s X(5) // Expands to "5" (as a string literal) X(F) // Expands to "F" not "42" If you add one level of recursive macro expansion, though, that expands the macro argument as well as the macro definition: #define F 42 #define X(s) #s #define S(s) X(s) S(5) // "5" as expected S(F) // "42" because S(F) expands to X(42) expands to "42"
  - enriquto 2y ago
    In that case it's just to convert the value of a macro into a string constant cat > a.c #define X(s) #s #define S(s) X(s) char *f(void) return { S(A); } ^D cc -E a.c char *f(void) return { "A"; } cc -DA=B -E a.c char *f(void) return { "B"; } More interestingly, you can use this trick to create code where some user-specified word appears as a string and as the name of the function. Exercice: write a macro M(x) such that compiling the code M(foo) M(bar) results in void print_foo(void) { puts("foo"); } voir print_bar(void) { puts("bar"); }
    - delta_p_delta_x 2y ago
      > you can use this trick to create code where some user-specified word appears as a string and as the name of the function Ah, the poor man's introspection and reflection with C macros. shudder
      - wruza 2y ago
        Print an expression code to stdout using rich man’s introspection and reflection.
jiggawatts 2y ago
I got the chance to evaluate vendors for a huge enterprise because I was assisting their CTO. I vividly remember the sales guy who flew from Redmond to pitch the shiny new Hyper-V virtual machine platform Microsoft had just developed to compete head-to-head with VMware. “I tried the beta and it couldn’t install successfully if I set my regional options to en-AU.” “Umm… that’s just a cosmetic issue.” “It’s a hypervisor kernel, it is going to host tens of thousands of our most critical applications and it crashes if I change one of only three things it asks during setup. My confidence is not super high right now.” Etc… I got the impression that Microsoft is used to selling to PHBs based on the look of shock on the guy’s face when I told him that I not only installed the product, but benchmarked it too for good measure.
- n_plus_1_acc 2y ago
  MS still makes they same mistake. Date format in Teams is just broken. https://learn.microsoft.com/en-us/answers/questions/1403096/new-teams-date-time-format https://learn.microsoft.com/en-us/answers/questions/1403096/...
  - mnau 2y ago
    Didn't they fixed it? https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/commit/2d4c712f3d45c73eb5b1a8d1e06ee9b8b00280b8 https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness...
    - n_plus_1_acc 2y ago
      IIRC, they fixed most of the date displays, but not inside Planner and maybe the search results.
  - magicalhippo 2y ago
    I absolutely hate that they've reduced it to a single "regional settings". Just because I don't want Norwegian text everywhere does not mean I want dates and time to be displayed in some weird way. However I also utterly despise the Norwegian official way of writing decimal numbers with , rather than . as the decimal separator. We've had fine-grained control over this for ages, apps can handle it fine, just let us get it the way we want it.
userbinator 2y ago
As someone who has worked on Windows for a long time, the title was entirely unsurprising. Widening or narrowing always uses the current codepage. If a name contains values beyond ASCII — technically out of spec I'm not sure what spec it's referring to, but this is normal and expected for files in non-English systems. Such tools often incorrectly assume UTF-8, which is what motivated this article. Those tools are likely to be from the *nix world, where UTF-8 is far more common for the multibyte encoding --- but even there, you can have different codepages; and I have worked on Linux systems using CP1252 and 932 before.
- mananaysiempre 2y ago
  > I'm not sure what spec [the prohibition on non-ASCII names is] referring to, but this is normal and expected for files in non-English systems. The description of import directories[1,2] in the PE/COFF spec explicitly (if somewhat glibly) restricts imported DLLs to being referenced using ASCII only: > Name RVA - The address of an ASCII string that contains the name of the DLL. [1] https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#import-directory-table https://learn.microsoft.com/en-us/windows/win32/debug/pe-for... (current, unversioned) [2] https://github.com/tpn/pdfs/blob/master/Microsoft%20Portable%20Executable%20and%20Common%20Object%20File%20Format%20Specification%20-%201999%20(pecoff).pdf https://github.com/tpn/pdfs/blob/master/Microsoft%20Portable... §6.4.1 (version 6.0, 1999)
  - userbinator 2y ago
    That is, for lack of a better term, a "Microsoft-ism"; in MS documentation, "ASCII" consistently means "single-byte or MBCS" and is interpreted relative to the current codepage, as opposed to "Unicode" which means "UCS-2 or UTF-16". You can also see examples of "Unicode" in the docs you link to.
    - mananaysiempre 2y ago
      “ASCII”, not “ANSI”?
xhrpost 2y ago
Is this what is meant when people say that Windows doesn't have stable APIs? I've not programmed windows/desktop software and have often been confused as to why if you're writing an assembly program, you have to use at least a little C if you want to make Windows system calls[1]. The compiler must be compiling the call into "something" so can't you just write that "something" directly in assembly? [1]: The classic example being Chris Sawyer writing nearly all of Rollercoaster Tycoon in x86 assembly but requiring just enough C for the system calls.
- carom 2y ago
  Windows system call numbers aren't stable. It is done through their system libraries. So rather than invoking a syscall number as you do on Linux, if you want your code to be portable you import the DLL with that functionality and call the function. You can do this all from ASM without needing C, though the system libraries were likely written in a higher level language like C. You can also invoke system calls by number directly, it is just liable to not work on another version of Windows.
- xeeeeeeeeeeenu 2y ago
  Unlike Linux, Windows syscalls aren't documented and their IDs constantly change[1]. Instead, you're supposed to call wrapper functions provided by ntdll.dll. That said, most programs use even higher level functions from kernel32.dll and friends. You don't need C/C++ to call a function from a DLL, but it makes things easier, especially for more complex APIs like DirectX. [1] - https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/
- userbinator 2y ago
  You don't need to use C if you want to call Win32 functions, although the calling convention is based on C. The stable API is not the user-kernel boundary but instead a set of DLLs (kernel32, user32, gdi32, etc.), which makes sense if you look at the history: Windows started out as a GUI on top of DOS. The actual system call mechanism is very different between the DOS-based lineage and the NT-based ones, but the Win32 API remains nearly the same.
- dist-epoch 2y ago
otikik 2y ago
That has strong “let’s put this in as a temporary solution, we will fix it later” energy.
Dwedit 2y ago
The worst thing that Windows has that's tied to your locale is case sensitivity for filenames. You can have two filenames be created under one locale, change your locale, then the filenames aren't considered different anymore.
- Scharkenberg 2y ago
  Case sensitivity on Windows?

149 points by signa11 2y ago

This is a neat hack, and very well written. I felt a bit of nostalgia looking at the (classical) cpp flex to avoid repeating an identifier #define X(s) #s #define S(s) X(s)

shawnz 2y ago
For the unfamiliar, what does this pattern do exactly?
- ahartmetz 2y ago
  It turns a macro argument into a string of that argument. Let's say you have a mydebug(expression) macro that should print both "expression" and the value of expression.
- jjnoakes 2y ago
  The macro X turns its argument into a string, but using it on its own means you can't turn other macro values into strings, only macro names. #define F 42 #define X(s) #s X(5) // Expands to "5" (as a string literal) X(F) // Expands to "F" not "42" If you add one level of recursive macro expansion, though, that expands the macro argument as well as the macro definition: #define F 42 #define X(s) #s #define S(s) X(s) S(5) // "5" as expected S(F) // "42" because S(F) expands to X(42) expands to "42"
- enriquto 2y ago
  In that case it's just to convert the value of a macro into a string constant cat > a.c #define X(s) #s #define S(s) X(s) char *f(void) return { S(A); } ^D cc -E a.c char *f(void) return { "A"; } cc -DA=B -E a.c char *f(void) return { "B"; } More interestingly, you can use this trick to create code where some user-specified word appears as a string and as the name of the function. Exercice: write a macro M(x) such that compiling the code M(foo) M(bar) results in void print_foo(void) { puts("foo"); } voir print_bar(void) { puts("bar"); }
  - delta_p_delta_x 2y ago
    > you can use this trick to create code where some user-specified word appears as a string and as the name of the function Ah, the poor man's introspection and reflection with C macros. shudder
    - wruza 2y ago
      Print an expression code to stdout using rich man’s introspection and reflection.

jiggawatts 2y ago

I got the chance to evaluate vendors for a huge enterprise because I was assisting their CTO. I vividly remember the sales guy who flew from Redmond to pitch the shiny new Hyper-V virtual machine platform Microsoft had just developed to compete head-to-head with VMware. “I tried the beta and it couldn’t install successfully if I set my regional options to en-AU.” “Umm… that’s just a cosmetic issue.” “It’s a hypervisor kernel, it is going to host tens of thousands of our most critical applications and it crashes if I change one of only three things it asks during setup. My confidence is not super high right now.” Etc… I got the impression that Microsoft is used to selling to PHBs based on the look of shock on the guy’s face when I told him that I not only installed the product, but benchmarked it too for good measure.

n_plus_1_acc 2y ago
MS still makes they same mistake. Date format in Teams is just broken. https://learn.microsoft.com/en-us/answers/questions/1403096/new-teams-date-time-format https://learn.microsoft.com/en-us/answers/questions/1403096/...
- mnau 2y ago
  Didn't they fixed it? https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/commit/2d4c712f3d45c73eb5b1a8d1e06ee9b8b00280b8 https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness...
  - n_plus_1_acc 2y ago
    IIRC, they fixed most of the date displays, but not inside Planner and maybe the search results.
- magicalhippo 2y ago
  I absolutely hate that they've reduced it to a single "regional settings". Just because I don't want Norwegian text everywhere does not mean I want dates and time to be displayed in some weird way. However I also utterly despise the Norwegian official way of writing decimal numbers with , rather than . as the decimal separator. We've had fine-grained control over this for ages, apps can handle it fine, just let us get it the way we want it.

userbinator 2y ago

As someone who has worked on Windows for a long time, the title was entirely unsurprising. Widening or narrowing always uses the current codepage. If a name contains values beyond ASCII — technically out of spec I'm not sure what spec it's referring to, but this is normal and expected for files in non-English systems. Such tools often incorrectly assume UTF-8, which is what motivated this article. Those tools are likely to be from the *nix world, where UTF-8 is far more common for the multibyte encoding --- but even there, you can have different codepages; and I have worked on Linux systems using CP1252 and 932 before.

mananaysiempre 2y ago
> I'm not sure what spec [the prohibition on non-ASCII names is] referring to, but this is normal and expected for files in non-English systems. The description of import directories[1,2] in the PE/COFF spec explicitly (if somewhat glibly) restricts imported DLLs to being referenced using ASCII only: > Name RVA - The address of an ASCII string that contains the name of the DLL. [1] https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#import-directory-table https://learn.microsoft.com/en-us/windows/win32/debug/pe-for... (current, unversioned) [2] https://github.com/tpn/pdfs/blob/master/Microsoft%20Portable%20Executable%20and%20Common%20Object%20File%20Format%20Specification%20-%201999%20(pecoff).pdf https://github.com/tpn/pdfs/blob/master/Microsoft%20Portable... §6.4.1 (version 6.0, 1999)
- userbinator 2y ago
  That is, for lack of a better term, a "Microsoft-ism"; in MS documentation, "ASCII" consistently means "single-byte or MBCS" and is interpreted relative to the current codepage, as opposed to "Unicode" which means "UCS-2 or UTF-16". You can also see examples of "Unicode" in the docs you link to.
  - mananaysiempre 2y ago
    “ASCII”, not “ANSI”?

xhrpost 2y ago

Is this what is meant when people say that Windows doesn't have stable APIs? I've not programmed windows/desktop software and have often been confused as to why if you're writing an assembly program, you have to use at least a little C if you want to make Windows system calls[1]. The compiler must be compiling the call into "something" so can't you just write that "something" directly in assembly? [1]: The classic example being Chris Sawyer writing nearly all of Rollercoaster Tycoon in x86 assembly but requiring just enough C for the system calls.

carom 2y ago
Windows system call numbers aren't stable. It is done through their system libraries. So rather than invoking a syscall number as you do on Linux, if you want your code to be portable you import the DLL with that functionality and call the function. You can do this all from ASM without needing C, though the system libraries were likely written in a higher level language like C. You can also invoke system calls by number directly, it is just liable to not work on another version of Windows.
xeeeeeeeeeeenu 2y ago
Unlike Linux, Windows syscalls aren't documented and their IDs constantly change[1]. Instead, you're supposed to call wrapper functions provided by ntdll.dll. That said, most programs use even higher level functions from kernel32.dll and friends. You don't need C/C++ to call a function from a DLL, but it makes things easier, especially for more complex APIs like DirectX. [1] - https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/
userbinator 2y ago
You don't need to use C if you want to call Win32 functions, although the calling convention is based on C. The stable API is not the user-kernel boundary but instead a set of DLLs (kernel32, user32, gdi32, etc.), which makes sense if you look at the history: Windows started out as a GUI on top of DOS. The actual system call mechanism is very different between the DOS-based lineage and the NT-based ones, but the Win32 API remains nearly the same.
dist-epoch 2y ago

otikik 2y ago

That has strong “let’s put this in as a temporary solution, we will fix it later” energy.

Dwedit 2y ago

The worst thing that Windows has that's tied to your locale is case sensitivity for filenames. You can have two filenames be created under one locale, change your locale, then the filenames aren't considered different anymore.

Scharkenberg 2y ago
Case sensitivity on Windows?