6 ms·

I wonder how many C projects prefer to avoid standard library, just invoking Linux syscalls directly. Much more fun to write software this way, IMO.

forrestthewoods 1y ago
You had me with “avoid C standard library” but lost me at “incoming Linux syscalls directly”. Windows support is a requirement, and no WSL2 doesn’t count. C standard library is pretty bad and it’d be great if not using it was a little easier and more common.
- pmc00 1y ago
  You can do this in Windows too, useful if you want tiny executables that use minimum resources. I wrote this little systemwide mute utility for Windows that way, annoying to be missing some parts of the CRT but not bad, code here: https://github.com/pablocastro/minimute https://github.com/pablocastro/minimute
  - gpm 1y ago
    I thought windows had an unstable syscall interface?
    - LegionMammal978 1y ago
      It looks like that project does link against the usual Windows DLLs, it just doesn't use a static or dynamic C runtime.
      - pmc00 1y ago
        Windows isn’t quite like Linux in that typically apps don’t make syscalls directly. Maybe you could say what’s in ntdll is the system call contract, but in practice you call the subsystem specific API, typically the Win32 API, which is huge compared to the Linux syscall list because it includes all sorts of things like UI, COM (!), etc. The project has some of the properties discussed above such as not having a typical main() (or winmain), because there’s no CRT to call it.
    - Dwedit 1y ago
      Pretty much yeah. You have your usual Win32 API functions found in libraries like Kernel32, User32, and GDI32, but since after Windows XP, those don't actually make system calls. The actual system calls are found in NTDLL and Win32U. Lots of functions you can import, and they're basically one instruction long. Just SYSENTER for the native version, or a switch back to 64-bit mode for a WOW64 DLL. The names of the function always begin with Nt, like NtCreateFile. There's a corresponding Kernel mode call that starts with Zw instead, so in Kernel mode you have ZwCreateFile. But the system call numbers used with SYSENTER are indeed reordered every time there's a major version change to Windows, so you just call into NTDLL or Win32U instead if you want to directly make a system call.
- antihero 1y ago
  > Windows support is a requirement Why, exactly?
- AnimalMuppet 1y ago
  > Windows support is a requirement... For what? There is some software for which Windows support is required. There are others for which it is not, and never will be. (And for an article about running ELF files on RiscV with a Linux OS, the "Windows support" complaint seems a bit odd...)
- throwawaysoxjje 1y ago
  A requirement from whom? To do what?
- rfl890 1y ago
  You can make CRT-free Win32 programs, read this guide[1] and you're all set. I've written a couple CLI utilities which are completely CRT-free and weigh just under a few kilobytes. [1]: https://nullprogram.com/blog/2023/02/15/ https://nullprogram.com/blog/2023/02/15/
  - forrestthewoods 1y ago
    Great post!
  - matheusmoreira 1y ago
    Almost freestanding. It still requires you to link against kernel32 and use the functions it provides. This is because issuing system calls directly to the Windows kernel is not supported. The kernel developers reserve the right to change things like system call numbers, so they can't be hardcoded into the application.
    - forrestthewoods 1y ago
      > Almost freestanding. It still requires you to link against kernel32 Nitpick: the phrase “link against kernel32” feels like a Linux-ism. If you’re only calling a few function you need to load kernel32.dll and call some functions in it. But that’s a slightly different operation than linking against it. At least how I’ve always used the term link. You’re not wrong in principle. But Linux and Windows do a lot of things differently wrt linking and loading libs. (I think Windows does it waaay better but ymmv)
      - matheusmoreira 1y ago
        Loading means creating a memory image of the library. Linking means resolving the symbols to addresses within that memory image. Loading a library and calling some functions from it is linking. The function pointer you receive is your link to the library function.
        forrestthewoods 1y ago
- WJW 1y ago
  Obviously only a requirement if you intend your software to run under windows. But if you don't, why bother. Not all software is intended to be distributed to users far and wide. Some of it is just for yourself, and some of it will only ever run on linux servers.
  - forrestthewoods 1y ago
    > some of it will only ever run on linux servers. I’ve spent quite a lot of time dealing with code that will ever run on Linux which did not in fact only ever run on Linux! Obviously for hobby projects anyone can do what they want. But adult projects should support Windows imho and consider Windows support from the start. Cross-platform is super easy unless you choose to make it hard.
    - deleted 1y ago
      [deleted]
    - WJW 1y ago
      I don't think we are talking about the same type of software? The type I was talking about will only ever run on Linux because it's a (HTTP-ish) server that will only ever run on Linux. Probably a server that is only ever run by a single company on a single CPU type. That company will have complete control of the OS stack, so if it says no Windows, then no Windows has to be supported.
      - forrestthewoods 1y ago
        cool
    - matheusmoreira 1y ago
      > But adult projects should support Windows imho and consider Windows support from the start. Hope whatever "adult" is working on the project this is getting paid handsomely. They'd certainly need to pay me big bucks to care about Windows support. In any case, Linux system call ABI is becoming a lingua franca of systems programming. BSDs have implemented Linux system calls. Windows has straight up included Linux in the system. It looks like simply targeting Linux can easily result in a binary that actually does run anywhere.
jjmarr 1y ago
Tons of driver code does this.
1718627440 1y ago
I generally try to stay portable, but file descriptors are just to nice, to not use them.
- Retr0id 1y ago
  File descriptors are part of the linux syscall API, not libc. Are you thinking of FILE?
  - 1718627440 1y ago
    I did mean file descriptors.
    - Retr0id 1y ago
      Then I'm confused by what you meant, because you can use fds with or without libc.
      - 1718627440 1y ago
        I don't want to bypass libc in general, because I care about portability, but fds are just a nice interface, so I still use them instead of FILE, which would be the portable choice. My calls are still subject to OS choices, that differ from the kernel, since I don't bypass libc.
  - ajross 1y ago
    The "syscall API" is part of libc too. The read syscall is a trap, you put arguments in the right registers and issue the correct instruction[1] to enter the kernel. That's not something that can be expressed in C. The read() function that your C code actually uses is a C function provided by the C library. [1] "svc 0" on ARM, "int 0x80" on i386, etc...
    - Retr0id 1y ago
      syscalls are an implementation detail of some libc impls on some platforms, but the C spec does not mention syscalls.
electroly 1y ago
Not exactly the same, but on Windows if you use entirely Win32 calls you can avoid linking any C runtime library. Win32 is below the C standard library on Windows and the C runtime is optional.
- okanat 1y ago
  This is one of the cornerstones that guarantee Windows can easily upgrade the C runtime and make performance and security upgrades. Win32 APIs have a different function calling ABI too. So only part of that gets "bloated" is Win32 API itself (which is spread across multiple DLLs and don't actually bloat RAM usage). Most of the time even those functions and structures are carefully designed to have some future-proofness but it is usual to see APIs like CreateFile, CreateFile2, CreateFile3. Internally the earlier versions are upgraded to call the latest version. So not so much bloating there either. When the C runtime and the OS system calls are combined into the single binary like POSIX, it creates the ABI hell we're in with the modern Unix-likes. Either the OSes have to regularly break the C ABI compatibility for the updates or we have to live with terrible implementations. GNU libc and Linux combo is particularly bad. On GNU/Linux (or any other current libc replacements), the dynamic loading is also provided by the C library. This makes "forever" binary file compatibility particularly tricky to achieve. Glibc broke certain games / Steam by removing some parts of their ELF implementation: https://sourceware.org/bugzilla/show_bug.cgi?id=32653 https://sourceware.org/bugzilla/show_bug.cgi?id=32653 . They backed due to huge backlash from the community. If "the year of Linux desktop" would ever happen, they need to either do an Android and change the definition of what a software package is, or split Glibc into 3 parts: syscalls, dynamic loader and the actual C library. PS: There is actually a catch to your " C runtime is optional." argument. Microsoft still intentionally holds back the ability of compiling native ABI Windows programs without Visual Studio. The structured exception handlers (equivalent of Windows for SIGILL, SIGBUS etc.. not for SIGINT or SIGTERM though) are populated by the object files from the C runtime libraries (called VCRuntime/VCStartup). So it is actually not possible to have official Windows binaries without MSVC or any other C runtime like Mingw-64 that provides those symbols. It looks like some developers in Microsoft wanted to open-source VCRuntime / VCStartup but it was ~vetoed~ not fully approved by some people: https://github.com/microsoft/STL/issues/4560#issuecomment-2348622212 https://github.com/microsoft/STL/issues/4560#issuecomment-23... , https://www.reddit.com/r/cpp/comments/1l8mqlv/is_msvc_ever_going_open_source/mxa2m8k/ https://www.reddit.com/r/cpp/comments/1l8mqlv/is_msvc_ever_g...
matheusmoreira 1y ago
I once wrote a liblinux project just for this!! It was indeed extremely fun. Details in my other comment: https://news.ycombinator.com/item?id=45709141 https://news.ycombinator.com/item?id=45709141 I abandoned it because Linux itself now has a rich set of nolibc headers. Now I'm working on a whole programming language based around this concept. A freestanding lisp interpreter targeting Linux directly with builtin system call support. The idea is to complete the interpreter and then write the standard library and Linux user space in lisp using the system calls. It's been an amazing journey. It's incredible how far one can take this.
codedokode 1y ago
I think using syscalls directly is a worse idea than loading shared libraries, and new kernel features, like ALSA (audio playback), DRM (graphics rendering) and other use libraries instead of documenting syscalls and ioctls. This is better because it allows intercepting and subverting the calls, adding support for features even if the kernel doesn't support it, makes it easier to port code to other OSes, support different architectures (32-bit code on 64-bit kernel), and allows changing kernel interface without breaking anything. So Windows-style approach with system libraries is better in every aspect.

forrestthewoods 1y ago

You had me with “avoid C standard library” but lost me at “incoming Linux syscalls directly”. Windows support is a requirement, and no WSL2 doesn’t count. C standard library is pretty bad and it’d be great if not using it was a little easier and more common.

pmc00 1y ago
You can do this in Windows too, useful if you want tiny executables that use minimum resources. I wrote this little systemwide mute utility for Windows that way, annoying to be missing some parts of the CRT but not bad, code here: https://github.com/pablocastro/minimute https://github.com/pablocastro/minimute
- gpm 1y ago
  I thought windows had an unstable syscall interface?
  - LegionMammal978 1y ago
    It looks like that project does link against the usual Windows DLLs, it just doesn't use a static or dynamic C runtime.
    - pmc00 1y ago
      Windows isn’t quite like Linux in that typically apps don’t make syscalls directly. Maybe you could say what’s in ntdll is the system call contract, but in practice you call the subsystem specific API, typically the Win32 API, which is huge compared to the Linux syscall list because it includes all sorts of things like UI, COM (!), etc. The project has some of the properties discussed above such as not having a typical main() (or winmain), because there’s no CRT to call it.
  - Dwedit 1y ago
    Pretty much yeah. You have your usual Win32 API functions found in libraries like Kernel32, User32, and GDI32, but since after Windows XP, those don't actually make system calls. The actual system calls are found in NTDLL and Win32U. Lots of functions you can import, and they're basically one instruction long. Just SYSENTER for the native version, or a switch back to 64-bit mode for a WOW64 DLL. The names of the function always begin with Nt, like NtCreateFile. There's a corresponding Kernel mode call that starts with Zw instead, so in Kernel mode you have ZwCreateFile. But the system call numbers used with SYSENTER are indeed reordered every time there's a major version change to Windows, so you just call into NTDLL or Win32U instead if you want to directly make a system call.
antihero 1y ago
> Windows support is a requirement Why, exactly?
AnimalMuppet 1y ago
> Windows support is a requirement... For what? There is some software for which Windows support is required. There are others for which it is not, and never will be. (And for an article about running ELF files on RiscV with a Linux OS, the "Windows support" complaint seems a bit odd...)
throwawaysoxjje 1y ago
A requirement from whom? To do what?
rfl890 1y ago
You can make CRT-free Win32 programs, read this guide[1] and you're all set. I've written a couple CLI utilities which are completely CRT-free and weigh just under a few kilobytes. [1]: https://nullprogram.com/blog/2023/02/15/ https://nullprogram.com/blog/2023/02/15/
- forrestthewoods 1y ago
  Great post!
- matheusmoreira 1y ago
  Almost freestanding. It still requires you to link against kernel32 and use the functions it provides. This is because issuing system calls directly to the Windows kernel is not supported. The kernel developers reserve the right to change things like system call numbers, so they can't be hardcoded into the application.
  - forrestthewoods 1y ago
    > Almost freestanding. It still requires you to link against kernel32 Nitpick: the phrase “link against kernel32” feels like a Linux-ism. If you’re only calling a few function you need to load kernel32.dll and call some functions in it. But that’s a slightly different operation than linking against it. At least how I’ve always used the term link. You’re not wrong in principle. But Linux and Windows do a lot of things differently wrt linking and loading libs. (I think Windows does it waaay better but ymmv)
    - matheusmoreira 1y ago
      Loading means creating a memory image of the library. Linking means resolving the symbols to addresses within that memory image. Loading a library and calling some functions from it is linking. The function pointer you receive is your link to the library function.
      - forrestthewoods 1y ago
WJW 1y ago
Obviously only a requirement if you intend your software to run under windows. But if you don't, why bother. Not all software is intended to be distributed to users far and wide. Some of it is just for yourself, and some of it will only ever run on linux servers.
- forrestthewoods 1y ago
  > some of it will only ever run on linux servers. I’ve spent quite a lot of time dealing with code that will ever run on Linux which did not in fact only ever run on Linux! Obviously for hobby projects anyone can do what they want. But adult projects should support Windows imho and consider Windows support from the start. Cross-platform is super easy unless you choose to make it hard.
  - deleted 1y ago
    [deleted]
  - WJW 1y ago
    I don't think we are talking about the same type of software? The type I was talking about will only ever run on Linux because it's a (HTTP-ish) server that will only ever run on Linux. Probably a server that is only ever run by a single company on a single CPU type. That company will have complete control of the OS stack, so if it says no Windows, then no Windows has to be supported.
    - forrestthewoods 1y ago
      cool
  - matheusmoreira 1y ago
    > But adult projects should support Windows imho and consider Windows support from the start. Hope whatever "adult" is working on the project this is getting paid handsomely. They'd certainly need to pay me big bucks to care about Windows support. In any case, Linux system call ABI is becoming a lingua franca of systems programming. BSDs have implemented Linux system calls. Windows has straight up included Linux in the system. It looks like simply targeting Linux can easily result in a binary that actually does run anywhere.

jjmarr 1y ago

Tons of driver code does this.

1718627440 1y ago

I generally try to stay portable, but file descriptors are just to nice, to not use them.

Retr0id 1y ago
File descriptors are part of the linux syscall API, not libc. Are you thinking of FILE?
- 1718627440 1y ago
  I did mean file descriptors.
  - Retr0id 1y ago
    Then I'm confused by what you meant, because you can use fds with or without libc.
    - 1718627440 1y ago
      I don't want to bypass libc in general, because I care about portability, but fds are just a nice interface, so I still use them instead of FILE, which would be the portable choice. My calls are still subject to OS choices, that differ from the kernel, since I don't bypass libc.
- ajross 1y ago
  The "syscall API" is part of libc too. The read syscall is a trap, you put arguments in the right registers and issue the correct instruction[1] to enter the kernel. That's not something that can be expressed in C. The read() function that your C code actually uses is a C function provided by the C library. [1] "svc 0" on ARM, "int 0x80" on i386, etc...
  - Retr0id 1y ago
    syscalls are an implementation detail of some libc impls on some platforms, but the C spec does not mention syscalls.

electroly 1y ago

Not exactly the same, but on Windows if you use entirely Win32 calls you can avoid linking any C runtime library. Win32 is below the C standard library on Windows and the C runtime is optional.

okanat 1y ago
This is one of the cornerstones that guarantee Windows can easily upgrade the C runtime and make performance and security upgrades. Win32 APIs have a different function calling ABI too. So only part of that gets "bloated" is Win32 API itself (which is spread across multiple DLLs and don't actually bloat RAM usage). Most of the time even those functions and structures are carefully designed to have some future-proofness but it is usual to see APIs like CreateFile, CreateFile2, CreateFile3. Internally the earlier versions are upgraded to call the latest version. So not so much bloating there either. When the C runtime and the OS system calls are combined into the single binary like POSIX, it creates the ABI hell we're in with the modern Unix-likes. Either the OSes have to regularly break the C ABI compatibility for the updates or we have to live with terrible implementations. GNU libc and Linux combo is particularly bad. On GNU/Linux (or any other current libc replacements), the dynamic loading is also provided by the C library. This makes "forever" binary file compatibility particularly tricky to achieve. Glibc broke certain games / Steam by removing some parts of their ELF implementation: https://sourceware.org/bugzilla/show_bug.cgi?id=32653 https://sourceware.org/bugzilla/show_bug.cgi?id=32653 . They backed due to huge backlash from the community. If "the year of Linux desktop" would ever happen, they need to either do an Android and change the definition of what a software package is, or split Glibc into 3 parts: syscalls, dynamic loader and the actual C library. PS: There is actually a catch to your " C runtime is optional." argument. Microsoft still intentionally holds back the ability of compiling native ABI Windows programs without Visual Studio. The structured exception handlers (equivalent of Windows for SIGILL, SIGBUS etc.. not for SIGINT or SIGTERM though) are populated by the object files from the C runtime libraries (called VCRuntime/VCStartup). So it is actually not possible to have official Windows binaries without MSVC or any other C runtime like Mingw-64 that provides those symbols. It looks like some developers in Microsoft wanted to open-source VCRuntime / VCStartup but it was ~vetoed~ not fully approved by some people: https://github.com/microsoft/STL/issues/4560#issuecomment-2348622212 https://github.com/microsoft/STL/issues/4560#issuecomment-23... , https://www.reddit.com/r/cpp/comments/1l8mqlv/is_msvc_ever_going_open_source/mxa2m8k/ https://www.reddit.com/r/cpp/comments/1l8mqlv/is_msvc_ever_g...

matheusmoreira 1y ago

I once wrote a liblinux project just for this!! It was indeed extremely fun. Details in my other comment: https://news.ycombinator.com/item?id=45709141 https://news.ycombinator.com/item?id=45709141 I abandoned it because Linux itself now has a rich set of nolibc headers. Now I'm working on a whole programming language based around this concept. A freestanding lisp interpreter targeting Linux directly with builtin system call support. The idea is to complete the interpreter and then write the standard library and Linux user space in lisp using the system calls. It's been an amazing journey. It's incredible how far one can take this.

codedokode 1y ago

I think using syscalls directly is a worse idea than loading shared libraries, and new kernel features, like ALSA (audio playback), DRM (graphics rendering) and other use libraries instead of documenting syscalls and ioctls. This is better because it allows intercepting and subverting the calls, adding support for features even if the kernel doesn't support it, makes it easier to port code to other OSes, support different architectures (32-bit code on 64-bit kernel), and allows changing kernel interface without breaking anything. So Windows-style approach with system libraries is better in every aspect.