4 ms·

Notice how Windows, FreeBSD, NetBSD, and (removed in 6.0) OpenBSD all have Linux emulation/compat layers. This is in fact incredibly sad, for multiple reasons:

Notice how Windows, FreeBSD, NetBSD, and (removed in 6.0) OpenBSD all have Linux emulation/compat layers. This is in fact incredibly sad, for multiple reasons: - This is what POSIX was meant to address; there aren't all that many Linux-specific APIs that ordinary applications actually need. Proof: browse the ports/packages on any BSD; all the apps (with extremely few notable omissions) are there and working fine. - Vocal FOSS enthusiasts are ready, every day of the week, to preach to you about open standards; you must use Jitsi and not Zoom, Matrix and not WhatsApp, heck I've been told to play Mindustry instead of Factorio. I don't know if these are the same people who write "#ifdef linux" in their code, but I've sent out enough one-line patches to fix the build on OpenBSD to wonder. - The Linux userland is one hell of a horror, in terms of being a viable target for others to aim at; try running literally any precompiled binary on Alpine or Void(musl) to get a feel. The glibc seems to be introducing new versioned symbols and other backward incompatible changes every release - just for the sake of it; and many open source libraries resist static linking with every inch of their autohell existence. (I've tried making a static build of Love2d for Linux, and it was an exercise in futility.) Yeah guys, unless you're writing a container runtime, please stop targeting Linux. You don't need to strictly adhere to POSIX, just please at least try to compile your program in a VM with any BSD; it will flush out 99% of the non-portable stuff.

zlg_codes 3y ago
This is relevant to my interests. What roadblocks did you run into when you attempted to build statically? I assume your goal was to have a single binary of Love2D so you can swap it around and not worry about glibc version? (As God intended imo; not religious but games should be one contained thing imo) Am curious to see what it does to avoid a static build.
- rollcat 3y ago
  The love2d static build story was a tangent to my 2016 challenge to ship one game every month of the year. The tangent ended up delaying the March game indefinitely, and thus ending the challenge... My memory is a bit hazy, so apologies for the scarce/imprecise details. I really, really wanted to ship binary builds for the three major platforms: Windows, Mac, and Linux; so that my friends and strangers could actually play the games, without dicking around downloading some framework. Love2d uses the relatively well-known hack of opening "argv[0]" as a ZIP file (the ZIP header starts at the end of a file, so this usually[0] just works). Creating and testing the builds for Windows was trivial - even though I didn't even have Windows on any of my computers; I borrowed a friend's laptop to verify that "cat love.exe game.love > game.exe" just works, and indeed it did. They got some scary warnings about binaries downloaded from the Internet, but the game ran well. The Mac needed a bit more fiddling, and I didn't have a Mac back then to sign or test the build. But I followed the instructions and someone reported success (modulo scary warnings). Woohoo. On to Linux... I already knew it was going to be the most "fun", despite being the only platform among these three that didn't do code signing or scary warnings. I didn't even realize at the time, how much of a clusterfuck glibc actually is; my primary motivation was that Love2d kept breaking their Lua APIs, and I've already found that lots of older (and even recent) Love2d games simply couldn't cope. My game had to be bound to a specific version of Love2d, and many distributions shipped different versions, so the only reasonable path forward was to bundle, just like on Windows/Mac. I started by downloading the Love2d sources, verifying that I can make a standard/dynamic build, and then "cat love game.love > game && chmod +x game && ./game". Indeed that was easy, but "ldd game" revealed several dozen shared libraries for things like PNG, Vorbis, etc, etc. I've looked at the .so numbers, and realized Love2d breaking is gonna be the lesser of my worries - judging by how high these numbers were, I was signing up for DLL hell. I didn't even want PNG or Vorbis - all of my graphics were 100% procedural, and I was yet to try adding sound to any game. So I've disabled most options, and this is where the easy part ended. I don't recall where exactly I gave up... I managed to find & download the sources for a whole bunch of these libraries, make the ".a" archives for static builds, and so on... I think at some point I've ran into Mesa (Love2d actually requires GPU acceleration) and decided that this is enough insanity. I still firmly believe in static linking on Linux! I only changed my approach: just use Go (with CGO_ENABLED=0). Unfortunately, Go is not without its own share of problems[0]; while XGB allows cgo-less X11, Mesa remains elusive. [0]: https://flak.tedunangst.com/post/the-three-line-single-binary-compiler-free-blog https://flak.tedunangst.com/post/the-three-line-single-binar...
  - zlg_codes 3y ago
    Thanks for sharing this journey! I've been considering Love2D for a while so it's important to read about the challenges others run into. You should definitely not have to package Mesa to ship a game! I wonder why it hooks so far deeply.. Coming from PICO-8 myself, I'm wondering if I should just skip the frameworks to babystep and just jump into C+SDL, or pygame or something. I like the different ways that games can be made -- like your procedural graphics not needing a graphics library.
    - rollcat 3y ago
      The "can I link Mesa statically" journey was futile, I had much less understanding how things worked back then (not that I'm any kind of an expert now, but you shouldn't be one to ship a f.in game). See, a pretty big chunk of the graphics driver stack on Linux sits in the .so's provided by Mesa: the i915_dri.so, r600_dri.so, nouveau_dri.so, libvulkan_intel.so, libvulkan_radeon.so, etc are ALL exactly what it says on the package. It's not just 50+MB of x86-64 code, it's also your peace of mind that whenever AMD/Intel/NVidia/Apple/RPi/etc drop a new GPU model, or whenever the kernel changes things on their end, you don't need to relink and re-release. I'm torn between "all of this stuff belongs in the kernel goddammit" and "these guys probably know better". OpenBSD and macOS actually force all syscalls to go through a dynamically linked libc, so perhaps it's the latter. Libraries/frameworks/engines such as PICO-8, Love2d, SDL, Allegro, Pygame, Godot, etc exist precisely to abstract away these details; you're not meant to care for libGL.so.1, you're meant to care for fixing your physics engine's timestep and using cubic splines to interpolate animation. Love2d was not a mature choice in 2016, so don't take my horror story or my hubris as any indication of what it looks like today; do your own research and pick the tool for the job ;) TBH I would love to go back to making games, but recently got too absorbed by StarCraft 2 and shitposting.
      - zlg_codes 3y ago
        Part of my interest is also getting better at Lua, and it seems to excel in embedded contexts like making small 2D games. So maybe Love2D will still be good for that, and maybe the statically linking story is better now. My eventual goal is similar to yours -- I want to ship a game with a single binary per platform that players don't have to get confused about. Completing a game at all would be good enough, but I want to do it right!
bheadmaster 3y ago
POSIX only defines source-code level compatibility. However, all OSes are free to implement their system calls any way they want. In order to run binaries compiled for a specific OS, you need to emulate that OS's (i.e. kernel's) ABI. There's no way around that. The only way to achieve true binary compatibility would be to use a system-level virtual machine, ala Inferno [0] or PhantomOS [1]. [0] https://www.inferno-os.org/ https://www.inferno-os.org/ [1] http://phantomos.org/ http://phantomos.org/
- jay-barronville 3y ago
  I don’t think the point was about ABI compatibility across operating systems—that’d be unreasonable. It’s more so about using POSIX API’s as much as possible to enable portability rather than relying on Linux-specific API’s that make compiling the code an unnecessarily frustrating experience. Sometimes, using Linux-specific API’s is inevitable, but in the overwhelming majority of cases, POSIX API’s work just fine and would make the code instantly more portable.
  - bheadmaster 3y ago
    Even on the source level, there are certain limitations to POSIX, e.g. linux uses epoll() which is more performant than poll(), but isn't defined by POSIX. So application writes sometimes have to choose between making code more portable or making it more performant.
    - woooooo 3y ago
      Anyone know why we can't get a new POSIX with some epoll/kqueue equivalent, maybe baking in some of the lessons learned from epoll's interface? It's insane in 2023 that people are typing things like EPOLL_ONESHOT and it's actually the best option.
      - pjmlp 3y ago
        Linux wars replaced UNIX wars, and POSIX moves slow.
Borg3 3y ago
Well said, I hate all that autoconf hell really. Most of my programs can be compiled pretty much anywhere (or easy ported). Not that I write big complicated stuff, but its doable. One thing is unavoidable tho, but its ok, because its special platform: #ifdef __CYGWIN__
jay-barronville 3y ago
Nice rant. Upvoted.
AshamedCaptain 3y ago
I agree with the rant and think that Linux today is overall doing a net negative effect to computing. The absolutely lack of stable API/ABI efforts and the "cavalier" attitude regarding free software are creating terrible precedents over the industry. Linux is today the kernel/operating system with the largest hardware support, but my ability to actually use the operating system I want with the hardware I want has moved nilch. The next free operating system which even supports my AMD GPU? It's FreeBSD, and they do that by linking in the code from Linux (not even forking: forking would be way too much effort). There is _no_ other OS which supports it. Also, you can forget about DRM support anywhere other than Linux. Netflix? Linux-only. It's a very sad picture and much definitely worse than 10-20 years ago.

zlg_codes 3y ago

This is relevant to my interests. What roadblocks did you run into when you attempted to build statically? I assume your goal was to have a single binary of Love2D so you can swap it around and not worry about glibc version? (As God intended imo; not religious but games should be one contained thing imo) Am curious to see what it does to avoid a static build.

rollcat 3y ago
The love2d static build story was a tangent to my 2016 challenge to ship one game every month of the year. The tangent ended up delaying the March game indefinitely, and thus ending the challenge... My memory is a bit hazy, so apologies for the scarce/imprecise details. I really, really wanted to ship binary builds for the three major platforms: Windows, Mac, and Linux; so that my friends and strangers could actually play the games, without dicking around downloading some framework. Love2d uses the relatively well-known hack of opening "argv[0]" as a ZIP file (the ZIP header starts at the end of a file, so this usually[0] just works). Creating and testing the builds for Windows was trivial - even though I didn't even have Windows on any of my computers; I borrowed a friend's laptop to verify that "cat love.exe game.love > game.exe" just works, and indeed it did. They got some scary warnings about binaries downloaded from the Internet, but the game ran well. The Mac needed a bit more fiddling, and I didn't have a Mac back then to sign or test the build. But I followed the instructions and someone reported success (modulo scary warnings). Woohoo. On to Linux... I already knew it was going to be the most "fun", despite being the only platform among these three that didn't do code signing or scary warnings. I didn't even realize at the time, how much of a clusterfuck glibc actually is; my primary motivation was that Love2d kept breaking their Lua APIs, and I've already found that lots of older (and even recent) Love2d games simply couldn't cope. My game had to be bound to a specific version of Love2d, and many distributions shipped different versions, so the only reasonable path forward was to bundle, just like on Windows/Mac. I started by downloading the Love2d sources, verifying that I can make a standard/dynamic build, and then "cat love game.love > game && chmod +x game && ./game". Indeed that was easy, but "ldd game" revealed several dozen shared libraries for things like PNG, Vorbis, etc, etc. I've looked at the .so numbers, and realized Love2d breaking is gonna be the lesser of my worries - judging by how high these numbers were, I was signing up for DLL hell. I didn't even want PNG or Vorbis - all of my graphics were 100% procedural, and I was yet to try adding sound to any game. So I've disabled most options, and this is where the easy part ended. I don't recall where exactly I gave up... I managed to find & download the sources for a whole bunch of these libraries, make the ".a" archives for static builds, and so on... I think at some point I've ran into Mesa (Love2d actually requires GPU acceleration) and decided that this is enough insanity. I still firmly believe in static linking on Linux! I only changed my approach: just use Go (with CGO_ENABLED=0). Unfortunately, Go is not without its own share of problems[0]; while XGB allows cgo-less X11, Mesa remains elusive. [0]: https://flak.tedunangst.com/post/the-three-line-single-binary-compiler-free-blog https://flak.tedunangst.com/post/the-three-line-single-binar...
- zlg_codes 3y ago
  Thanks for sharing this journey! I've been considering Love2D for a while so it's important to read about the challenges others run into. You should definitely not have to package Mesa to ship a game! I wonder why it hooks so far deeply.. Coming from PICO-8 myself, I'm wondering if I should just skip the frameworks to babystep and just jump into C+SDL, or pygame or something. I like the different ways that games can be made -- like your procedural graphics not needing a graphics library.
  - rollcat 3y ago
    The "can I link Mesa statically" journey was futile, I had much less understanding how things worked back then (not that I'm any kind of an expert now, but you shouldn't be one to ship a f.in game). See, a pretty big chunk of the graphics driver stack on Linux sits in the .so's provided by Mesa: the i915_dri.so, r600_dri.so, nouveau_dri.so, libvulkan_intel.so, libvulkan_radeon.so, etc are ALL exactly what it says on the package. It's not just 50+MB of x86-64 code, it's also your peace of mind that whenever AMD/Intel/NVidia/Apple/RPi/etc drop a new GPU model, or whenever the kernel changes things on their end, you don't need to relink and re-release. I'm torn between "all of this stuff belongs in the kernel goddammit" and "these guys probably know better". OpenBSD and macOS actually force all syscalls to go through a dynamically linked libc, so perhaps it's the latter. Libraries/frameworks/engines such as PICO-8, Love2d, SDL, Allegro, Pygame, Godot, etc exist precisely to abstract away these details; you're not meant to care for libGL.so.1, you're meant to care for fixing your physics engine's timestep and using cubic splines to interpolate animation. Love2d was not a mature choice in 2016, so don't take my horror story or my hubris as any indication of what it looks like today; do your own research and pick the tool for the job ;) TBH I would love to go back to making games, but recently got too absorbed by StarCraft 2 and shitposting.
    - zlg_codes 3y ago
      Part of my interest is also getting better at Lua, and it seems to excel in embedded contexts like making small 2D games. So maybe Love2D will still be good for that, and maybe the statically linking story is better now. My eventual goal is similar to yours -- I want to ship a game with a single binary per platform that players don't have to get confused about. Completing a game at all would be good enough, but I want to do it right!

bheadmaster 3y ago

POSIX only defines source-code level compatibility. However, all OSes are free to implement their system calls any way they want. In order to run binaries compiled for a specific OS, you need to emulate that OS's (i.e. kernel's) ABI. There's no way around that. The only way to achieve true binary compatibility would be to use a system-level virtual machine, ala Inferno [0] or PhantomOS [1]. [0] https://www.inferno-os.org/ https://www.inferno-os.org/ [1] http://phantomos.org/ http://phantomos.org/

jay-barronville 3y ago
I don’t think the point was about ABI compatibility across operating systems—that’d be unreasonable. It’s more so about using POSIX API’s as much as possible to enable portability rather than relying on Linux-specific API’s that make compiling the code an unnecessarily frustrating experience. Sometimes, using Linux-specific API’s is inevitable, but in the overwhelming majority of cases, POSIX API’s work just fine and would make the code instantly more portable.
- bheadmaster 3y ago
  Even on the source level, there are certain limitations to POSIX, e.g. linux uses epoll() which is more performant than poll(), but isn't defined by POSIX. So application writes sometimes have to choose between making code more portable or making it more performant.
  - woooooo 3y ago
    Anyone know why we can't get a new POSIX with some epoll/kqueue equivalent, maybe baking in some of the lessons learned from epoll's interface? It's insane in 2023 that people are typing things like EPOLL_ONESHOT and it's actually the best option.
    - pjmlp 3y ago
      Linux wars replaced UNIX wars, and POSIX moves slow.

Borg3 3y ago

Well said, I hate all that autoconf hell really. Most of my programs can be compiled pretty much anywhere (or easy ported). Not that I write big complicated stuff, but its doable. One thing is unavoidable tho, but its ok, because its special platform: #ifdef __CYGWIN__

jay-barronville 3y ago

Nice rant. Upvoted.

AshamedCaptain 3y ago

I agree with the rant and think that Linux today is overall doing a net negative effect to computing. The absolutely lack of stable API/ABI efforts and the "cavalier" attitude regarding free software are creating terrible precedents over the industry. Linux is today the kernel/operating system with the largest hardware support, but my ability to actually use the operating system I want with the hardware I want has moved nilch. The next free operating system which even supports my AMD GPU? It's FreeBSD, and they do that by linking in the code from Linux (not even forking: forking would be way too much effort). There is _no_ other OS which supports it. Also, you can forget about DRM support anywhere other than Linux. Netflix? Linux-only. It's a very sad picture and much definitely worse than 10-20 years ago.