5 ms·

I ran into this issue when porting term.everything[0] from typescript to go. I had some c library dependencies that I did need to link, so I had to use cgo. My

by mmulet 10mo ago

I ran into this issue when porting term.everything[0] from typescript to go. I had some c library dependencies that I did need to link, so I had to use cgo. My solution was to do the build process on alpine linux[1] and use static linking[2]. This way it statically links musl libc, which is much friendlier with static linking than glibc. Now, I have a static binary that runs in alpine, Debian, and even bare containers. Since I have made the change, I have not had anyone open any issues saying they had problems running it on their machines. (Unlike when I was using AppImages, which caused much more trouble than I expected) [0] https://github.com/mmulet/term.everything https://github.com/mmulet/term.everything look at distribute.sh and the makefile to see how I did it. [1]in a podman or docker container [2] -ldflags '-extldflags "-static"'

cxr 10mo ago
I didn't see an explanation in the README that part of what the first GIF[1] shows is an effect created by video editing software (and not a screencapture that's just demonstrating the program actually running). "Screen images simulated" are the words usually chosen to start off the disclaimers in fine print shown at the bottom of the screen when similar effects appear in commercials. I think that it would make sense to adopt a similar explanation wrt the effect used for the GIF. 1. <https://github.com/mmulet/term.everything/blob/main/resources/graphics/warp_in_2.gif https://github.com/mmulet/term.everything/blob/main/resource...>
- plufz 10mo ago
  Why would an open source project need to have any disclaimer? They are not selling anything.
  - nofriend 10mo ago
    Because lying is wrong even when open source projects do it.
    - plufz 10mo ago
      I think it is a big stretch calling this visual effect lying. I don’t know if it is a cultural American thing or just difference in interpretation but I had no difficulty understanding that this was a visual effect. But in my country ads don’t come with disclaimers. Do you feel like these disclaimers are truly helpful?
    - cxr 10mo ago
      I don't feel that the person I responded to is lying or being intentionally deceptive.
- hmans 10mo ago
  [dead]
- nebezb 10mo ago
  > “in commercials where such effects appear” Good thing this isn’t a commercial then.
jchw 10mo ago
IMO this is the best approach, but it is worth noting that musl libc is not without its caveats. I'd say for most people it is best to tread carefully and make sure that differences between musl libc and glibc don't cause additional problems for the libraries you are linking to. There is a decent list of known functional differences on the musl libc wiki: https://wiki.musl-libc.org/functional-differences-from-glibc.html https://wiki.musl-libc.org/functional-differences-from-glibc... Overall, though, the vast majority of software works perfectly or near perfectly on musl libc, and that makes this a very compelling option indeed, especially since statically linking glibc is not supported and basically does not work. (And obviously, if you're already using library packages that are packaged for Alpine Linux in the first place, they will likely already have been tested on musl libc, and possibly even patched for better compatibility.)
apitman 10mo ago
Note that you don't have to compile on an Alpine system to achieve this. These instructions should work on most distros: https://www.arp242.net/static-go.html https://www.arp242.net/static-go.html
tasuki 10mo ago
Huh. Does term.everything just work, or are there some gotchas? This seems like it could be supremely useful!
- mmulet 10mo ago
  It works so far! No major gotchas that I know of yet. From the perspective of the apps, they are just talking to a normal Wayland compositor, so everything works as expected. Just try it for your workflow, and if you run into any problems just open an issue and I’ll fix it.
nly 10mo ago
> and even bare containers. Strange, i thought the whole point of containers was to solve this problem.
- jdub 10mo ago
  Depends how much you care about the size and security footprint of your container images.
  - nly 10mo ago
    Static linking doesn't solve security issues either.
    - jdub 10mo ago
      Static linking can be a layer of defence against some security issues, depending on your circumstances. But what I said was "reduced security footprint", considering the trade offs between a single statically linked binary and a full (or even cut down) Linux distribution.
- vrighter 10mo ago
  The whole point of containers is to ship almost the whole OS with the application (It is a technical implementation of the "works on my machine" concept). If the OS you put in your container (by just pulling in a prebuilt image from somewhere) doesn't have the necessary things, then the application would fail to work just the same as if you ran it on the bare operating system with the the same missing libraries.
nickcw 10mo ago
That is a nice approach. I'll have to give that a try with rclone. I tried lots of things in the past but not using Alpine which is a great idea Another alternative is https://github.com/ebitengine/purego https://github.com/ebitengine/purego You can use this to dynamic load shared objects / DLLs so in the OP example they could disable systemd support if the systemd shared object did not load. This technique is used in the cgofuse library ( https://github.com/winfsp/cgofuse https://github.com/winfsp/cgofuse ) rclone uses which means rclone can run even if you don't have libfuse/winfsp installed. However the rclone mount subcommand won't work. The purego lib generalizes this idea. I haven't got round to trying this yet but it looks very promising.
- kokada 10mo ago
  I am using purego indirectly in two pet projects of mine. While it has its own issues it definitely solves the issue of cross-compilation. In this particular case it may be that they will need to write a wrapper to abstract differences between the systemd C API if it is not stable, but at least they still can compile a binary from macOS to Linux without issues. The other issue as other said is to use journalctl and just parse the JSON format. Very likely that this would be way more stable, but not sure if it is performant enough.
johnisgood 10mo ago
I use `-ldflags '-extldflags "-static"` as well. From the .go file, you just do `// #cgo LDFLAGS: -L. -lfoo`. You definitely do not need Alpine Linux for this. I have done this on Arch Linux. I believe I did not even need musl libc for this, but I potentially could have used it. I did not think I was doing something revolutionary! In fact, let me show you a snippet of my build script: # Build the Go project with the static library if go build -o $PROG_NAME -ldflags '-extldflags "-static"'; then echo "Go project built with static library linkage" else echo "Error: Failed to build the Go project with static library" exit 1 fi # Check if the executable is statically linked if nm ./$PROG_NAME | grep -q "U "; then echo "Error: The generated executable is dynamically linked" exit 1 else echo "Successfully built and verified static executable '$PROG_NAME'" fi And like I said, the .go file in question has this: // #cgo LDFLAGS: -L. -lfoo It works perfectly, and should work on any Linux distribution.
- mmulet 10mo ago
  I use alpine for this [1] reason, but I will admit that this is a premature-optimization. I haven’t actually ran into the problem myself. —— Your code is great, I do basically the same thing (great minds think alike!). The only thing I want to add is that cgo supports pkg-config directly [2] via // #cgo pkg-config: $lib So you don’t have to pass in linker flags manually. It’s incredibly convenient. [1]https://stackoverflow.com/questions/57476533/why-is-statically-linking-glibc-discouraged https://stackoverflow.com/questions/57476533/why-is-statical... [2]https://github.com/mmulet/term.everything/blob/def8c93a3db25211be0a37ab8bb44567ac5b7218/framebuffertoansi/ChafaInfo.go#L3 https://github.com/mmulet/term.everything/blob/def8c93a3db25...
  - johnisgood 10mo ago
    Thanks! I did not use pkg-config because the compiled .c is the one I have written specifically for this Go program. :D I did it all on my own system as well (Void Linux).
imcritic 10mo ago
What troubles did you have with AppImages?
- mmulet 10mo ago
  List of troubles: [1]https://github.com/mmulet/term.everything/issues/28 https://github.com/mmulet/term.everything/issues/28 [2]https://github.com/mmulet/term.everything/issues/18 https://github.com/mmulet/term.everything/issues/18 (although this issue later gets sidetracked to a build issue) [3]https://github.com/mmulet/term.everything/issues/14 https://github.com/mmulet/term.everything/issues/14 [4]https://github.com/mmulet/term.everything/issues/7 https://github.com/mmulet/term.everything/issues/7
pansa2 10mo ago
> do the build process on alpine linux and […] statically link musl libc IIRC it used to be common to do builds on an old version of RHEL or CentOS and dynamically link an old version of glibc. Binaries would then work on newer systems because glibc is backwards compatible. Does anyone still use that approach?
- Xylakant 10mo ago
  If you need glibc for any kind of reason, that approach is still used. But that won’t save you if no glibc is available. And since the folks here want to produce a musl build anyways for alpine, the easier approach is to just go for musl all the way.

by mmulet 10mo ago

cxr 10mo ago

I didn't see an explanation in the README that part of what the first GIF[1] shows is an effect created by video editing software (and not a screencapture that's just demonstrating the program actually running). "Screen images simulated" are the words usually chosen to start off the disclaimers in fine print shown at the bottom of the screen when similar effects appear in commercials. I think that it would make sense to adopt a similar explanation wrt the effect used for the GIF. 1. <https://github.com/mmulet/term.everything/blob/main/resources/graphics/warp_in_2.gif https://github.com/mmulet/term.everything/blob/main/resource...>

plufz 10mo ago
Why would an open source project need to have any disclaimer? They are not selling anything.
- nofriend 10mo ago
  Because lying is wrong even when open source projects do it.
  - plufz 10mo ago
    I think it is a big stretch calling this visual effect lying. I don’t know if it is a cultural American thing or just difference in interpretation but I had no difficulty understanding that this was a visual effect. But in my country ads don’t come with disclaimers. Do you feel like these disclaimers are truly helpful?
  - cxr 10mo ago
    I don't feel that the person I responded to is lying or being intentionally deceptive.
hmans 10mo ago
[dead]
nebezb 10mo ago
> “in commercials where such effects appear” Good thing this isn’t a commercial then.

jchw 10mo ago

IMO this is the best approach, but it is worth noting that musl libc is not without its caveats. I'd say for most people it is best to tread carefully and make sure that differences between musl libc and glibc don't cause additional problems for the libraries you are linking to. There is a decent list of known functional differences on the musl libc wiki: https://wiki.musl-libc.org/functional-differences-from-glibc.html https://wiki.musl-libc.org/functional-differences-from-glibc... Overall, though, the vast majority of software works perfectly or near perfectly on musl libc, and that makes this a very compelling option indeed, especially since statically linking glibc is not supported and basically does not work. (And obviously, if you're already using library packages that are packaged for Alpine Linux in the first place, they will likely already have been tested on musl libc, and possibly even patched for better compatibility.)

apitman 10mo ago

Note that you don't have to compile on an Alpine system to achieve this. These instructions should work on most distros: https://www.arp242.net/static-go.html https://www.arp242.net/static-go.html

tasuki 10mo ago

Huh. Does term.everything just work, or are there some gotchas? This seems like it could be supremely useful!

mmulet 10mo ago
It works so far! No major gotchas that I know of yet. From the perspective of the apps, they are just talking to a normal Wayland compositor, so everything works as expected. Just try it for your workflow, and if you run into any problems just open an issue and I’ll fix it.

nly 10mo ago

> and even bare containers. Strange, i thought the whole point of containers was to solve this problem.

jdub 10mo ago
Depends how much you care about the size and security footprint of your container images.
- nly 10mo ago
  Static linking doesn't solve security issues either.
  - jdub 10mo ago
    Static linking can be a layer of defence against some security issues, depending on your circumstances. But what I said was "reduced security footprint", considering the trade offs between a single statically linked binary and a full (or even cut down) Linux distribution.
vrighter 10mo ago
The whole point of containers is to ship almost the whole OS with the application (It is a technical implementation of the "works on my machine" concept). If the OS you put in your container (by just pulling in a prebuilt image from somewhere) doesn't have the necessary things, then the application would fail to work just the same as if you ran it on the bare operating system with the the same missing libraries.

nickcw 10mo ago

That is a nice approach. I'll have to give that a try with rclone. I tried lots of things in the past but not using Alpine which is a great idea Another alternative is https://github.com/ebitengine/purego https://github.com/ebitengine/purego You can use this to dynamic load shared objects / DLLs so in the OP example they could disable systemd support if the systemd shared object did not load. This technique is used in the cgofuse library ( https://github.com/winfsp/cgofuse https://github.com/winfsp/cgofuse ) rclone uses which means rclone can run even if you don't have libfuse/winfsp installed. However the rclone mount subcommand won't work. The purego lib generalizes this idea. I haven't got round to trying this yet but it looks very promising.

kokada 10mo ago
I am using purego indirectly in two pet projects of mine. While it has its own issues it definitely solves the issue of cross-compilation. In this particular case it may be that they will need to write a wrapper to abstract differences between the systemd C API if it is not stable, but at least they still can compile a binary from macOS to Linux without issues. The other issue as other said is to use journalctl and just parse the JSON format. Very likely that this would be way more stable, but not sure if it is performant enough.

johnisgood 10mo ago

I use `-ldflags '-extldflags "-static"` as well. From the .go file, you just do `// #cgo LDFLAGS: -L. -lfoo`. You definitely do not need Alpine Linux for this. I have done this on Arch Linux. I believe I did not even need musl libc for this, but I potentially could have used it. I did not think I was doing something revolutionary! In fact, let me show you a snippet of my build script: # Build the Go project with the static library if go build -o $PROG_NAME -ldflags '-extldflags "-static"'; then echo "Go project built with static library linkage" else echo "Error: Failed to build the Go project with static library" exit 1 fi # Check if the executable is statically linked if nm ./$PROG_NAME | grep -q "U "; then echo "Error: The generated executable is dynamically linked" exit 1 else echo "Successfully built and verified static executable '$PROG_NAME'" fi And like I said, the .go file in question has this: // #cgo LDFLAGS: -L. -lfoo It works perfectly, and should work on any Linux distribution.

mmulet 10mo ago
I use alpine for this [1] reason, but I will admit that this is a premature-optimization. I haven’t actually ran into the problem myself. —— Your code is great, I do basically the same thing (great minds think alike!). The only thing I want to add is that cgo supports pkg-config directly [2] via // #cgo pkg-config: $lib So you don’t have to pass in linker flags manually. It’s incredibly convenient. [1]https://stackoverflow.com/questions/57476533/why-is-statically-linking-glibc-discouraged https://stackoverflow.com/questions/57476533/why-is-statical... [2]https://github.com/mmulet/term.everything/blob/def8c93a3db25211be0a37ab8bb44567ac5b7218/framebuffertoansi/ChafaInfo.go#L3 https://github.com/mmulet/term.everything/blob/def8c93a3db25...
- johnisgood 10mo ago
  Thanks! I did not use pkg-config because the compiled .c is the one I have written specifically for this Go program. :D I did it all on my own system as well (Void Linux).

imcritic 10mo ago

What troubles did you have with AppImages?

mmulet 10mo ago
List of troubles: [1]https://github.com/mmulet/term.everything/issues/28 https://github.com/mmulet/term.everything/issues/28 [2]https://github.com/mmulet/term.everything/issues/18 https://github.com/mmulet/term.everything/issues/18 (although this issue later gets sidetracked to a build issue) [3]https://github.com/mmulet/term.everything/issues/14 https://github.com/mmulet/term.everything/issues/14 [4]https://github.com/mmulet/term.everything/issues/7 https://github.com/mmulet/term.everything/issues/7

pansa2 10mo ago

> do the build process on alpine linux and […] statically link musl libc IIRC it used to be common to do builds on an old version of RHEL or CentOS and dynamically link an old version of glibc. Binaries would then work on newer systems because glibc is backwards compatible. Does anyone still use that approach?

Xylakant 10mo ago
If you need glibc for any kind of reason, that approach is still used. But that won’t save you if no glibc is available. And since the folks here want to produce a musl build anyways for alpine, the easier approach is to just go for musl all the way.