4 ms·

> these executables will be easily redistributable and work on any distro While this sounds amazing and I'm sure is very useful in various situations, I think

by ris 9y ago

> these executables will be easily redistributable and work on any distro While this sounds amazing and I'm sure is very useful in various situations, I think encouraging a culture of distributing software in such a fashion is an incredibly bad idea. This is a big issue I have with the Go culture. Downloading a statically linked binary from some site and just sticking it somewhere is extremely bad practise to encourage. The user will never get prompted to update it, vulnerabilities and bugs in the vendored libraries will never be addressed... the list of issues is endless. Distributions are not really about distribution, they're about maintenance.

heavenlyblue 9y ago
It reminds me of why the protobuf compiler had never received any functionality regarding the generation of python code with the redefined library roots [1]: Google has a monorepo, therefore they never needed this. It is especially amusing when it is later being explained [2]: > Overall, your directory structure appears to be more complicated > than what we generally do at Google (which is the environment where > all this behavior was designed/evolved). At Google we generally have > a single directory structure for all source files, including protos. Well, yes. You do... At Google. [1] https://github.com/google/protobuf/issues/881 https://github.com/google/protobuf/issues/881 [2] https://github.com/google/protobuf/issues/1491 https://github.com/google/protobuf/issues/1491
- haberman 9y ago
  Hi there, those are my words you quoted above. I just wanted to emphasize that we would help open-source users out on this if we could. This isn't a case of "we didn't need it, so we never implemented it." The problem is that no one has proposed a solution that is compatible with our need to do parallel compilation inside Google. The only proposal I saw involved removing the 1:1 file correspondence between .proto input and _pb2.py output. I can understand why people would want this. But inside Google, we compile all our .proto files in parallel, and our whole build system depends on being able to parallelize things in this way. If you have a solution to this problem that doesn't break any of our existing use cases, we're all ears.
curun1r 9y ago
There is cargo-deb [0] which is, IMHO, a far superior way to distribute Rust software. I've been hoping for something similar for Mac/PC distribution (.pkg and .msi...stuff like homebrew is not a substitute for proper packages). I've been tempted to work on it myself, but haven't found the time. It's not particularly hard to do if you're only scripting the various platform-specific commands, but making something that will output artifacts for all platforms is much harder. [0] https://github.com/mmstick/cargo-deb https://github.com/mmstick/cargo-deb
- voltagex_ 9y ago
  I looked into the same kind of thing for Python years ago - it's hard, and not many things get it right (even the new .NET bits get it really wrong). Please: single exe/msi, one click to install/run.
  - icebraining 9y ago
    I've recently built an installer using nsis (+pynsist), it seemed to work fine, but now you're making me worry about what I might have missed.
    - voltagex_ 9y ago
      I don't think pynsis existed when I was investigating this so you're probably fine.
  - iknowstuff 9y ago
    Windows 10's .appx is what you're looking for. https://www.howtogeek.com/285410/how-to-install-.appx-or-.appxbundle-software-on-windows-10/ https://www.howtogeek.com/285410/how-to-install-.appx-or-.ap... I think the old Windows model of installers polluting the registry and disk drive in random places was a very bad idea and it's time to move on from .exe and .msi.
hsivonen 9y ago
Yet, people find static Go and Rust binaries a useful feature, so clearly there is some notable problem with the Linux ecosystem that static linking addresses. In particular there's tension between running enterprise distros on servers and getting stuff deployed when the system libs are very old.
- ilammy 9y ago
  It's not even about being old, but having different versions. Take libcurl and OpenSSL for example. Some distros use OpenSSL 1.0.2, some default to OpenSSL 1.1.0, in some cases libcurl depends on 1.0.2, but the system in general prefers OpenSSL 1.1.0. Which one should you use in your application for crypto? Obviously, the sane decision is to provide a separate build for each distro which uses the libraries bundled with the distro. Or you can just link everything statically (and then security updates of OpenSSL become your problem). Then also come QA people and management, which want to support multiple distros, but do not want to support multiple packages (as they are different and require separate QA effort), plus the users would find it hard to install an appropriate package for their distro. So... it's more convenient to just link statically.
- zaarn 9y ago
  Static linking used to be how everyone deployed software. The argument for dynamic linking was mainly "we can save a lot of diskspace and memory" (which it didn't when you account for how UNIX worked back then) Of course nowadays you also use it to quickly patch security holes in applications without having to relink them. IMO the proper way would be to simply relink the binary with patched libraries as default and only use dynamic linking where it's necessary. Statically linked binaries work far better in my experience, I can run (some) go programs on ancient 2.3 linux kernels (last i tested atleast) without complaint but any modern Firefox will likely refuse since the libraries in that specific distro are ancient af.
  - pjmlp 9y ago
    I find it ironic that young generations only used to GNU/Linux glibc dynamic linking, find static linking something out of the ordinary. Like you say, it was how everyone deployed software.
Goopplesoft 9y ago
> I think encouraging a culture of distributing software in such a fashion is an incredibly bad idea It isn't obvious to me how that is dramatically different from traditional package managers as far as the update process goes. Upgrading decencies seems like a generally active and involved process as far as most package managers I've seen go. pip/apt/npm/go-get/glide/yum/nixos etc all require you to actively discover and upgrade your dependencies and I've never been prompted to upgrade a package from any of the programs (a few do ask if you actively engage special subcommands on CLI e.g apt list --upgrade). Unattended-upgrades might be close but you can really only enable that on security releases and most package managers don't have the resources to setup special distros (and fewer backport security fixes). So is the pain really just not having a quick upgrade cli and is that dramatically different from going to a github page and getting the new URL for a new binary? Would something as simple as writing a script to list and download versions of binaries from github releases make this a non issue?
- jhasse 9y ago
  Fedora Workstation asks you to automatically install updates on shutdown/reboot. Also: an update prompt which you get on Ubuntu, is still better than nothing.
- timlyo 9y ago
  Nixos can automatically upgrade. https://nixos.org/nixos/manual/#idm140737316798064 https://nixos.org/nixos/manual/#idm140737316798064
- sambe 9y ago
  They don't require you to discover - they all have a clear and easy option to go and find updates automatically. Most of the system-level ones also have various schedulers & notifiers (both CLI and GUI) which are normally on by default for common distributions.
- rekado 9y ago
  Debian has watch files to discover updates. Guix has updater programmes for various kinds of origins to discover updates.

by ris 9y ago

heavenlyblue 9y ago

It reminds me of why the protobuf compiler had never received any functionality regarding the generation of python code with the redefined library roots [1]: Google has a monorepo, therefore they never needed this. It is especially amusing when it is later being explained [2]: > Overall, your directory structure appears to be more complicated > than what we generally do at Google (which is the environment where > all this behavior was designed/evolved). At Google we generally have > a single directory structure for all source files, including protos. Well, yes. You do... At Google. [1] https://github.com/google/protobuf/issues/881 https://github.com/google/protobuf/issues/881 [2] https://github.com/google/protobuf/issues/1491 https://github.com/google/protobuf/issues/1491

haberman 9y ago
Hi there, those are my words you quoted above. I just wanted to emphasize that we would help open-source users out on this if we could. This isn't a case of "we didn't need it, so we never implemented it." The problem is that no one has proposed a solution that is compatible with our need to do parallel compilation inside Google. The only proposal I saw involved removing the 1:1 file correspondence between .proto input and _pb2.py output. I can understand why people would want this. But inside Google, we compile all our .proto files in parallel, and our whole build system depends on being able to parallelize things in this way. If you have a solution to this problem that doesn't break any of our existing use cases, we're all ears.

curun1r 9y ago

There is cargo-deb [0] which is, IMHO, a far superior way to distribute Rust software. I've been hoping for something similar for Mac/PC distribution (.pkg and .msi...stuff like homebrew is not a substitute for proper packages). I've been tempted to work on it myself, but haven't found the time. It's not particularly hard to do if you're only scripting the various platform-specific commands, but making something that will output artifacts for all platforms is much harder. [0] https://github.com/mmstick/cargo-deb https://github.com/mmstick/cargo-deb

voltagex_ 9y ago
I looked into the same kind of thing for Python years ago - it's hard, and not many things get it right (even the new .NET bits get it really wrong). Please: single exe/msi, one click to install/run.
- icebraining 9y ago
  I've recently built an installer using nsis (+pynsist), it seemed to work fine, but now you're making me worry about what I might have missed.
  - voltagex_ 9y ago
    I don't think pynsis existed when I was investigating this so you're probably fine.
- iknowstuff 9y ago
  Windows 10's .appx is what you're looking for. https://www.howtogeek.com/285410/how-to-install-.appx-or-.appxbundle-software-on-windows-10/ https://www.howtogeek.com/285410/how-to-install-.appx-or-.ap... I think the old Windows model of installers polluting the registry and disk drive in random places was a very bad idea and it's time to move on from .exe and .msi.

hsivonen 9y ago

Yet, people find static Go and Rust binaries a useful feature, so clearly there is some notable problem with the Linux ecosystem that static linking addresses. In particular there's tension between running enterprise distros on servers and getting stuff deployed when the system libs are very old.

ilammy 9y ago
It's not even about being old, but having different versions. Take libcurl and OpenSSL for example. Some distros use OpenSSL 1.0.2, some default to OpenSSL 1.1.0, in some cases libcurl depends on 1.0.2, but the system in general prefers OpenSSL 1.1.0. Which one should you use in your application for crypto? Obviously, the sane decision is to provide a separate build for each distro which uses the libraries bundled with the distro. Or you can just link everything statically (and then security updates of OpenSSL become your problem). Then also come QA people and management, which want to support multiple distros, but do not want to support multiple packages (as they are different and require separate QA effort), plus the users would find it hard to install an appropriate package for their distro. So... it's more convenient to just link statically.
zaarn 9y ago
Static linking used to be how everyone deployed software. The argument for dynamic linking was mainly "we can save a lot of diskspace and memory" (which it didn't when you account for how UNIX worked back then) Of course nowadays you also use it to quickly patch security holes in applications without having to relink them. IMO the proper way would be to simply relink the binary with patched libraries as default and only use dynamic linking where it's necessary. Statically linked binaries work far better in my experience, I can run (some) go programs on ancient 2.3 linux kernels (last i tested atleast) without complaint but any modern Firefox will likely refuse since the libraries in that specific distro are ancient af.
- pjmlp 9y ago
  I find it ironic that young generations only used to GNU/Linux glibc dynamic linking, find static linking something out of the ordinary. Like you say, it was how everyone deployed software.

Goopplesoft 9y ago

> I think encouraging a culture of distributing software in such a fashion is an incredibly bad idea It isn't obvious to me how that is dramatically different from traditional package managers as far as the update process goes. Upgrading decencies seems like a generally active and involved process as far as most package managers I've seen go. pip/apt/npm/go-get/glide/yum/nixos etc all require you to actively discover and upgrade your dependencies and I've never been prompted to upgrade a package from any of the programs (a few do ask if you actively engage special subcommands on CLI e.g apt list --upgrade). Unattended-upgrades might be close but you can really only enable that on security releases and most package managers don't have the resources to setup special distros (and fewer backport security fixes). So is the pain really just not having a quick upgrade cli and is that dramatically different from going to a github page and getting the new URL for a new binary? Would something as simple as writing a script to list and download versions of binaries from github releases make this a non issue?

jhasse 9y ago
Fedora Workstation asks you to automatically install updates on shutdown/reboot. Also: an update prompt which you get on Ubuntu, is still better than nothing.
timlyo 9y ago
Nixos can automatically upgrade. https://nixos.org/nixos/manual/#idm140737316798064 https://nixos.org/nixos/manual/#idm140737316798064
sambe 9y ago
They don't require you to discover - they all have a clear and easy option to go and find updates automatically. Most of the system-level ones also have various schedulers & notifiers (both CLI and GUI) which are normally on by default for common distributions.
rekado 9y ago
Debian has watch files to discover updates. Guix has updater programmes for various kinds of origins to discover updates.