6 ms·

Packaging software is something that takes work

66 points by pabs3 3y ago

userbinator 3y ago
Perhaps the title needs "on Linux" added to it. I have a collection of, and written quite a few myself, software which is nothing more than a single EXE. Most of them probably work on anything Win95 and newer. Their "installation" and "uninstallation" are trivial: Extract and run, delete once you've had enough. In other words, packaging software is something that takes work only because of what might often be unnecessary complexity and a correspondingly bureaucratic process that's grown around it, and not because it necessarily needs to. "Win32 is the stable Linux ABI" is more than just a meme.
- NoZebra120vClip 3y ago
  So, don't need any configuration or preferences, then? It must suck to set up from scratch every time. InstallShield was invented for a reason: the Win32 devs started dumping crap all over the system, all over the registry, and didn't bother cleaning up after themselves. A cottage industry of file and registry cleaners arose. Soon, software publishers began to learn how to properly package their software to be sustainable, and MS cooperated with Windows features to manage apps. Packaged apps.
  - mikewarot 3y ago
    It's up to the application to maintain any preferences you set in it. The installer package generally takes care of path configurations, etc. It's amazing how easy it's been to make Win32 installable programs since about 1990.
    - NoZebra120vClip 3y ago
      Uninstall has got to properly clean the registry, and filesystem, and should require manifests.
      - ponderings 3y ago
        I cant remember the last time I've uninstalled anything.
  - userbinator 3y ago
    It must suck to set up from scratch every time. What? If you haven't used the software before, what else other than defaults do you? InstallShield was invented for a reason: the Win32 devs started dumping crap all over the system, all over the registry, and didn't bother cleaning up after themselves That's a problem created by advocating for use of the registry, but not a necessity. The single-file apps I have may create a config file alongside the binary, but that's it.
    - matkoniecz 3y ago
      > What? If you haven't used the software before, what else other than defaults do you? On Windows it (by the time I left it, around 2013) it apparently meant bundling various unrelated stuff (like installing Chrome, toolbars, McAfee antivirus trial). Half of installers had minigame "how to spot and stop crapware bundling". Or at least asking blatantly obvious questions and lying that I read and accepted piles of legalese written by their lawyers. Not sure has it got better. Linux style packaging eliminated this for user, at least in my case.
      - TeMPOraL 3y ago
        > Linux style packaging eliminated this for user, at least in my case. That's in part because of the target audience on Linux was historically of the kind that didn't give a damn about this. Compare how analogous CLI package managers (chocolatey, winget) work on Windows: the first thing after identifying what to install is to query whether you accept relevant licensing terms.
        NoZebra120vClip 3y ago
        Yeah I don't know about that. In F/OSS there is traditionally no EULA to accept. There is a license to use/modify/share the software, and it may be a page or two long, and most licenses are found in just a few categories, such as MIT, BSD, Apache, GPL. Literally none of these licenses, in their original forms, place any restrictions or conditions on how an end-user can use the software; they're aimed at developers and contributors' use and distribution of the source and binaries. Also, I don't know how non-Ubuntu package management organizes software, but in Ubuntu, you need to make a conscious decision to include the "nonfree" type of software. Now presumably for a Linux package manager, any installation action should hopefully import any license terms to a known, readable place for the user to peruse, before launch, if necessary. Perhaps many Linux users don't care, as you say, but I know a lot more who are exquisitely sensitive to licensing and EULAs and will scrutinize this stuff a lot more than a Windows user who just gobbles shareware from C|NET.
- medellin 3y ago
  I would venture to say you haven’t packaged or shipped any complex software or had to do so in a way that scales to hundreds or thousands of machines. This is along the line of “i could build uber with 50 engineers”.
  - userbinator 3y ago
    Take a look at https://news.ycombinator.com/item?id=14748924 https://news.ycombinator.com/item?id=14748924 and you may find a whole new world you probably haven't seen before.
    - userbinator 3y ago
      You can try to keep on thinking that it doesn't exist, but the truth is there for you to see.
  - ponderings 3y ago
    ...wait, didn't we have taxi's before computers? Rickshaw, Japan, 1860?
    - mdwalters 3y ago
      The first (mechanical) computer was made in 1822, and the computer has origins dating back to the 17th century. https://en.wikipedia.org/wiki/Computer https://en.wikipedia.org/wiki/Computer
      - TeMPOraL 3y ago
        Then again, taxis date back to at least Ancient Rome (IIRC they found a taximeter device using pebbles to count wheel rotations). And arguably, knowing humans, as soon as we figured out how to reliably use an animal to move a person, someone started offering moving people in exchange for money...
      - ponderings 3y ago
        The Antikythera mechanism suggests quite a bit of previous art. Making a device for counting wheel rotations wouldn't be very difficult. You could do the billing on a wheel too! One set for each customer! But it would be easier to just do the billing by carving it into a clay tablet.
- matkoniecz 3y ago
  From user viewpoint: Starting to using Linux the greatest part was no longer dealing with insane install processes. For example, end of game where every Java update was pushing some unrelated software and you needed to be careful to not end with it. Also, "is it official site for this software or some copy of website or hijacked site" (and around time I left Windows SourceForge was busy with hijacking open source projects and adding malicious software to this downloads, maybe it got better since that time) If all software would be unpackable and require no configuration by default and leave no waste files over system it would be OK.
- cratermoon 3y ago
  More correctly, the title could end "unless you only ever target Win32 on x86". Shipping software that's at all portable across operating systems and architectures, that's a different story.
  - userbinator 3y ago
    In my experience, that sort of portability is overrated anyway. Win32 on x86 is going to be fine for a majority of users.
    - cratermoon 3y ago
      The majority of desktop users, perhaps. Most users are going to be customers of a system that likely runs “in the cloud" and that means, by far, Linux.
  - orwin 3y ago
    Nah, even for that. I don't know what gp is on. I recently downloaded a project a friend made on rpgmaker. I had to manually download rpgmaker itself, a dll and a new font. It wasn't a complex project, he isn't even a developer, he mostly make illustrations and graphical assets. It was really, really basic stuff. That said, I have almost 10 .exe that just work and that I take from computer to computer since 2015, like I have around 30 .sh/.py/.lisp that i do too. Being charitable, I think GP just have a lot of scripts/small tools that just work, and he was referring to that.
    - cratermoon 3y ago
      It works on my machine!
- giomasce 3y ago
  That doesn't really align with the number of redistributable installers and "DLL not found" errors I've had to deal with during my Windows life. My wife's Win 10 system always boots to a "some DLL not found" fatal message, but after clicking OK everything seems to work as intended. I can't say I trust the way that system is set up.
- viraptor 3y ago
  > I have a collection of, and written quite a few myself, software which is nothing more than a single EXE. Trivial software for power users (who can download, run and find the exe again) has trivial packaging. This is the case on any system. Your app has no entry in the start menu, no file type binding, no services, no uninstallation entry in the control panel, no large dependencies, etc.
- jmrm 3y ago
  That's why I love so much the AppImage project. Just download, double click, and you can execute whatever you want.
duped 3y ago
It takes work because the design of FHS is outdated and will eventually die off when we have working package managers that can deal with incoherency of their packages, and figure out a less heavy handed approach to redundancy. Containerization isn't really the answer but it uses the same tools that the answer will have: chroot jails, Linux namespaces, and bind mounts + overlays. The problem with containers is that the layer model doesn't lead to anything actually being shared, but the basic feature set of the user lane gives the tools to a possible solution that doesn't suck. Nix is close, but suffers from design issues orthogonal to how it works. Maybe Nixos is the future once end users never need to read a flake.
- johnny22 3y ago
  isn't that part about containerization but using shared things at least partially handled by flatpak? I still hope we end up closer to what does or guix does, but at least it's a step in the right direction for various packages.
WhyNotHugo 3y ago
> sometimes you get lucky and the packaging instructions are easy to write (Debian rules, RPM specfiles, whatever Arch uses, etc) Funny way to word this. Arch’s PKGBUILD syntax is very very simple. If you now some programming (especially some shell scripting), you should be able to figure out all the basics. Here’s a non-trivial example: https://gitlab.archlinux.org/archlinux/packaging/packages/todoman/-/blob/main/PKGBUILD https://gitlab.archlinux.org/archlinux/packaging/packages/to... Alpine’s APKBUILD are pretty similar too. Here’s the same package as above: https://git.alpinelinux.org/aports/tree/testing/todoman/APKBUILD https://git.alpinelinux.org/aports/tree/testing/todoman/APKB... Debian packages? Good luck trying to write one. Fedora’s specfiles are far from simple too… though you might be able to at least read one and get a vague idea of what’s going on. I suspect both of these two are the way they are due to historical reasons… it’s a shame that they’re unlikely to change. There’s so much simpler way to write packages.
- mike_hock 3y ago
  https://salsa.debian.org/python-team/packages/todoman/-/blob/master/debian/control https://salsa.debian.org/python-team/packages/todoman/-/blob... https://salsa.debian.org/python-team/packages/todoman/-/blob/master/debian/rules https://salsa.debian.org/python-team/packages/todoman/-/blob... https://salsa.debian.org/python-team/packages/todoman/-/blob/master/debian/todoman.install https://salsa.debian.org/python-team/packages/todoman/-/blob... If you concatenate them, you get about the same as the PKGBUILD and APKBUILD files. There's a couple more one-liners that I omitted. I fail to see how one is supposed to be easy and the other is hard.
  - mkj 3y ago
    The debian ones seem to have more bespoke languages to learn rather than shell script fragments, and more mystery dh-* things that need to be known.
    - mike_hock 3y ago
      That is true, but it's not "good luck making one." The "bespoke language" of debian/control is trivial, debian/rules is a makefile and the rest is mostly "one file glob per line" or that sort of thing.
samsquire 3y ago
I have an idea how to implement automatic packaging but requires operating system be changed. It would enable seamless uninstall. You tag every resource, file and API calls or change with an unfabricatable correlation identifier or cause that can be traced back to origin. I once used checkinstall to create debian packages. Creating a package is just a recording, no manual labour. You can follow the chain of causality to an executable. If you delete the executable, you are left with orphans which you might want to delete.
em-bee 3y ago
packaging is not much more work than building something from source. it's all the same process until the end step where you use the packaging tools to assemble the whole thing. it can get tedious when you end up having to package dependencies as well. but it's mainly grunt work, and none of the work is special to packaging. without packaging everyone would have to do the same work. my favorite packaging tool was conary. it has automated packaging to such an extent that in most cases all that is needed is to tell it the command to run the build and install processes for that application. it figures out everything else by itself. which files are documentation, what are libraries. what are dependencies. one thing it did was to split a package into components. there is a :doc component, a :mysql component and others, which were especially useful for optional dependencies. if you didn't use mysql you could leave out the :mysql component and install the package without a mysql dependency. where with deb or rpm i had to manually figure out what goes where if i wanted to split up a package, in conary that process was mostly automatic. and, to address another comment here, conary worked just fine with the FHS. i do not think that the FHS was or is the problem. not being able to install multiple versions of a library is a limitation of the packaging tools. conary didn't have a problem with that, as long as packages were built without file conflicts conary did not get confused when you had multiple versions of the same package. it's a pity that the distribution built on it did not survive. that said, we do need containerization, but for a completely different reason: trust and security.
- pornel 3y ago
  Agreed just normalization of build systems is a big part of the hassle. And when the tooling needs to be flexible to accommodate every snowflake project layout, it increases its complexity.

66 points by pabs3 3y ago

Perhaps the title needs "on Linux" added to it. I have a collection of, and written quite a few myself, software which is nothing more than a single EXE. Most of them probably work on anything Win95 and newer. Their "installation" and "uninstallation" are trivial: Extract and run, delete once you've had enough. In other words, packaging software is something that takes work only because of what might often be unnecessary complexity and a correspondingly bureaucratic process that's grown around it, and not because it necessarily needs to. "Win32 is the stable Linux ABI" is more than just a meme.

NoZebra120vClip 3y ago
So, don't need any configuration or preferences, then? It must suck to set up from scratch every time. InstallShield was invented for a reason: the Win32 devs started dumping crap all over the system, all over the registry, and didn't bother cleaning up after themselves. A cottage industry of file and registry cleaners arose. Soon, software publishers began to learn how to properly package their software to be sustainable, and MS cooperated with Windows features to manage apps. Packaged apps.
- mikewarot 3y ago
  It's up to the application to maintain any preferences you set in it. The installer package generally takes care of path configurations, etc. It's amazing how easy it's been to make Win32 installable programs since about 1990.
  - NoZebra120vClip 3y ago
    Uninstall has got to properly clean the registry, and filesystem, and should require manifests.
    - ponderings 3y ago
      I cant remember the last time I've uninstalled anything.
- userbinator 3y ago
  It must suck to set up from scratch every time. What? If you haven't used the software before, what else other than defaults do you? InstallShield was invented for a reason: the Win32 devs started dumping crap all over the system, all over the registry, and didn't bother cleaning up after themselves That's a problem created by advocating for use of the registry, but not a necessity. The single-file apps I have may create a config file alongside the binary, but that's it.
  - matkoniecz 3y ago
    > What? If you haven't used the software before, what else other than defaults do you? On Windows it (by the time I left it, around 2013) it apparently meant bundling various unrelated stuff (like installing Chrome, toolbars, McAfee antivirus trial). Half of installers had minigame "how to spot and stop crapware bundling". Or at least asking blatantly obvious questions and lying that I read and accepted piles of legalese written by their lawyers. Not sure has it got better. Linux style packaging eliminated this for user, at least in my case.
    - TeMPOraL 3y ago
      > Linux style packaging eliminated this for user, at least in my case. That's in part because of the target audience on Linux was historically of the kind that didn't give a damn about this. Compare how analogous CLI package managers (chocolatey, winget) work on Windows: the first thing after identifying what to install is to query whether you accept relevant licensing terms.
      - NoZebra120vClip 3y ago
        Yeah I don't know about that. In F/OSS there is traditionally no EULA to accept. There is a license to use/modify/share the software, and it may be a page or two long, and most licenses are found in just a few categories, such as MIT, BSD, Apache, GPL. Literally none of these licenses, in their original forms, place any restrictions or conditions on how an end-user can use the software; they're aimed at developers and contributors' use and distribution of the source and binaries. Also, I don't know how non-Ubuntu package management organizes software, but in Ubuntu, you need to make a conscious decision to include the "nonfree" type of software. Now presumably for a Linux package manager, any installation action should hopefully import any license terms to a known, readable place for the user to peruse, before launch, if necessary. Perhaps many Linux users don't care, as you say, but I know a lot more who are exquisitely sensitive to licensing and EULAs and will scrutinize this stuff a lot more than a Windows user who just gobbles shareware from C|NET.
medellin 3y ago
I would venture to say you haven’t packaged or shipped any complex software or had to do so in a way that scales to hundreds or thousands of machines. This is along the line of “i could build uber with 50 engineers”.
- userbinator 3y ago
  Take a look at https://news.ycombinator.com/item?id=14748924 https://news.ycombinator.com/item?id=14748924 and you may find a whole new world you probably haven't seen before.
  - userbinator 3y ago
    You can try to keep on thinking that it doesn't exist, but the truth is there for you to see.
- ponderings 3y ago
  ...wait, didn't we have taxi's before computers? Rickshaw, Japan, 1860?
  - mdwalters 3y ago
    The first (mechanical) computer was made in 1822, and the computer has origins dating back to the 17th century. https://en.wikipedia.org/wiki/Computer https://en.wikipedia.org/wiki/Computer
    - TeMPOraL 3y ago
      Then again, taxis date back to at least Ancient Rome (IIRC they found a taximeter device using pebbles to count wheel rotations). And arguably, knowing humans, as soon as we figured out how to reliably use an animal to move a person, someone started offering moving people in exchange for money...
    - ponderings 3y ago
      The Antikythera mechanism suggests quite a bit of previous art. Making a device for counting wheel rotations wouldn't be very difficult. You could do the billing on a wheel too! One set for each customer! But it would be easier to just do the billing by carving it into a clay tablet.
matkoniecz 3y ago
From user viewpoint: Starting to using Linux the greatest part was no longer dealing with insane install processes. For example, end of game where every Java update was pushing some unrelated software and you needed to be careful to not end with it. Also, "is it official site for this software or some copy of website or hijacked site" (and around time I left Windows SourceForge was busy with hijacking open source projects and adding malicious software to this downloads, maybe it got better since that time) If all software would be unpackable and require no configuration by default and leave no waste files over system it would be OK.
cratermoon 3y ago
More correctly, the title could end "unless you only ever target Win32 on x86". Shipping software that's at all portable across operating systems and architectures, that's a different story.
- userbinator 3y ago
  In my experience, that sort of portability is overrated anyway. Win32 on x86 is going to be fine for a majority of users.
  - cratermoon 3y ago
    The majority of desktop users, perhaps. Most users are going to be customers of a system that likely runs “in the cloud" and that means, by far, Linux.
- orwin 3y ago
  Nah, even for that. I don't know what gp is on. I recently downloaded a project a friend made on rpgmaker. I had to manually download rpgmaker itself, a dll and a new font. It wasn't a complex project, he isn't even a developer, he mostly make illustrations and graphical assets. It was really, really basic stuff. That said, I have almost 10 .exe that just work and that I take from computer to computer since 2015, like I have around 30 .sh/.py/.lisp that i do too. Being charitable, I think GP just have a lot of scripts/small tools that just work, and he was referring to that.
  - cratermoon 3y ago
    It works on my machine!
giomasce 3y ago
That doesn't really align with the number of redistributable installers and "DLL not found" errors I've had to deal with during my Windows life. My wife's Win 10 system always boots to a "some DLL not found" fatal message, but after clicking OK everything seems to work as intended. I can't say I trust the way that system is set up.
viraptor 3y ago
> I have a collection of, and written quite a few myself, software which is nothing more than a single EXE. Trivial software for power users (who can download, run and find the exe again) has trivial packaging. This is the case on any system. Your app has no entry in the start menu, no file type binding, no services, no uninstallation entry in the control panel, no large dependencies, etc.
jmrm 3y ago
That's why I love so much the AppImage project. Just download, double click, and you can execute whatever you want.

duped 3y ago

It takes work because the design of FHS is outdated and will eventually die off when we have working package managers that can deal with incoherency of their packages, and figure out a less heavy handed approach to redundancy. Containerization isn't really the answer but it uses the same tools that the answer will have: chroot jails, Linux namespaces, and bind mounts + overlays. The problem with containers is that the layer model doesn't lead to anything actually being shared, but the basic feature set of the user lane gives the tools to a possible solution that doesn't suck. Nix is close, but suffers from design issues orthogonal to how it works. Maybe Nixos is the future once end users never need to read a flake.

johnny22 3y ago
isn't that part about containerization but using shared things at least partially handled by flatpak? I still hope we end up closer to what does or guix does, but at least it's a step in the right direction for various packages.

WhyNotHugo 3y ago

> sometimes you get lucky and the packaging instructions are easy to write (Debian rules, RPM specfiles, whatever Arch uses, etc) Funny way to word this. Arch’s PKGBUILD syntax is very very simple. If you now some programming (especially some shell scripting), you should be able to figure out all the basics. Here’s a non-trivial example: https://gitlab.archlinux.org/archlinux/packaging/packages/todoman/-/blob/main/PKGBUILD https://gitlab.archlinux.org/archlinux/packaging/packages/to... Alpine’s APKBUILD are pretty similar too. Here’s the same package as above: https://git.alpinelinux.org/aports/tree/testing/todoman/APKBUILD https://git.alpinelinux.org/aports/tree/testing/todoman/APKB... Debian packages? Good luck trying to write one. Fedora’s specfiles are far from simple too… though you might be able to at least read one and get a vague idea of what’s going on. I suspect both of these two are the way they are due to historical reasons… it’s a shame that they’re unlikely to change. There’s so much simpler way to write packages.

mike_hock 3y ago
https://salsa.debian.org/python-team/packages/todoman/-/blob/master/debian/control https://salsa.debian.org/python-team/packages/todoman/-/blob... https://salsa.debian.org/python-team/packages/todoman/-/blob/master/debian/rules https://salsa.debian.org/python-team/packages/todoman/-/blob... https://salsa.debian.org/python-team/packages/todoman/-/blob/master/debian/todoman.install https://salsa.debian.org/python-team/packages/todoman/-/blob... If you concatenate them, you get about the same as the PKGBUILD and APKBUILD files. There's a couple more one-liners that I omitted. I fail to see how one is supposed to be easy and the other is hard.
- mkj 3y ago
  The debian ones seem to have more bespoke languages to learn rather than shell script fragments, and more mystery dh-* things that need to be known.
  - mike_hock 3y ago
    That is true, but it's not "good luck making one." The "bespoke language" of debian/control is trivial, debian/rules is a makefile and the rest is mostly "one file glob per line" or that sort of thing.

samsquire 3y ago

I have an idea how to implement automatic packaging but requires operating system be changed. It would enable seamless uninstall. You tag every resource, file and API calls or change with an unfabricatable correlation identifier or cause that can be traced back to origin. I once used checkinstall to create debian packages. Creating a package is just a recording, no manual labour. You can follow the chain of causality to an executable. If you delete the executable, you are left with orphans which you might want to delete.

em-bee 3y ago

packaging is not much more work than building something from source. it's all the same process until the end step where you use the packaging tools to assemble the whole thing. it can get tedious when you end up having to package dependencies as well. but it's mainly grunt work, and none of the work is special to packaging. without packaging everyone would have to do the same work. my favorite packaging tool was conary. it has automated packaging to such an extent that in most cases all that is needed is to tell it the command to run the build and install processes for that application. it figures out everything else by itself. which files are documentation, what are libraries. what are dependencies. one thing it did was to split a package into components. there is a :doc component, a :mysql component and others, which were especially useful for optional dependencies. if you didn't use mysql you could leave out the :mysql component and install the package without a mysql dependency. where with deb or rpm i had to manually figure out what goes where if i wanted to split up a package, in conary that process was mostly automatic. and, to address another comment here, conary worked just fine with the FHS. i do not think that the FHS was or is the problem. not being able to install multiple versions of a library is a limitation of the packaging tools. conary didn't have a problem with that, as long as packages were built without file conflicts conary did not get confused when you had multiple versions of the same package. it's a pity that the distribution built on it did not survive. that said, we do need containerization, but for a completely different reason: trust and security.

pornel 3y ago
Agreed just normalization of build systems is a big part of the hassle. And when the tooling needs to be flexible to accommodate every snowflake project layout, it increases its complexity.