6 ms·

I seriously hate this package and the manner of combining different utils with different names in a same package in general. The reason being, "sponge" is an ac

by krick 6y ago

I seriously hate this package and the manner of combining different utils with different names in a same package in general. The reason being, "sponge" is an actually useful tool and for me it's pretty much the only useful tool in the package. So I need to install whole moreutils package on ubuntu to have "sponge" and I have to clog my bin namespace with all this trash. It would be mildly annoying from the perfectionist point of view, but ok. But there is also "parallel" command in this package, which is trash as well. Meanwhile GNU "parallel" is not trash, but an actually useful program, maybe more useful than "sponge", and it comes in a separate package. So I either have to do some tinkering, or I have to choose which one I need more: "sponge" or "parallel" and install only one package. It's year 2020. This is stupid. This is by the way the reason why GraphicsMagick is better than ImageMagick (I still use the latter, though, because it doesn't cause the same problems for me as moreutils does, and it's just more popular than GM).

nickthemagicman 6y ago
Is there a way to namespace on install? Or does that have to be configured in the package?
- krick 6y ago
  I am not aware of such a way with apt on ubuntu. It can be different with some other package managers, obviously.
  - andrewshadura 6y ago
    dpkg-divert
- waynesonfire 6y ago
  Docker
- kps 6y ago
  You put them in different directories, e.g. Solaris-style /opt/package/…, and arrange your PATH accordingly.
- jolmg 6y ago
  What I normally see in Archlinux is a rename of the less common conflicting utility when the package is built. In this example, the moreutils package's parallel utility and its manpage got renamed to parallel-moreutils.
appleflaxen 6y ago
you raise a great point. why not install the package, then delete the /usr/bin files you don't need? would it solve the problem if the author namespaced the commands with a hyphenated prefix? curious about these considerations, which it seems like you've spent time thinking about. what do you see as the "best practices" regarding a set of utilities that are maintained and published together?
- krick 6y ago
  In a most general sense, this is not really a problem of moreutils, but of how tools are installed/distributed in Linux distributions. But since we have to be realistic, yes, package authors should take such problems into consideration. I think if tools are absolutely unrelated, they just should be distributed as a separate packages. GNU coreutils is tolerated mostly because it's so ubiquitous (so much, that it causes Stallman to grumble about "you mean GNU/Linux, not Linux"). moreutils is late to the party, it isn't ubiquitous, and the usefulness of any tool in the package is questionable, so the author really shouldn't be so brave to assume that if he thought he needed all of them, everyone will. If there is a reason enough to distribute a package with several separate callable binaries (as with ImageMagick), I think git or GraphicsMagick are perfect examples of how it should be done. Hypenated prefix is also ok. Even if your tool is supposed to be used by somebody 50 times a day, too long of a name isn't really a problem, since user can always just make an alias (as I do with most of the tools I use frequently). Sure you can always combine binaries from 2 packages manually, but as I said, it just requires some tinkering, so I cannot simply have in some textfile a list of utils to install on a new PC in a matter of minutes.
  - boomboomsubban 6y ago
    Stick the binary wherever your textile is, copy it to path on setup.
- SQueeeeeL 6y ago
  Excepting the user to have to clean cruft to install the software they want is a terrible philosophy and not in line with Unix at all, now you've created a situation where to install "sponge" you need to place 10 other random binaries onto your machine and know enough about the process to know to delete them...
umvi 6y ago
How is sponge different from just normal file redirection?
- fred256 6y ago
  See the first example on the web page: % sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd If you tried this as: % sed "s/root/toor/" /etc/passwd | grep -v joey > /etc/passwd The shell would overwrite /etc/passwd with an empty file before sed has a chance to read from it.
bewuethr 6y ago
In version 7, ImageMagick has switched to just "magick" instead of "convert", "mogrify" etc., and for compatibility with version 6 it still supports subcommands like "magick convert" etc., but at least the namespace cluttering is gone. https://imagemagick.org/script/command-line-tools.php https://imagemagick.org/script/command-line-tools.php Edit: I wasn't quite right, the commands still exist, but they are symlinks to "magick" and can be called using the subcommand mentioned above; you still have to delete the symlinks to clean up the namespace. https://imagemagick.org/script/porting.php#cli https://imagemagick.org/script/porting.php#cli
swiley 6y ago
Don't most shells implement something like ">"? How is sponge an improvement on this?
- pfarrell 6y ago
  In the provided example % sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd It looks like this allows an in place modification to the same doc. If you used % sed "s/root/toor/" /etc/passwd | grep -v joey > /etc/passwd Bash will process the redirection first, then execute the commands in the pipe. So, it will have cleared out /etc/passwd with the non-appending redirection ">" before sed operates on it. You'd end up with a blank /etc/passwd file. (ymmv, I don't know how other shells would handle this) Sponge, it seems, would cause the pipe to allow the preceding commands to complete before it outputs.
- sigstoat 6y ago
  you can sudo sponge, and not run the entire command with elevated privileges. (i'd welcome a method of doing that in zsh.)
bryanlarsen 6y ago
Since Debian Stretch / Ubuntu 17.04 they do the right thing. If you install parallel & moreutils you get the GNU parallel as /usr/bin/parallel and moreutil's parallel as /usr/bin/parallel.moreutils. If you only install moreutils, it provides /usr/bin/parallel. You can use the debian alternatives system to flip the order. https://superuser.com/a/1253492/78988 https://superuser.com/a/1253492/78988
- dredmorbius 6y ago
  Another alternative might be to have a set of packages, 'moreutils-spnge, moreutiles-pee, ... moreutils-common, moreutils-doc', which could be installed independently, and, if desired, a 'moreutils' virtual package dependent on the whole set for simplified installation. Debian also has namespace-collision policies and an 'alternatives' facility for deciding which of multiple implementations of a tool (gawk, nawk, mawk; vim, vim-tiny, nvi; python3, python2; etc.) is primary on a given system. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597050 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597050 https://www.debian.org/doc/debian-policy/ch-binary.html#virtual-packages https://www.debian.org/doc/debian-policy/ch-binary.html#virt... https://www.debian.org/doc/debian-policy/ap-pkg-alternatives.html https://www.debian.org/doc/debian-policy/ap-pkg-alternatives...
hinkley 6y ago
I don't recall how difficult it is to maintain a debian package. It seems like it shouldn't be too much work to fork the repository, and tweak the packaging logic to only include this one binary. You'd still have to compile the whole thing every time, but rebasing would be straightforward.
plq 6y ago
On principle, I agree with you -- this project should be split into multiple packages. However, it's the distro's job to harmoniously put various open source packages together. It's impossible for package authors to make everyone happy. Plus, in the moreutils case, the author has already gone to great lengths to publish his work for free, he shouldn't have to put up with more work to the point where it stops being fun for him/her. Finally, it's just NOT appropriate to call parts of this project "trash". He scrached his itch and was kind enough to publish his work. If you don't like it, just don't use it, no need to be harsh about it. The right way for distros to package this is to split the moreutils source distribution into moreutils-* packages, plus a meta package that pulls in all of the commands. So your complaint actually belongs in your distro's bug tracker -- it has nothing to do with the moreutils project.
- saagarjha 6y ago
  Calling software "trash" is uncalled for, but squatting on a namespace is a real thing and not something that you can justify behind the usual "someone did this for free and owes you nothing" because this make the landscape worse for everyone else.

by krick 6y ago

nickthemagicman 6y ago

Is there a way to namespace on install? Or does that have to be configured in the package?

krick 6y ago
I am not aware of such a way with apt on ubuntu. It can be different with some other package managers, obviously.
- andrewshadura 6y ago
  dpkg-divert
waynesonfire 6y ago
Docker
kps 6y ago
You put them in different directories, e.g. Solaris-style /opt/package/…, and arrange your PATH accordingly.
jolmg 6y ago
What I normally see in Archlinux is a rename of the less common conflicting utility when the package is built. In this example, the moreutils package's parallel utility and its manpage got renamed to parallel-moreutils.

appleflaxen 6y ago

you raise a great point. why not install the package, then delete the /usr/bin files you don't need? would it solve the problem if the author namespaced the commands with a hyphenated prefix? curious about these considerations, which it seems like you've spent time thinking about. what do you see as the "best practices" regarding a set of utilities that are maintained and published together?

krick 6y ago
In a most general sense, this is not really a problem of moreutils, but of how tools are installed/distributed in Linux distributions. But since we have to be realistic, yes, package authors should take such problems into consideration. I think if tools are absolutely unrelated, they just should be distributed as a separate packages. GNU coreutils is tolerated mostly because it's so ubiquitous (so much, that it causes Stallman to grumble about "you mean GNU/Linux, not Linux"). moreutils is late to the party, it isn't ubiquitous, and the usefulness of any tool in the package is questionable, so the author really shouldn't be so brave to assume that if he thought he needed all of them, everyone will. If there is a reason enough to distribute a package with several separate callable binaries (as with ImageMagick), I think git or GraphicsMagick are perfect examples of how it should be done. Hypenated prefix is also ok. Even if your tool is supposed to be used by somebody 50 times a day, too long of a name isn't really a problem, since user can always just make an alias (as I do with most of the tools I use frequently). Sure you can always combine binaries from 2 packages manually, but as I said, it just requires some tinkering, so I cannot simply have in some textfile a list of utils to install on a new PC in a matter of minutes.
- boomboomsubban 6y ago
  Stick the binary wherever your textile is, copy it to path on setup.
SQueeeeeL 6y ago
Excepting the user to have to clean cruft to install the software they want is a terrible philosophy and not in line with Unix at all, now you've created a situation where to install "sponge" you need to place 10 other random binaries onto your machine and know enough about the process to know to delete them...

umvi 6y ago

How is sponge different from just normal file redirection?

fred256 6y ago
See the first example on the web page: % sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd If you tried this as: % sed "s/root/toor/" /etc/passwd | grep -v joey > /etc/passwd The shell would overwrite /etc/passwd with an empty file before sed has a chance to read from it.

bewuethr 6y ago

In version 7, ImageMagick has switched to just "magick" instead of "convert", "mogrify" etc., and for compatibility with version 6 it still supports subcommands like "magick convert" etc., but at least the namespace cluttering is gone. https://imagemagick.org/script/command-line-tools.php https://imagemagick.org/script/command-line-tools.php Edit: I wasn't quite right, the commands still exist, but they are symlinks to "magick" and can be called using the subcommand mentioned above; you still have to delete the symlinks to clean up the namespace. https://imagemagick.org/script/porting.php#cli https://imagemagick.org/script/porting.php#cli

swiley 6y ago

Don't most shells implement something like ">"? How is sponge an improvement on this?

pfarrell 6y ago
In the provided example % sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd It looks like this allows an in place modification to the same doc. If you used % sed "s/root/toor/" /etc/passwd | grep -v joey > /etc/passwd Bash will process the redirection first, then execute the commands in the pipe. So, it will have cleared out /etc/passwd with the non-appending redirection ">" before sed operates on it. You'd end up with a blank /etc/passwd file. (ymmv, I don't know how other shells would handle this) Sponge, it seems, would cause the pipe to allow the preceding commands to complete before it outputs.
sigstoat 6y ago
you can sudo sponge, and not run the entire command with elevated privileges. (i'd welcome a method of doing that in zsh.)

bryanlarsen 6y ago

Since Debian Stretch / Ubuntu 17.04 they do the right thing. If you install parallel & moreutils you get the GNU parallel as /usr/bin/parallel and moreutil's parallel as /usr/bin/parallel.moreutils. If you only install moreutils, it provides /usr/bin/parallel. You can use the debian alternatives system to flip the order. https://superuser.com/a/1253492/78988 https://superuser.com/a/1253492/78988

dredmorbius 6y ago
Another alternative might be to have a set of packages, 'moreutils-spnge, moreutiles-pee, ... moreutils-common, moreutils-doc', which could be installed independently, and, if desired, a 'moreutils' virtual package dependent on the whole set for simplified installation. Debian also has namespace-collision policies and an 'alternatives' facility for deciding which of multiple implementations of a tool (gawk, nawk, mawk; vim, vim-tiny, nvi; python3, python2; etc.) is primary on a given system. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597050 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597050 https://www.debian.org/doc/debian-policy/ch-binary.html#virtual-packages https://www.debian.org/doc/debian-policy/ch-binary.html#virt... https://www.debian.org/doc/debian-policy/ap-pkg-alternatives.html https://www.debian.org/doc/debian-policy/ap-pkg-alternatives...

hinkley 6y ago

I don't recall how difficult it is to maintain a debian package. It seems like it shouldn't be too much work to fork the repository, and tweak the packaging logic to only include this one binary. You'd still have to compile the whole thing every time, but rebasing would be straightforward.

plq 6y ago

On principle, I agree with you -- this project should be split into multiple packages. However, it's the distro's job to harmoniously put various open source packages together. It's impossible for package authors to make everyone happy. Plus, in the moreutils case, the author has already gone to great lengths to publish his work for free, he shouldn't have to put up with more work to the point where it stops being fun for him/her. Finally, it's just NOT appropriate to call parts of this project "trash". He scrached his itch and was kind enough to publish his work. If you don't like it, just don't use it, no need to be harsh about it. The right way for distros to package this is to split the moreutils source distribution into moreutils-* packages, plus a meta package that pulls in all of the commands. So your complaint actually belongs in your distro's bug tracker -- it has nothing to do with the moreutils project.

saagarjha 6y ago
Calling software "trash" is uncalled for, but squatting on a namespace is a real thing and not something that you can justify behind the usual "someone did this for free and owes you nothing" because this make the landscape worse for everyone else.