19 ms·

Moreutils: A collection of Unix tools that nobody thought to write long ago

• deleted 4y ago
  [deleted]
• I_complete_me 4y ago
  I only install moreutils for vidir which is simply brilliant IMO
  • teddyh 4y ago
    I prefer Emacs dired, where pressing C-x C-q starts editing the opened directory (including any inserted subdirectories).
    • deleted 4y ago
      [deleted]
  • VTimofeenko 4y ago
    One alias I always do is "vidir" -> "vidir --verbose" so that it would tell me what it's doing
• zzo38computer 4y ago
  I have this installed and have used some of these sometimes; it is good. (I do not use all of them, though)
• jchw 4y ago
  How might one use sponge in a way that shell redirection wouldn’t be more fully-featured? The best I can currently think of is that it’s less cumbersome to wrap (for things like sudo.)
  • JoshTriplett 4y ago
    Sponge exists for cases where shell redirection wouldn't work, namely where you want the source and sink to be the same file. If you write: somecmd < somefile | othercmd | anothercmd > somefile the output redirection will truncate the file before it can get read as input. Sponge "soaks up" all the output before writing any of it, so that you can write pipelines like that: somecmd < somefile | othercmd | anothercmd | sponge somefile
    • jchw 4y ago
      Thank you, I missed this bit of nuance. That indeed would be useful, and now the example makes a lot more sense.
    • scbrg 4y ago
      This can be done with regular shell redirection, even though I wouldn't recommend it. Easy to get wrong, and fairly opaque: $ cat foo foo bar baz $ ( rm foo && grep ba > foo ) < foo $ cat foo bar baz $
• JoshTriplett 4y ago
  I use a few of these regularly: ts timestamps each line of the input, which I've found convenient for ad-hoc first-pass profiling in combination with verbose print statements in code: the timestamps make it easy to see where long delays occur. errno is a great reference tool, to look up error numbers by name or error names by number. I use this for two purposes. First, for debugging again, when you get an errno numerically and want to know which one it was. And second, to search for the right errno code to return, in the list shown by errno -l. And finally, vipe is convenient for quick one-off pipelines, where you know you need to tweak the input at one point in the pipeline but you know the nature of the tweak would take less time to do in an editor than to write the appropriate tool invocation to do it.
  • ducktective 4y ago
    I actually installed moreutils just for errno, but I got disappointed. Here it the whole `errno -l` : http://ix.io/3VeV http://ix.io/3VeV Like, none of the everyday tools I use produce exit codes which correspond to these explanations. For my scripts, I just return 1 for generic erros and 2 for bad usage. I wished I could be more specific in that and adhered to some standard.
    • JoshTriplett 4y ago
      errno codes aren't used in the exit codes of command-line tools; they're used in programmatic APIs like syscalls and libc functions. There's no standard for process exit codes, other than "zero for success, non-zero for error".
      • zzo38computer 4y ago
        Some programs do follow a specification (sysexits.h) where numbers 64 and higher are used for common uses, such as usage error, software error, file format error, etc.
        • ithkuil 4y ago
          Aldo when the oomkiller process kills another process it causes it to exit with a well known exit code: 137
• usr1106 4y ago
  I have searched ts for a long time. Used it many years ago but forgot the exact name of the tool and package. No search machine could find it or I just entered to wrong search words.
• topher200 4y ago
  The one I use the most is `vipe`. > vipe: insert a text editor into a pipe It's useful when you want to edit some input text before passing it to a different function. For example, if I want to delete many of my git branches (but not all my git branches): $ git branch | vipe | xargs git branch -D `vipe` will let me remove some of the branch names from the list, before they get passed to the delete command.
  • bombcar 4y ago
    This is useful - I usually either use an intermediate file or a bunch of grep -v
  • orblivion 4y ago
    And what if you decide mid-vipe "oh crap I don't want to do this anymore"? In the case of branches to delete you could just delete every line. In other cases maybe not?
    • saagarjha 4y ago
      Send a SIGTERM to the editor, maybe?
      • Beltalowda 4y ago
        And there's always the power button on your computer :-)
      • orblivion 4y ago
        Kind of ugly, but yeah, that's what I'd imagine doing.
    • throwaway09223 4y ago
      It will depend on the commands in question. The entire unix pipeline is instantiated in parallel, so the commands following vipe will already be running and waiting on stdin. You could kill them before exiting the editor, if that's what you want. Or you could do something else. The other commands in the pipeline are run by the parent shell, not vipe, so handling this would not be vipe specific.
• escot 4y ago
  Using `vipe` you can do things like: $ pbpaste | vipe | pbcopy Which will open your editor so you can edit whatever is in your clipboard.
  • IshKebab 4y ago
    Just realised what the pb stands for. Did they really not think of clipboard? Who came up with the "clipboard" name?
    • zarzavat 4y ago
      The pasteboard, like many things in OS X, is from NextStep. As for why they called it a pasteboard and not a clipboard, I have no idea, presumably someone thought it would be more descriptive.
    • deleted 4y ago
      [deleted]
    • Delk 4y ago
      Wikipedia says "clipboard" was coined by Larry Tesler (in the 70's?)
• marcodiego 4y ago
  Long time ago, a colleague of mine created "evenmoreutils" : https://github.com/rudymatela/evenmoreutils https://github.com/rudymatela/evenmoreutils
  • efreak 4y ago
    Anywait is the tool I've always wanted, and implemented in pretty much the same way I would do so. However, besides waiting on a pid, I occasionally wait on `pidof` instead, to wait on multiple instances of the same process, running under different shells (wait until all build jobs are done, not just the current job). ched also looks quite useful; automatically cleaning the old data after some time is great, as I commonly leave it lying around. age also looks great for processing recent incoming files in a large directory (my downloads, for example) p looks great to me, I rarely need the more advanced features of parallel, and will happily trade them for color coded outputs. I looked to see what nup is because I don't understand the description...only to find out it doesn't actually exist. I'm assuming it's intended to send a signal to a process? But if so, why not just use `kill -s sigstop`? pad also doesn't exist, but seems like printf or column could replace it, as these are what I usually use. I think there's also a way to pad variables in bash/zsh/etc. whl is literally just `while do_stuff; do; done` and repeat is just `while true; do do_stuff; done`. It never even occurred to me to look for a tool to do untl; I usually just use something along the lines of `while ! do_stuff; do c=$((c + 1)); echo $c; done`. While the interval and return codes make it almost worthwhile, they themselves are still very little complexity; parsing the parameters adds more complexity than their implementation does. spongif seems useful, but is really just a variation of the command above.
    • prosaole 4y ago
      For color coding try: seq 10 | parallel --lb --ctag ping {}.1.1.1
• sundarurfriend 4y ago
  The "What's included" section direly needs either clearer/longer descriptions, or at least links to the tools' own pages (if they have them) where their use case and usage is explained. I've understood a lot more about (some of) the tools from the comments here than from the page - and I'd likely have skipped over these very useful tools if not for these comments!
  • sundarurfriend 4y ago
    Ok, longer descriptions from the tools' man pages: --- chronic runs a command, and arranges for its standard out and standard error to only be displayed if the command fails (exits nonzero or crashes). If the command succeeds, any extraneous output will be hidden. A common use for chronic is for running a cron job. Rather than trying to keep the command quiet, and having to deal with mails containing accidental output when it succeeds, and not verbose enough output when it fails, you can just run it verbosely always, and use chronic to hide the successful output. --- combine combines the lines in two files. Depending on the boolean operation specified, the contents will be combined in different ways: and Outputs lines that are in file1 if they are also present in file2. not Outputs lines that are in file1 but not in file2. or Outputs lines that are in file1 or file2. xor Outputs lines that are in either file1 or file2, but not in both files. The input files need not be sorted --- ifdata can be used to check for the existence of a network interface, or to get information about the interface, such as its IP address. Unlike ifconfig or ip, ifdata has simple to parse output that is designed to be easily used by a shell script. --- lckdo: Now that util-linux contains a similar command named flock, lckdo is deprecated, and will be removed from some future version of moreutils. --- mispipe: mispipe pipes two commands together like the shell does, but unlike piping in the shell, which returns the exit status of the last command; when using mispipe, the exit status of the first command is returned. Note that some shells, notably bash, do offer a pipefail option, however, that option does not behave the same since it makes a failure of any command in the pipeline be returned, not just the exit status of the first. --- pee: [my own description: `pee cmd1 cmd2 cmd3` takes the data from the standard input, sends copies of it to the commands cmd1, cmd2, and cmd3 (as their stdin), aggregates their outputs and provides that at the standard output.] --- sponge, ts and vipe have been described in other comments in this thread. (And I've also skipped some easier-to-understand ones like errno and isutf8 for the sake of length.) --- zrun: Prefixing a shell command with "zrun" causes any compressed files that are arguments of the command to be transparently uncompressed to temp files (not pipes) and the uncompressed files fed to the command. The following compression types are supported: gz bz2 Z xz lzma lzo [One super cool thing the man page mentions is that if you create a link named z<programname> eg. zsed, with zrun as the link target, then when you run `zsed XYZ`, zrun will read its own program name, and execute 'zrun sed XYZ' automatically.] ---
• HellsMaddy 4y ago
  I find sponge really useful. Have you ever wanted to process a file through a pipeline and write the output back to the file? If you do it the naive way, it won’t work, and you’ll end up losing the contents of your file: awk '{do_stuff()}' myfile.txt | sort -u | column --table > myfile.txt The shell will truncate myfile.txt before awk (or whatever command) has a chance to read it. So you can use sponge instead, which waits until it reads EOF to truncate/write the file. awk '{do_stuff()}' myfile.txt | sort -u | column --table | sponge myfile.txt
  • jamespwilliams 4y ago
    Are there any legitimate reasons to have a particular file both as an input to a pipe and as an output? I wonder whether a shell could automatically “sponge” the pipe’s output if it detected that happening.
    • teawrecks 4y ago
      Yeah, it seems like the kind of command that you only need because of a quirk in how the underlying system happens to work. Not something that should pollute the logic of the command, imo. I would expect a copy-on-write filesystem to be able to do this automatically for free.
      • paulmd 4y ago
        > I would expect a copy-on-write filesystem to be able to do this automatically for free. this is an artifact of how handles work (in relation to concurrency), not the filesystem. copy-on-write still guarantees a consistent view of the data, so if you write on one handle you're going to clobber the data on the other, because that's what's in the file. what you really want is an operator which says "I want this handle to point to the original snapshot of this data even if it's changed in the meantime", which a CoW filesystem could do, but you'd need some additional semantics here (different access-mode flag?) which isn't trivially granted just by using a CoW filesystem underneath.
      • caymanjim 4y ago
• PaulDavisThe1st 4y ago
  The need for ifdata(1) has become even more acute with the essential replacement of ifconfig(1) by ip(1), an even more inscrutable memory challange. However, it would be even nicer if its default action when not given an interface name was do <whatever> for all discovered interfaces.
  • teddyh 4y ago
    Do not “ip -brief address show” and “ip -brief link show” serve as suitable replacements for most common uses of ifdata(1)? The ip(8) command even supports JSON output using “-json” instead of “-brief”.
    • zwayhowder 4y ago
      TIL. Thanks. One does have to wonder though, why isn't -brief the default and the current default set to -verbose or -long. I look at -brief on either command and it has all the information I am ever looking for.
    • Filligree 4y ago
      Yes, I suppose, but I can never remember those flags.
      • wang_li 4y ago
        Amazing how many new things come about because people don't know or don't remember how to use the existing things.
      • teddyh 4y ago
        ifdata is not meant for interactive use.
    • hedora 4y ago
      Well, they're strictly worse for interactive use than ifconfig, from what I can tell from your comment.
• JulianWasTaken 4y ago
  moreutils indeed has some great utils, but a minor annoyance it causes is still shipping a `parallel` tool which is relatively useless, but causes confusion for new users or conflict (for package managers) with the way way way more indispensable GNU parallel.
  • yesenadam 4y ago
    When I installed moreutils v0.67 with macports just now it said: moreutils has the following notes: The binary parallel is no longer in this port; please install the port parallel instead. i.e. GNU parallel
    • JulianWasTaken 4y ago
      Yup, homebrew and other package managers do similar it's what I meant by > conflict (for package managers)
  • omoikane 4y ago
    `parallel` seems redundant because it appears that `xargs -P` can accomplish the same effect, except the "-l maxload" option.
    • rmilk 4y ago
      But that’s the point of parallel. The use case is for when you have N processors and a 10Gb NIC. Each job is CPU bound or concurrent license bound, or some jobs may take longer than others. Parallel allows you to run X jobs simultaneously to keep the CPU or licenses busy.
    • JulianWasTaken 4y ago
      If you mean GNU parallel, it has way way more features than xargs -P, (and a few saner defaults). See e.g. https://www.gnu.org/software/parallel/parallel_tutorial.html https://www.gnu.org/software/parallel/parallel_tutorial.html If you mean moreutils parallel, yeah I agree it's not useful.
  • 4y ago
• aendruk 4y ago
  See also the related announcement [1] on the most recent “Volunteer Responsibility Amnesty Day” [2]. [1]: https://joeyh.name/blog/entry/Volunteer_Responsibility_Amnesty_Day/ https://joeyh.name/blog/entry/Volunteer_Responsibility_Amnes... [2]: https://www.volunteeramnestyday.net/ https://www.volunteeramnestyday.net/
  • lupire 4y ago
    > id :: a -> a > id x = x That's "echo".
• BoneZone 4y ago
  I chuckled at pee
  • theandrewbailey 4y ago
    I outright laughed that its juxtaposed with sponge.
• moralestapia 4y ago
  >pee: tee standard input to pipes Nice tool, great name.
  • pixelbeat__ 4y ago
    pee is not really needed these days as with bash you can: tee -p >(command | line) Note the -p option available now with GNU tee that doesn't exit tee if the pipe closes early
• caymanjim 4y ago
  I was prepared to mock this before I even clicked, but I have to say this looks like a nice set of tools that follow the ancient Unix philosophy of "do one thing, play nice in a pipeline, stfu if you have nothing useful to say". Bookmarking this to peer at until I internalize the apps. There's even an Ubuntu package for them. I don't think it's a good idea to rely on any of these being present. If you write a shell script to share and expect them to be there, you aren't being friendly to others, but for interactive command line use, I'm happy to adopt new tools.
  • _dain_ 4y ago
    >I don't think it's a good idea to rely on any of these being present. If you write a shell script to share and expect them to be there, you aren't being friendly to others, but for interactive command line use, I'm happy to adopt new tools. Isn't that a shame though? Where does it say in the UNIX philosophy that the canon should be closed?
    • caymanjim 4y ago
      I think it's fine if you're on a dev team that decides to include these tools in its shared toolkit, but none of these rise to the level that I think warrants them being a dependency for a broadly-distributed shell script. There are slightly-less-terse alternatives for most of the functionality that only rely on core utilities. I don't think it's being a good citizen to say "go install moreutils and its dozen components because I wanted to use sponge instead of >output.txt".
    • Delk 4y ago
      It's not that different than refraining from using non-POSIX syntax in shell scripts that are meant to be independent of a specific flavour of unix, or sticking with standard C rather than making assumptions that are only valid in one compiler. There are shades of grey, of course. Bash is probably ubiquitous enough that it may not be a big issue if a script that's meant to be universal depends on it, as long as the script explicitly specifies bash in the shebang. Sometimes some particular functionality is not technically part of a standard but is widely enough supported in practice. Sometimes the standards (either formal or de facto) are expanded to include new functionality, and that's of course totally fine, but it's not likely to be a very quick process because there are almost certainly going to be differing opinions on what should be part of the core and what shouldn't. Either way, sometimes you want to write for the lowest common denominator, and moreutils certainly aren't common enough that they could be considered part of that.
• pjungwir 4y ago
  These are necessarily bash functions, not executables, but here are two tools I'm proud of, which seem similar in spirit to vidir & vipe: # Launch $EDITOR to let you edit your env vars. function viset() { if [ -z "$1" ]; then echo "USAGE: viset THE_ENV_VAR" exit 1 else declare -n ref=$1 f=$(mktemp) echo ${!1} > $f $EDITOR $f ref=`cat $f` export $1 fi } # Like viset, but breaks up the var on : first, # then puts it back together after you're done editing. # Defaults to editing PATH. # # TODO: Accept a -d/--delimiter option to use something besides :. function vipath() { varname="${1:-PATH}" declare -n ref=$varname f=$(mktemp) echo ${!varname} | tr : "\n" > $f $EDITOR $f ref=`tr "\n" : < $f` export $varname } Mostly I use vipath because I'm too lazy to figure out why tmux makes rvm so angry. . . . I guess a cool addition to viset would be to accept more than one envvar, and show them on multiple lines. Maybe even let you edit your entire env if you give it zero args. Having autocomplete-on-tab for viset would be cool too. Maybe even let it interpret globs so you can say `viset AWS*`. Btw I notice I'm not checking for an empty $EDITOR. That seems like it could be a problem somewhere.
• _joel 4y ago
  Working with *nix for over 25 years and I've only just heard of sponge.
• ChrisGranger 4y ago
  It's been a very long time since this happened, but in my early days of using Linux, I experienced naming collisions with both sponge and parallel, and at the time I didn't know how to resolve them. I don't remember which other sponge there was, but I imagine most Linux users are familiar with GNU parallel at this point.
• Davertron 4y ago
  So just today I was wondering if there was a cli tool (or maybe a clever use of existing tools...) that could watch the output of one command for a certain string, parse bits of that out, and then execute another command with that parsed bit as input. For example, I have a command I run that spits out a log line with a url on it, I need to usually manually copy out that url and then paste it as an arg to my other command. There are other times when I simply want to wait for something to start up (you'll usually get a line like "Dev server started on port 8080") and then execute another command. I know that I could obviously grep the output of the first command, and then use sed or awk to manipulate the line I want to get just the url, but I'm not sure about the best way to go about the rest. In addition, I usually want to see all the output of the first command (in this case, it's not done executing, it continues to run after printing out the url), so maybe there's a way to do that with tee? But I usually ALSO don't want to intermix 2 commands in the same shell, i.e. I don't want to just have a big series of pipes, Ideally I could run the 2 commands separately in their own terminals but the 2nd command that needs the url would effectively block until it received the url output from the first command. I have a feeling maybe you could do this with named pipes or something but that's pretty far out of my league...would love to hear if this is something other folks have done or have a need for.
  • sillysaurusx 4y ago
    I used Python subprocess module for this. … good luck, is my best advice. It’s not straightforward to handle edge cases.
  • ufo 4y ago
    A named pipe sounds like a good way to fulfill your requirement of having the command runs on separate shells.. In the first terminal, shove the output of commend A into the named pipe. In the second terminal, have a loop that reads from the named pipe line by line and invokes command B with the appropriate arguments. You can create a named pipe using "mkfifo", which creates a pipe "file" with the specified name. Then, you can tell your programs to read and write to the pipe the same way you'd tell them to read and write from a normal file. You can use "<" and ">" to redirect stdout/stderr, or you can pass the file name if it's a program that expects a file name.
• figital 4y ago
  here's a little wrapper around i made around "find" which i always have to install on every new box i manage .... https://github.com/figital/fstring https://github.com/figital/fstring (just shows you more useful details about what is found)
  • hnlmorg 4y ago
    Coincidentally I have a very similar (almost identical) function in my shell profile.
• theteapot 4y ago
  Some of this looks useful, but a non exhaustive critique - from non expert - of some of the rest: > chronic: runs a command quietly unless it fails Isn't that just `command >/dev/null`? > ifdata: get network interface info without parsing ifconfig output `ip link show <if>`? > isutf8: check if a file or standard input is utf-8 `file` for files. For stdin, when is it not utf8 - unless you've got some weird system configuration? > lckdo: execute a program with a lock held `flock`?
  • compsciphd 4y ago
    > > chronic: runs a command quietly unless it fails > Isn't that just `command >/dev/null`? no. that just shows your stderr, not stdout if it failed. and you get stderr even if it doesn't fail.
    • spr93 4y ago
      so use '2>&1' to redirect stderr to stdout. new ideas are great. but this isn't a new idea. "run a command silently unless it fails" is basic; it's the sort of thing that should make one think "i should search the shell man pages" rather than "i should roll my own new utility."
      • NavinF 4y ago
        Read his comment again. '2>&1' just redirects stderr.
  • ninkendo 4y ago
    > > chronic: runs a command quietly unless it fails > Isn't that just `command >/dev/null`? Often times you want to run a command silently (like in a build script), but if it fails with a nonzero exit status, you want to then display not only the stderr but the stdout as well. I’ve written makefile hacks in the past that do this to silence overly-chatty compilers where we don’t really care what it’s outputting unless it fails, in which case we want all the output. It would’ve been nice to have this tool at the time to avoid reinventing it.
  • 4y ago
• mc4ndr3 4y ago
  How does sponge compare with tee?
  • dredmorbius 4y ago
    tee(1) duplexes its input, streaming to both the specified file and stdout. sponge(1) soaks up input and writes it to the specified file at end of data. tee doesn't sponge. sponge doesn't tee.
• suprjami 4y ago
  Have loved this collection for a long time. I use errno almost daily.
• tails4e 4y ago
  I wrote a utility like ts before, called it teetime, was thrilled with my pun. It was quiteand useful when piping stdout from a compute heavy tool (multi hour EDA tool run) as you could see by the delta time between logs what the most time consuming parts were.
• WalterBright 4y ago
  > errno: look up errno names and descriptions I like this. Reminds me of a couple very useful things I've done: 1. Add a -man switch to command line programs. This causes a browser to be opened on the web page for the program. For example: dmd -man opens https://dlang.org/dmd-windows.html https://dlang.org/dmd-windows.html in your default browser. 2. Fix my text editor to recognize URLs, and when clicking on the URL, open a browser on it. This silly little thing is amazingly useful. I used to keep bookmarks in an html file which I would bring up in a browser and then click on the bookmarks. It's so much easier to just put them in a plain text file as plain text. I also use it for source code, for example the header for code files starts with: /* * Takes a token stream from the lexer, and parses it into an abstract syntax tree. * * Specification: $(LINK2 https://dlang.org/spec/grammar.html, D Grammar) * * Copyright: Copyright (C) 1999-2020 by The D Language Foundation, All Rights Reserved * Authors: $(LINK2 http://www.digitalmars.com, Walter Bright) * License: $(LINK2 http://www.boost.org/LICENSE_1_0.txt, Boost License 1.0) * Source: $(LINK2 https://github.com/dlang/dmd/blob/master/src/dmd/parse.d, _parse.d) * Documentation: https://dlang.org/phobos/dmd_parse.html * Coverage: https://codecov.io/gh/dlang/dmd/src/master/src/dmd/parse.d */ and I'll also use URLs in the source code to reference the spec on what the code is implementing, and to refer to closed bug reports that the code fixes. Very, very handy!
  • WalterBright 4y ago
    P.S. I mentioned having links in the source code to the part of the spec. The only problem with this is when the spec (i.e. the C11 Standard) is not in html form. I can only add the paragraph number in the code. What an annoying waste of time every time I want to check that the implementation is exactly right. For contrast, there's this site: https://www.felixcloutier.com/x86/index.html https://www.felixcloutier.com/x86/index.html which is a godsend to me. Now, in the dmd code generator, I put in links to the detail page for an instruction when the code generator is generating that instruction. Oh, how marvelous that is! And there is joy in Mudville.
    • mhh__ 4y ago
      Intel actually lags behind the industry in that they don't really have a formal specification. Arm have a machine readable specification that can be verified by a computer whereas Intel have this weird pseudocode. Also uops.info is a good reference for how fast the instructions are
• hgomersall 4y ago
  Are they written in rust? If not, it's just more programmes someone is going you have to rewrite at some point sigh.
  • hgomersall 4y ago
    Fyi, this is a joke.
• mftb 4y ago
  I have used vidir from this collection quite a bit. If you're a vi person, it's makes it quite convenient to use vi/vim for renaming whole directories full of files.
  • gsinclair 4y ago
    I prefer vifm for this, but I’ll give visit a try as well, now that I know about it.
    • mftb 4y ago
      Ha! I use vifm now too, since this functionality is pretty much built in, but I still use vidir for one-offs and I thought it might appeal to some of the true minimalists.
• jedberg 4y ago
  My favorite missing tool of all time is `ack` [0]. It's grep if grep were made now. I use it all the time, and it's the first thing I install on a new system. It has a basic understanding of common text file structures and also directories, which makes it super powerful. [0] https://beyondgrep.com https://beyondgrep.com
  • outworlder 4y ago
    What about the Silver Searcher? It's even faster than ack https://github.com/ggreer/the_silver_searcher https://github.com/ggreer/the_silver_searcher
    • jedberg 4y ago
      Had never heard of it before, but I'll definitely check it out now. Thanks!
    • makeworld 4y ago
      What about ripgrep? It's even faster than the Silver Searcher https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep
  • renewiltord 4y ago
    ripgrep is a modern written-in-Rust[0] equivalent that I really like. 0: I like this because it's much easier to edit IMHO
• eichin 4y ago
  heh. I use `chronic` all the time, `ifdata` in some scripts that predate Linux switching to `ip`. I occasionally use `sponge` for things but it's almost always an alternative to doing something correctly :-) Looking at the other comments, I suspect one of the difficulties in finding a new maintainer will be that lots of people use 2 or 3 commands from it, but nobody uses the same 2 or 3, and actually caring about all of them is a big stretch...
• dreamcompiler 4y ago
  A lot of these things -- and a lot of shell tools in general -- strike me as half-baked attempts to build monads for the Unix command line. No disrespect intended; nobody understood monads when Unix was invented. But it makes me wonder what a compositional pipe-ish set of command line tools would look like if it were architected with modern monad theory in mind.
• Brian_K_White 4y ago
  This comment is in praise of this idea I promise. I think I could pick apart about half of these and show how they aren't needed, and the showpiece example is particularly weak since sed can do that do that entire pipeline itself in one shot, especially any version with -i. You don't need either grep or sponge. Maybe sponge is still useful over simple shell redirection, but this example doesn't show it. One of the other comments here suggests that the real point of sponge vs '>' is that it doesn't clobber the outoutput until the input is all read. In that case maybe the problem is just that the description doesn't say anything about that. But even then there is still a problem, in that it should also stress that you must not unthinkingly do "sponge > file" because the > is done by the shell and not controlled by executable, and the shell may zero out the file immediately on parsing the line before any of the commands get to read it. This makes sponge prone to unpleasant surprise because it leads the user to think it prevents something it actually has no power to prevent. The user still has to operate their shell correctly to get the result they want, just like without sponge. So it's a footgun generator. Maybe it's still actually useful enough to be worth writing and existing, but just needs some better example to show what the point is. To me though, from what is shown, it just looks like an even worse example of the "useless use of cat" award, where you not only use cat for no reason, you also write a new cat for no reason and then use it for no reason. But there is still something here. Some of these sound either good or at least near some track to being good. I like it.
  • vinkelhake 4y ago
    Just a small comment about sponge: looking at the example, it's doing "sponge file", not "sponge > file". Given that, it's totally up to sponge to decide when it's going to open the output file.
    • Brian_K_White 4y ago
      I know that. That's exactly my point.
• adeltoso 4y ago
  Is this `parallel` a new tool? The well known "GNU Parallel" can cover all your parallelizing needs I would bet.
• LukeShu 4y ago
  Today the tagline is that moreutils is a "collection of the unix tools that nobody thought to write long ago when unix was young", but the original[1] tagline was that it was a "collection of the unix tools that nobody thought to write thirty years ago". Well, Joey started moreutils in 2006, so it's more than half way to that original "30 years ago" threshold! [1]: http://source.joeyh.branchable.com/?p=source.git;a=blob;f=moreutils.mdwn;hb=daaa73442e918c2fe0a3107ce64e529e74efb470 http://source.joeyh.branchable.com/?p=source.git;a=blob;f=mo...
• a9h74j 4y ago
  I don't know how to specify it, but it would surely be popular: oops
• thiht 4y ago
  Every time I stumble across moreutils, I can’t understand what any of its tool do. Take all these for instance: pee: tee standard input to pipes sponge: soak up standard input and write to a file ts: timestamp standard input vidir: edit a directory in your text editor vipe: insert a text editor into a pipe zrun: automatically uncompress arguments to command Wth do they do? What does any of this means? I know tee but have no idea what « tee stdin to pipes » would do
  • oftenwrong 4y ago
    You will want to read the man pages. 'pee' is straightforward to understand if you already understand 'tee'. 'pee' is used to pipe output from one command to two downstream commands. https://en.wikipedia.org/wiki/Tee_(command) https://en.wikipedia.org/wiki/Tee_(command)
    • thiht 4y ago
      I know tee but have no idea what pee would do considering this description. From your description I guess something like « pee a b » would be the same as « a | b »? If so that’s cool, but the one line descriptions definitely need a rework.
      • nerdponx 4y ago
        I believe it's more like `a | pee b x y z | c`. This would run `b x y x` as well as `c` using the output from `a`. I think Zsh supports this natively with its "multios" option, `a > >(b c y z) > >(c)`. But then you have to write the rest of your pipe inside the ().
      • somat 4y ago
        I can only guess from that description is that it is the cat command.
      • oftenwrong 4y ago
        It loosely follows the same metaphor as the T-shaped pipe connector that 'tee' is named for. $ seq 3 | tee somefile 1 2 3 ╔═somefile seq═tee═╣ ╚═stdout $ seq 10 | pee 'head -n 3' 'tail -n 3' 1 2 3 8 9 10 ╔═head═╗ seq═pee═╣ ╠═stdout ╚═tail═╝
• rurban 4y ago
  Most of them are nice to have, but they still ship an incompatible suboptimal parallel, which you explicitly have to check against in your configure, if you expect GNU parallel. Name it at least c-parallel
  • stingraycharles 4y ago
    Oh so that’s it! I use GNU parallel a lot, and installed moreutils yesterday, and parallel seemed to behave a bit… different. Couldn’t quite understand why, as I didn’t expect moreutils to replace parallel when you install them. I’m using Arch, btw.
    • rurban 4y ago
      They wanted to get rid of perl, so someone reimplemented 60% of it in C. Nobody continued with the rest, and so it's just annoyance.
    • kefyras 4y ago
      I'm sorry, what? First, moreutils package installs its parallel as parallel-moreutils. Second, pacman (like any other pm) wouldn't allow overwriting files belonging to other packages.
• opan 4y ago
  I was introduced to vidir through ranger's :bulkrename feature. Extremely handy. I don't think I've used the other stuff, but from reading the thread, vipe sounds great.
• hamasho 4y ago
  My favorite is `ts`! It adds a timestamp at the beginning of each input line. I often use it with tee to save the log output of any command. $ ping google.com | ts '[%Y%m%d-%H:%M:%.S]' | tee /tmp/ping.log [20220416-21:57:20.837983] PING google.com (172.217.175.78): 56 data bytes [20220416-21:57:20.838391] 64 bytes from 172.217.175.78: icmp_seq=0 ttl=53 time=6.028 ms [20220416-21:57:21.817189] 64 bytes from 172.217.175.78: icmp_seq=1 ttl=53 time=9.621 ms [20220416-21:57:22.818339] 64 bytes from 172.217.175.78: icmp_seq=2 ttl=53 time=9.443 ms [20220416-21:57:23.823126] 64 bytes from 172.217.175.78: icmp_seq=3 ttl=53 time=8.921 ms
• dmd 4y ago
  The absolute number one Unix tool that should have - and, more to the point, COULD have - been written long ago is mlr. https://github.com/johnkerl/miller https://github.com/johnkerl/miller "Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON." This should have been part of the standard unix toolkit for the last 40 years.
• spb 4y ago
  I tried to prototype my own implementation of `vidir` in Node.JS a while back (not realizing that it existed under this name in moreutils), and I ended up getting derailed after realizing how incomplete Node.JS's support was for getting the group / username corresponding to a G/UID: https://github.com/stuartpb/whomst https://github.com/stuartpb/whomst