7 ms·

There's a few programs I wish were standard on all Unix systems by now: * tree - print directory structure * bat - cat, but actually designed for reading file

• beojan 6y ago
  > rg Never going to happen because it's written in rust. ag (https://github.com/ggreer/the_silver_searcher https://github.com/ggreer/the_silver_searcher) on the other hand is written in C.
  • feanaro 6y ago
    What's never going to happen?
    • saagarjha 6y ago
      A Rust program being standard on all UNIX systems.
      • genuinebyte 6y ago
        I'll bite: Why?
        • saagarjha 6y ago
          Rust is designed for a fairly small number of relatively modern systems; there are many close-to-POSIX systems that can’t (and I would guess likely will never) be able to run Rust code in any formal capacity. To the hopeful: even C++ has been unable to win here, despite having many more years of a head start and much more accommodation for stranger platforms. It’s just not going to happen.
          • MaxBarraclough 6y ago
            What kind of platforms are you thinking of? People have got Rust running on the Raspberry Pi. https://old.reddit.com/r/rust/comments/cdcads/ https://old.reddit.com/r/rust/comments/cdcads/
            • saagarjha 6y ago
              Linux on ARM is a very mainstream platform from the perspective I’m looking at this from, I’m afraid. I’m talking about strange Unices running on architectures that GCC may or may not maintain a backend for, or maybe a BusyBox available on some debug interface, maybe a school project to build a simple POSIX-compatible OS or a bootstrap for a platform that had been recently jailbroken. In these cases C is almost always the go-to language and I suspect it will remain so for the foreseeable future as this is precisely the long tail of distributions that standards were intended to provide a base set of tooling for.
              • dividedbyzero 6y ago
                Could you give some actual examples? This sounds interesting but I'm not sure I've ever seen something like that in the wild.
                • hvdijk 6y ago
                  This is clearly not one of the examples saagarjha was thinking of, but an actual example where Rust is not an option is the x32 ABI for x86_64 platforms. This ABI works well with GCC, but LLVM suffers from multiple code generation bugs. As Rust is based on LLVM, Rust for the x32 ABI does not work well either. While it's possible to use a mixture of ABIs, so that you have a mostly-x32 system but with some x64 binaries, this requires a multilib setup with multiple copies of system libraries. A pure x32 system cannot currently have Rust utilities. (I have some patches to improve things but I have not been able to submit them to LLVM yet, and with those patches I did manage to get a working x32 build of rg on my system. I hope to be able to do so in the future.)
            • 6y ago
        • sannee 6y ago
          The Debian ripgrep binary is 5.1MB. That would consume over 15% of a 32MB flash, which is the common constraint you have to work with in many (actually, discounting Android, it might even be most) Linux deployments.
          • rcxdude 6y ago
            You're not likely to even find full-fat grep on such platforms (only busybox)
  • nindalf 6y ago
    Never say never :) If Rust being added to the Linux kernel[1] isn't far-fetched, I don't think adding a utility in Rust is crazy either. [1] - https://lore.kernel.org/lkml/CAKwvOdmuYc8rW_H4aQG4DsJzho=F+djd68fp7mzmBp3-wY--Uw@mail.gmail.com/T/#u https://lore.kernel.org/lkml/CAKwvOdmuYc8rW_H4aQG4DsJzho=F+d...
    • chalst 6y ago
      Interesting. Isn't it the case that LLVM doesn't support all architectures that Linux does?
      • saagarjha 6y ago
        I think this is just for kernel modules that you can choose to compile out.
        • geofft 6y ago
          That's what I expected too (as one of the people on the thread / one of the authors of the farthest-along set of bindings for modules), but Linus and Greg KH both seem fairly inclined for it to be on by default and used for core code (small bits of core code, in the beginning) and not simply modules. I'm not sure how that's going to play out with architecture support, and that's one of the things I want to get out of the Plumbers session this upcoming week. https://github.com/fishinabarrel/linux-kernel-module-rust/issues/112 https://github.com/fishinabarrel/linux-kernel-module-rust/is... is a chart of Linux architectures and whether Rust and LLVM support them.
          • saagarjha 6y ago
            Oh, wow, that would be huge. Not just architecture support, but even having a toolchain on some platforms might be problematic as I can imagine making rustc a requirement to build Linux to be fairly controversial in and of itself. And that architecture list is not looking very good…would be very interested to know what they decide on as well!
  • saagarjha 6y ago
    bat is Rust too.
  • isatty 6y ago
    +1 for ag (the_silver_searcher)
• wodenokoto 6y ago
  Good list of things you can install to better your CLI experience, but I am not sure if I agree that they fill out missing features. Why is bat better than less? I admit to catting files for reading as much as the next guy, but I almost always have the tiniest regret that I didn’t feed it into less and kept my terminal cleaner. As for RG and AG. other than being faster, how are they better? I thought they were api complainant drop-in replacements so it’s not like they fill in a missing role.
  • masklinn 6y ago
    > As for RG and AG I thought they were api complainant drop-in replacements To grep? Not that I know, I don't think either respects POSIX grep to say nothing of GNU grep. Though the trivial usage is identical, and many basic options carry over (e.g. -C and friends). > other than being faster, how are they better? Way better defaults for one. `rg` defaults to being recursive, ignoring binary files, respecting various types of ignore files, not being limited to BREs, coloring, … They also have useful convenience features like "file categories" (e.g. `-t` will expand to a bunch of predefined include glob patterns so you don't have to input them by hand, which can be tedious), or parallelism (recursive ripgrep will parallelise searches across files, with grep you have to remember to combine `find`, `xargs -P` and `grep` for that to happen).
  • burntsushi 6y ago
    > I thought they were api complainant drop-in replacements so it’s not like they fill in a missing role. Could you please let me know how you came to that understanding? Because I'd really like to fix it. ripgrep was never intended to be POSIX compatible. It would have been a straight-jacket over its functionality. See: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos... > As for RG and AG. other than being faster, how are they better? For ripgrep at least, a lot of it comes down to speed, better output formatting by default and suppressing results you probably don't want by default (but that can be turned off quite easily). But there are other features, like better Unicode support, built-in support for searching compressed files, support for file types and a few other things. The README includes a bit more. ;-)
• saagarjha 6y ago
  One of the reasons is Rust, but another reason is that the standard shied away from adding anything with frills because UNIX-likes run on all kinds of systems, many which don’t have space for this. There is a reason why something as seemingly ubiquitous as bash or vim are not available everywhere (but sh and vi are). The base set of utilities is really “do more with less” and the ability to keep to this philosophy means that I can connect to a router with 2 MB of MMC storage and still be able to have an extremely functional base set of tools, rather than having some arbitrary subset of the standard utilities because they just can’t be supported on platforms like that.
  • p00f 6y ago
    Exactly, I can telnet into my shitty Chinese router and still have cat and sh
  • rozab 6y ago
    My phrasing was a bit narrow-minded there, I meant consumer *nix distributions like Debian and what have you. These all manage to make room for Thunderbird and whatever awful music client they've come up with, but lack handy CLI tools.
• enedil 6y ago
  Why do you want automatic paging by bat, if you can just open it in Vim.
• pbhjpbhj 6y ago
  >cat, but actually designed for reading files. Syntax highlighting, line numbers, automatic paging. // `less` not good enough?
  • atsaloli 6y ago
    I’ve never seen syntax highlighting in “less”.
    • atsaloli 6y ago
      Ha! Here’s how to have syntax highlighting while using less: https://superuser.com/questions/71588/how-to-syntax-highlight-via-less https://superuser.com/questions/71588/how-to-syntax-highligh...
      • kps 6y ago
        While I don't use syntax highlighting, I find `lesspipe` useful for files that have a reasonable ‘presentation form’ as well as a source form — html, markdown, etc.
        • Gee19 6y ago
          bat is another great alternative https://github.com/sharkdp/bat https://github.com/sharkdp/bat
          • saagarjha 6y ago
            Mentioned at the top of this thread ;)
• krick 6y ago
  I don't think they should be standard at all. I am familiar with all of them pretty much since the moment they are born (which is not that long, TBH) and the only one I actually use is tree. Which is also quite a judgement call, since it can be replaced by exa, which is more powerful (I'm considering it; I'm pretty sure that you will like it, BTW, so take a look). bat is useless, since there is vim/view. rg is actually good, but cannot be used as a drop-in replacement for ag (I don't remember what the problem is exactly, but I tried several times and burntsushi even gave me some advice on that, and I still use ag, which is slower, so I assume there actually was some serious reason). And, by the way, no, neither of them is actually a replacement of grep. (Also, yeah, let's ignore that Rust in the standard libraries is still a problem.)
• enriquto 6y ago
  > tree I agree, tree is great. It can be somehow replicated by using "find .", if you are in a hurry. > bat If you need that functionality, why don't you open the file with vim, for example? > rg - better grep Grep is pretty nifty, I don't see how could it be improved. What is the main advantage of of rg over grep? > direnv - local environment variables I read the manpage for direnv and I was really scared. What is it for? I never used an .envr. If I need to see my environment I can simply use "env". What is it missing?
  • gpanders 6y ago
    Re: bat. I agree with you, I don’t get the hype over this program either. It’s pretty easy to configure 'less’ to use syntax highlighting and ‘less’ is installed basically everywhere. The main advantage of rg is how much faster it is than grep. It’s WAY faster. I don’t use direnv but I have colleagues who do. I think the idea is that it can set environment variables/run arbitrary commands when you cd into a certain directory (such as activating a Python virtual env, for instance).
    • haberman 6y ago
      I also like that rg's default mode is to search all files recursively, skipping binary files and honoring .gitignore. 90% of the time this is just what I want.
  • breuleux 6y ago
    direnv is super useful if you're working in Python and have multiple projects: you can add an .envrc file that activates the right virtualenv whenever you enter a directory. It's one less thing to worry about. I figure you can also use it in conjunction with `module load` to automatically load the right module when you enter a project's directory. Modules are used in many computing clusters to manage environments. Also direnv will never execute an .envrc without permission, you need to run `direnv allow` whenever there is a new or newly modified .envrc.
    • flukus 6y ago
• kps 6y ago
  A few that I wish hadn't been ignored from 4BSD: jot -- print sequential or random data rs -- reshape a data array vis -- display non-printable characters in a visual format and from Unix >V7: apply -- apply a command to a set of arguments mc -- multicolumn print and from AT&T: tw -- file tree walk And then the obligatory personal tools that have stood the test of time: align -- align text columns crop -- crop lines to a width dabl -- delete adjacent blank lines dtb -- delete trailing blanks emboxxen / deboxxen -- convert to/from box drawing characters eol -- convert line endings field -- simple line field extraction freeze / thaw -- cross-shell synchronization mergl -- merge lines into previous lines' blanks pad -- pad lines to a width put / take -- cross-shell pipe uni -- unicode character properties and search
  • vram22 6y ago
    Some of the above can easily be written in C or Python or Ruby (and some of the simpler ones even in shell), at least for simple versions, maybe without all the frills of the originals. And seq in Linux is like jot for sequential data, at least. And from my blog: An Unix seq-like utility in Python: https://jugad2.blogspot.com/2017/01/an-unix-seq-like-utility-in-python.html?m=0 https://jugad2.blogspot.com/2017/01/an-unix-seq-like-utility...
    • kps 6y ago
      Yes, they're generally available in some form (now generally including the ‘originals’ now that V8–V10 have been published). And they're all very simple, which is the whole point of Unix tools. I've found I'm using the `put`/`take` pair, for splitting a pipeline across shells, more again in the WFH era where I often have multiple ssh sessions into a machine, when I would probably have used the clipboard in a GUI session. Also useful for things like seeing stderr from different parts of a pipeline in different windows. `put` is just `cat >>$(mkpipe "$1")` and `take` is `cat $(mkpipe "$1")` where `mkpipe` is fifo="${TMPDIR:-/tmp}/fifo-$(id -u)-$1}" test -p "$fifo" || mkfifo "$fifo" echo "$fifo"
      • vram22 6y ago