14 ms·

12 Factor CLI Apps

524 points by dickeytk 8y ago

willio58 8y ago
Just the help factor alone is a big one.
davemp 8y ago
In regards to 7, prompts are great for teaching new users how the program should be used. Instead of failing then spitting out --help or manpage style info, the program just ask the user enter the needed argument or flag to continue. Having more ways to learn usage is always good IMO.
- dickeytk 8y ago
  yep +1. A lot of times users are only ever going to run a command once. Better to ask for the right information than bailing out because it's not perfect syntax.
woodruffw 8y ago
> The user may have reasons for just not wanting this fancy output. Respect this if TERM=dumb or if they specify --no-color. Or if they specify `NO_COLOR`[1]! [1]: https://no-color.org/ https://no-color.org/
- dickeytk 8y ago
  yes, adding this
  - deleted 8y ago
    [deleted]
drewmassey 8y ago
I’m a huge admirer or well crafted cli apps, they can massively boost the effectiveness of a team. An oldie but goodie here: https://eng.localytics.com/exploring-cli-best-practices/ https://eng.localytics.com/exploring-cli-best-practices/
stepvhen 8y ago
> I would skip man pages are they just aren’t used that often anymore. I understand that man pages might represent a minority, but I cannot express enough how wonderful it is to get the full manual of a program without interfacing with the web. Not to mention how powerful that is, since most apps have short names that are difficult to search for, but how accessible that makes the application.
- norova 8y ago
  I agree that man pages are wonderful. This point in the article really irked me.
- dickeytk 8y ago
  The fact they don't run on windows means some subset of your users cannot even use them if they wanted to. Better to spend your time on something they can all read. I'm not saying they're not useful. If you've got plenty of time to write up docs, go ahead, but the reality is we only have so much time and I think we should spend our time writing in-CLI docs and web docs before we start man pages. Also, you don't need web access to use in-CLI docs either, and that works on all platforms. Having said this, I do plan on having man pages be an export type of the oclif docs (which is currently in-CLI and markdown). I intentionally made the output very similar to man pages already so it should be relatively easy to do.
  - deleted 8y ago
    [deleted]
  - therealjumbo 8y ago
    For man pages you could suggest writing markdown and using a build process to automatically generate man pages [1] in the event they aren't using oclif. EDIT: I think we've all been in areas without network access like on a plane and not having a man page in that scenario is very annoying. Also, you briefly say a few things about CLI apps using a remote API, you may want to add to that and say a few things about the proxy environment variables [2]. These are indispensible for corporate users. I think some early, early version of npm didn't respect the no_proxy environment variable, and for the http_proxy and https_proxy it required some arcane combination of: proxy in a flag, proxy in a config file, proxy environment variable set. It really should be an OR not an AND... Last but not least, another annoying thing was tools changing their config format or location. I think it was docker that changed their config file format and/or location like two or three times. Absolutely infuriating. 1. https://rtomayko.github.io/ronn/ronn.1.html https://rtomayko.github.io/ronn/ronn.1.html 2. https://wiki.archlinux.org/index.php/proxy_settings https://wiki.archlinux.org/index.php/proxy_settings
013a 8y ago
This is all great advice. The one thing this does miss is distribution, which is a HUGE part of offering a great CLI app. Specifically, I'd say: 1. Make your OFFICIAL distribution channel the primary package manager on each platform (ex: on Mac, homebrew. Ubuntu, apt/snap). Beyond that, support as many as you have capacity to. 2. Also offer an official docker image which fully encapsulates the CLI tool and all of its dependencies. This can be a great way to get a CLI tool loaded into a bespoke CI environment.
- dickeytk 8y ago
  yeah I agree 100%. Distribution of CLIs is a headache for the maintainer, but it's really important to get right if you want users to have a seamless experience.
- curun1r 8y ago
  Homebrew is NOT the primary package manager on Mac and I wish people would stop perpetuating that falsehood. Apple includes pkgutil/pkgbuild in the OS and that official package management strategy plays much better with corporate IT control of managed machines. In my experience, Homebrew always eventually results in pain and complex debugging and it's almost impossible to audit software it installs to prevent the installation of prohibited or dangerous software. It's really not that hard to build a .pkg file and developers that want to properly support the Mac platform should go down that path before offering Homebrew support.
  - deathanatos 8y ago
    Well, I'll admit this is the first I'm learning of this tool. That said… The top hit on Google for "os x pkgbuild" is a link into Apple's documentation that 404s. (Further Googling turns up some blog posts, and man pages, so that's good.) Does this support dependencies? How do I get updates to users? How does a user receive updates?
    - curun1r 8y ago
      Wow...I'm not sure why Apple has taken down it's documentation for the command-line tools. It's been so long since I looked at anything beyond the man pages, I didn't realize the web docs weren't up. If I had to hazard a guess, I'd bet they're trying to move developers towards an XCode-centric approach. But there are plenty of other tools that can generate that format too. And it's the only format that's natively understood by the OS, so I still stand by my assertion that Homebrew is not the official package manager for Mac.
sigjuice 8y ago
Nitpicks: Replace “pipe content” with “redirect content”
- dickeytk 8y ago
  nice catch, ty
breckuh 8y ago
> 1. Great help is essential I like how this is their #1. In my opinion the best way to do this is with tldr. https://github.com/tldr-pages/tldr https://github.com/tldr-pages/tldr I'd highly recommend folks create a tldr page for their CLI app. Add 4-8 examples to cover 80%+ of the most common use cases. -h flags, readmes & man pages can cover the other 20%.
- dickeytk 8y ago
  I almost want to rewrite the help section to encourage examples even more. They're incredibly valuable. I hadn't considered this before you mentioned it, but oclif CLIs could integrate to tldr pretty well. It already supports arrays of strings for examples.
  - tetha 8y ago
    Yup. On CPAN, it is encouraged that the first part of your documentation after the table of contents is the synopsis[1]. The synopsis should clearly show how to do the common tricks with the library. From there you can link and refer to the more detailed documentation. We're doing that for our internal CLI applications and it's great to be able to just copy-paste the common use case from the top of the documentation without searching much. 1: https://metacpan.org/pod/Carp https://metacpan.org/pod/Carp
    - dickeytk 8y ago
      I feel the synopsis section of man pages often just becomes a bunch of useless garbage above the fold (for instance, look at `man git`). Using it less as a complete docopts kind of thing and more of multiple common usages (like `man tar` and what you linked) is far more useful. I think there is something here I hadn't really considered before. It's not an example, but also not a useless dump of flags. Food for thought I suppose.
  - Riverheart 8y ago
    Agreed on making examples more important. Powershell, for example, let's you "get-help command -examples" to just retrieve those.
kpcyrd 8y ago
I disagree on the 2nd point. Flags prevent globbing by the shell and make the help text less clear. Consider a usage line like this: prog <user> [password] This tells you which argument is mandatory and which argument is optional in a second, without searching for the help text of --user and --password. Also, an example like this: git add <pathspec>... Tells you that git add accepts multiple paths and you can invoke it with: git add src/* What's more important in my opinion is making sure your argument parser can handle values that start with a dash and respects a double dash to stop the option parser. Consider an interface like this: prog [--rm] [--name <name>] [args...] And you invoke it like this: prog --name --rm -- --name foo This should result in: { "name": "--rm", "args: ["--name", "foo"] } Getting things like this wrong can result in security issues.
- dickeytk 8y ago
  I may try to expand on this in the article, but it's in there if you read between the lines. There is a difference between something that takes in multiple args of the same type and multiple TYPES of args. I'm arguing against multiple args of different types, not the same. By definition any CLI that accepts variable args is fine here as it's all the same type. The -- is a great point as well. It solves a lot of problems users have but a lot of time people don't even know about it. It solves issues with `heroku run` for example. EDIT: updated to clarify my point
- chriswarbo 8y ago
  I don't like using arguments for key/value data, e.g. `prog --name foo`. Much better to use env vars like `NAME=foo prog`, since the OS handles the mapping automatically, reading them out is trivial in every language, passing args to subprocesses is easier, etc.
ISO-morphism 8y ago
These are all good points, and I wish more clis were like this. My own pet peeve is un-disablable stdout logging. > It’s important that each row of your output is a single ‘entry’ of data. It felt weird to me to use `ls` as an example as it's not immediately obvious it adheres to the advice from the printed output. I suppose they were also trying to highlight the earlier point of differing output format depending on whether output is a tty/pipe. Unrelated, but I didn't know `ls` was that smart about isatty. Once upon a time I read the '-1' option to print one name per line in the man page and assumed it was necessary for that functionality. Thanks!
- dickeytk 8y ago
  I just picked `ls` as it's a common utility everyone understands and isn't some contrived example using `cat`. I am going to add a note about `ls`'s behavior with isatty. It's sort of conflating a couple of things, but I think it's interesting enough to leave it in.
  - ISO-morphism 8y ago
    I agree, it is a good example, and hey I learned something. I really appreciate that you're taking the time to respond to all the feedback in this thread and wade through everyone's nitpicks. Looking forward to more articles.
    - dickeytk 8y ago
      Thank you for the great feedback!
- dotancohen 8y ago
  > My own pet peeve is un-disablable stdout logging. I really think that we need a stdmeta file descriptor. See my comments at the link below, I would appreciate your feedback: https://unix.stackexchange.com/questions/197809/propose-additional-file-descriptor-stdmeta https://unix.stackexchange.com/questions/197809/propose-addi...
nimish 8y ago
#6 is great, except if it causes performance issues: https://github.com/npm/npm/issues/11283 https://github.com/npm/npm/issues/11283 Speed is the ultimate fancy enhancement ;)
- roylez 8y ago
  Even if it does not cause any performance issues, I dislike it. If the thing runs in terminal, it should be expect to be used in a script thus it would be better to make no assumptions of terminal capabilities and leave the fancy part to external tools, if one is interested. I always hate systemctl's piping to a pager by default. "Do one thing, do it well", don't try to surprise users with fanciness because at work we don't like surprises.
  - dickeytk 8y ago
    In practice I've had overwhelming feedback praising our use of spinners in the Heroku CLI and not a single complaint I can think of. In fact, I've had more praise for adding spinners and color than any other change we've put in over 4+ years of development. That said, you need to be careful. Don't use a spinner if it's not a tty or TERM=dumb. Do use it in some CI environments that support it (Travis, CircleCI) That handles all the issues we've seen and everyone seems to be happy.
- OptionX 8y ago
  And more than that can cause compatibility issues. For example, its a bit of a pain to use utf-8 in the normal windows cmd.
- AgentME 8y ago
  Wasn't that issue fixed?
emmanueloga_ 8y ago
Don't get me wrong! I love command line apps. But I wonder if we all have a bit of an Stockholm syndrome... there are several things that suck about them... While writing this I'm thinking on my experience trying to do anything with ffmpeg or imagemagick... or even find. * For any sufficiently complicated cmd line app, the list of arguments can be huge and the --help so terse as to be become useless. For man pages, the problem is the opposite... the forest hides the tree! I'm sure we all end up using google to look for example invocations. * Very often completion doesn't work, since custom per-app machinery is needed. For instance: git-completion for with bash-completion. * Sometimes I end up passing the help output through grep, then copy-pasting the flags from the output, and then hoping I got the right flag. * ...how about things like regular expressions parameters... always so hard to remember the escaping rules! (and the regex flavor accepted by each different app). * Not to talk about more complicated setups involved -print0 parameters or anything involving xargs, tee, and formatting with sed and cut, etc. Is there a better way? Not sure. I like powershell a bit but some of the things I mention above still apply. I think we may be able to get a workflow that is a bit closer to the tooling we use for writing programs while not being perceived as verbose and heavy (I'm thinking, the kind of workflow I get with a Clojure repl).
- NeedMoreTea 8y ago
  For a quick args reminder, without having to dig through man pages, try tldr: https://tldr.sh/ https://tldr.sh/
- forkerenok 8y ago
  Speaking for myself of ffmpeg and imagemagick, I don't find the CLI itself difficult, I find that their application domain is a bit complicated for a layman. Meaning I usually struggle to comprehend what particular options mean rather than how they bind to CLI. Maybe indeed a good intuitive UI would give more intuition about the more obscure options. But then again, a good intuitive UI is a quest of its own.
  - tajen 8y ago
    What is difficult with ffmpeg is the sub-language in each argument, which depends on the filter you’re using. I can learn to lookup the manual, but if I have to go to the internet to learn the syntax of the filter, I’m better off googling my usecas.
henkdevries 8y ago
Man I wish OpenVMS was still a thing. All the commands worked the same due to the DCL enforcing it. https://en.m.wikipedia.org/wiki/DIGITAL_Command_Language https://en.m.wikipedia.org/wiki/DIGITAL_Command_Language
- jwilk 8y ago
  Non-mobile link: https://en.wikipedia.org/wiki/DIGITAL_Command_Language https://en.wikipedia.org/wiki/DIGITAL_Command_Language
- stunthamsterio 8y ago
  Me and you both. OpenVMS got so much right back in the day. Good error conventions, versioned file system... I still fondly run an OpenVMS workstation under my desk so I can revisit the good old days sometimes. Then realise my skills have atrophied to the point I can barely remember how to use edit.
- bigpicture 8y ago
  A) PowerShell was inspired by OpenVMS DCL. B) OpenVMS on x86 is due to be released in 2019 (it's in private beta right now).
OJFord 8y ago
> 12. Follow XDG-spec I'm so glad to see this included. I don't like $HOME being cluttered with .<app> config directories, but worse than that, far too many when releasing on macOS say Oh Library/Application\ Support/<app>/vom/something is the standard config location on Mac, so I'll respect XDG on Linux but on Mac it should go there. No! Such an unfriendly location for editable config files.
- dickeytk 8y ago
  I am having an impossible time verifying this, but I recall reading that MacOS deletes unaccessed files eventually from `~/Library/Caches/*`. Which would be a compelling reason to use that for cache. (Not being able to verify this I didn't add it to the article) If anyone can verify that I'm either right or wrong here that would be helpful.
  - zbentley 8y ago
    OSX does not automatically delete from that directory. See: https://developer.apple.com/library/archive/documentation/General/Conceptual/MOSXAppProgrammingGuide/AppRuntime/AppRuntime.html https://developer.apple.com/library/archive/documentation/Ge... > Your app is responsible for cleaning out cache data files when they are no longer needed. The system does not delete files from this directory. However, many third-party tools delete from that directory with minimal caution if any, so it's a good idea to consider it ephemeral.
    - dickeytk 8y ago
      appreciate the clarification!
- andreareina 8y ago
  I agree that seeing a bunch of ~/.<app> directories is annoying, but at the same time I do think that it makes sense for each application to manage its own hierarchy, rooted under e.g. ~/.apps/<app> instead of splitting it into ~/.config/<app>, ~/.local/share/<app>, etc. Regardless, I think it probably makes sense to have a uniform interface for getting said directories, so that however the OS decides things should be laid out, the developer just needs to `local_config_dir(app_name)`. If the user (or at least administrator) can decide between <app>/<function> and <function>/<app>, all the better.
keithnz 8y ago
While a bit quirky, I really like powershells approach where you aren't limited to text streams. All the same advice applies.
jstanley 8y ago
> Error: EPERM - Invalid permissions on myfile.out > Cannot write to myfile.out, file does not have write permissions > Fix with: chmod +w myfile.out I actually much prefer: "can't write myfile.out: Permission denied" This shows the same information as the first 2 lines combined from the example, and the 3rd line is not necessarily the correct way to fix the problem anyway (e.g. you might be running it as your user when it should be root, chmod +w would not help). If you are so convinced that chmod +w is the way to fix the problem, why not just do that and carry on without bugging the user? And having each error confined to one line also means it's much less likely that some of the lines are missed, e.g. when grepping a log. EDIT: And to add to this: it's sometimes useful to prefix the error messages with the name of the program that generated them, so that when you're looking at a combined log from different places, you know which of the programs actually wrote the error, e.g. "mycli: can't write myfile.out: Permission denied".
- Beowolve 8y ago
  So, I understand the basis of your comment. You have the knowledge to know that there are other things that may be "the right way" given your situation. I think what the author is getting at is that there are users who don't have that knowledge. Giving them a hint that is verbose and non arcane can make a world of difference. Speaking from personal experience, there are many developers that I have met who don't have basic *nix knowledge, much less knowledge of a terminal. The reality of the situation is that a business is going to hire people regardless of that ability. They want someone who can move the features out the door. Whether this is good or bad is probably beyond this conversation. I think, for those users, these sorts of helpful hints are extremely important because it makes them feel like they aren't stuck and helpless. I think that, to your point, it may be useful to have the CLI offer a "pro" mode in which you could set a config to not give you as verbose error messages. Annoying? Yes. However, it would strike a balance and serve both needs.
  - jstanley 8y ago
    I understand what you're saying, but this conversation is about the best way to design CLIs. I agree that you can do lots of stuff suboptimally and still have a usable tool, but I disagree with the author on what the ideal error output looks like.
hornetblack 8y ago
Also on Color. Don't got go all pschadelic. I've found that some programs using 256-color is unreadable with my terminal colors. Also unix commands tend to have illegible colors in Powershell on Windows. (Ripgrep for example). Powershell defaults to a blue background.
- dickeytk 8y ago
  Yeah if I get some time I should expand on that. There are some colors that don't work well at all for common scenarios (solarized, windows). Some of them we've blacklisted and do not use. Also for colors it's good to try to pick a decent palette of a couple-three colors and stick with it rather than try to categorize a bunch of different bits of output on a single command. Still, colors are awesome and make your CLI look and feel 10x better than it is, so it's worth the extra effort.
  - gvalkov 8y ago
    A problem with color is that people end up optimizing the aesthetics for their own terminal setup. There is a wild number of different color schemes out there and it's really hard to make something that looks good on all of them. In my experience the only safe choice is bold text (i.e. \033[1m) since it stands out in all cases.
    - dickeytk 8y ago
      red, yellow, green, cyan, and magenta are pretty safe. We've had complaints about others, but these are pretty reliable in my experience. dim is safe too, but doesn't work all the time. (Not working meaning just not dimmed)
      - roryokane 8y ago
        I would definitely not call yellow a safe color. I use a terminal with a white background, and yellow text is unreadable unless the text is also given a custom background color.
- 8y ago
gnomewascool 8y ago
I almost stopped reading at "I would skip man pages", but the rest of the article was mostly great advice. I disagree about 11 (using "main_command sub_command:sub-sub_command" rather than "main sub sub-sub" syntax), but it's mostly a matter of taste. Seriously, though, if you've already taken the time to write documentation, then there's no reason not to also generate a manpage. Just using pandoc to convert your, say, README.md gives good-enough results: pandoc -s -f markdown_github -t man -o your_cli.1 README.md (There probably are other good conversion methods.) Why I like man: Advantage over online docs: It's offline and available directly in the terminal, without having to open a browser and it has a distraction-free, clean look. The only slight disadvantage is the lack of support for images, which are occasionally helpful, but in a pinch, for some use-cases, you can have ascii diagrams. Advantages over "--help": 1. Conventionally, "--help" just provides a brief rundown/reminder of the options, so having full documentation is valuable. 2. If "--help" provides the full docs then: a) You lose the option of having the brief rundown, which is also very valuable. b) "man command" is slightly faster than "command --help" :p (yes, it is a slight pity that accessing the full docs is faster than accessing the brief version, if you use convention). c) man deals with things like having nice output, with proper margins, at different terminal widths. d) man deals with the formatting for you, providing consistency with all other applications. FWIW I think that texinfo is (mostly) even better than man, as it considerably improves on the navigation, but it's been crippled by the FSF-Debian GFDL feud, which meant that the info pages weren't actually installed on many systems, and it's mostly a lost cause now.
- enriquto 8y ago
  > Advantages of man over "--help": You can have the best both worlds if the manpages are built automatically from the "--help" output (e.g., using help2man). Then you can have "-h" give a brief rundown and "--help" give the full docs. > FWIW I think that texinfo is (mostly) even better than man, as it considerably improves on the navigation I am curious about that. Do you really like texinfo navigation? I find it completely unusable, to the point of prefering to download and print a pdf from the web instead of opening (gasp!) the dreaded "info" program.
  - 8y ago
sytelus 8y ago
Does these CLI features like tables, OS notifications etc work cross-platform (Linux, Windows, OSX)? IS there any good library to develop such CLI apps?
- dickeytk 8y ago
  we use https://github.com/mikaelbr/node-notifier https://github.com/mikaelbr/node-notifier
sudofail 8y ago
I'm probably being picky, but I would also include that the CLI be a self-contained binary. I'm tired of managing Python / Ruby / Node versions.
- nhumrich 8y ago
  Its bad both ways. The advantage to python/ruby for example is you can simply pip/gem install, or update. With a binary, you have to download, move, and change permissions every update. For experienced linux users, the binary is fine, but for newer users, its much more "friction"
  - Sean1708 8y ago
    IMO using a language's package manager to install applications is a massive anti-pattern, that should be handled by your OS package manager.
    - sudofail 8y ago
      This is what I prefer as well. Let me use my OS's package manager for managing my packages.
      - mixmastamyk 8y ago
        That would still be the case if the packages weren't 1-3 years out of date.
        jetblackio 8y ago
        That is true. But there are ways around that. Including documentation on how to build it locally is pretty standard. And hosting prebuilt binaries with package installation for targeted platforms is also pretty common as well. With interpreted languages with language-specific package managers, you have to: 1) Install the language 1a) Possibly have to install a language version manager (rbenv, pyenv, etc) 2) Install the language's package manager 3) Install the CLI utility via the language's package manager Here's the order I think CLI maintainers should strive to making their utilities available: 1) Install via OS package manager 2) Install via prebuilt release with OS-specific package, from hosting site (GitHub, etc). 3) Install from source 4) Install via language-specific package manager 5) Install via curl | sh :)
Walkman 8y ago
Unless you already know your users will want man pages, I wouldn’t bother also outputting them as they just aren’t used often enough anymore. I don't know where this is coming from, me and my colleagues are reading man pages every day. I would be interested how much others read them.
- kolme 8y ago
  I strongly disagree on this one too. That's the first place I look for help and it annoys me to no end when a CLI program that doesn't come with one. I stopped taking seriously the article at that point and quickly skimmed through the rest of it. Man pages are a great unix culture heritage, please new developers don't give up on them!
  - davidhcefx 8y ago
    I myself also love reading man pages, but speaking of compatibility, I have to say that “--help” is a more universal way of showing help pages. Of course it’s better to have both of them though.
    - larkeith 8y ago
      --help is fine, but almost never a substitute for a full man page, except for the most trivial of applications (unless your --help is as complete as a man page, in which case... good on you for providing full documentation, but I'll hate you a bit every time I unthinkingly drop two hundred lines of text in my terminal.)
      - smarinov 8y ago
        Unless it just opens the man page if it is so long. Like git.
    - JdeBP 8y ago
      I strongly suspect that --help is on the contrary less universal, given that there exist entire toolsets lacking the --help convention but having manual pages. There was almost a quarter of a century's worth of Unix tools that developed before --help was invented. * https://unix.stackexchange.com/a/416796/5132 https://unix.stackexchange.com/a/416796/5132 * https://unix.stackexchange.com/questions/207136/ https://unix.stackexchange.com/questions/207136/
bayindirh 8y ago
I don't agree with 7 and 8. I like silent apps while working, and actually I'm used to applications saying nothing if everything is correct. Also, "outputting something to stdout just because I can" kills scriptability a lot. Using tables, colors and other stuff requires a lot of terminal support. MacOS terminal, iTerm, Linux terminals supports a lot of stuff, but not always (our team is generally using XTerm for example). Implementing these are acceptable if there's a robust code detecting terminal capabilities and falling back gracefully and without treating these more streamlined terminals as lesser citizens, and this requires a lot of development, head banging and maintenance. If you're accepting the challenge, then go on. BTW, That unicode spinner is nice. Very nice.
- oblio 8y ago
  You can just have a "quiet" mode for scripting. Or even better, detect if you're connected to a TTY.
  - bayindirh 8y ago
    Checking whether you're connected to a TTY is a good practice, but there's also cognitive advantages of silent applications too. Also, similarly you can have a -v --verbose set to make the application talkative. It's also an option.
  - scbrg 8y ago
    I run scripts from a tty all the time...
- Sean1708 8y ago
  > Also, "outputting something to stdout just because I can" kills scriptability a lot. The article does specifically recommend using stderr for messages to the user, is there any reason that would kill scriptability?
  - bayindirh 8y ago
    I missed that. It's actually a much bigger problem. stdout and stderr(or) is two output paths to explicitly diverge error logs and problems from standard output. stdout / stdin is reserved for user interaction, informational messages (md5sum), actual output (ls, less, etc.), and the like. OTOH, stderr is explicitly for error messages only, and it's very useful (and necessary). A real use case from my daily job: I'm an administrator of a large system (approx ~1K servers), and I have substantial amount of cattle, and a lot of pets [0]. All of these servers have cron jobs, and other automated tasks on them. All servers can mail to a local mail server to report us problems, so our cron automatically mails any outputs to us. All the tools we use, and scripts we write have the following properties: - If everything is OK, they are silent by default. - They output to stderr, if anything notable happens. - Also we copy (think tee) all stderr to their respective logs (both local and on a centralized server). Now consider: - The (e-mail, log) noise if all the tools were writing their outputs to stdout. - The work required to silence all tools. What if they don't have any --quiet switches? - Furthermore, they wrote everything to stderr. How can I know whether thing has worked as it should? - How can I find the problem quickly if everything is written to "Error" log? - Furthermore how can I revise the errors happened quickly? Info & Error on the same file. grep galore! - How can I silent a tool (by redirecting to /dev/null) if both normal and error logs are written via stderr? These are the simple problems that I can come up on a real, big production system in five minutes. I can find more problematic scenarios which will happen on a daily basis, if I think more. All these conventions, folklore, recommended usage and facilities are in place because of the needs and the experience acquired in the history of these systems. Running around them amok, just because they enable color, pretty spinners, or justify some narrow usage scenario is not correct. These conventions and philosophy [1] allowed *NIX systems to scale without needing excessive administrative elbow grease. Ignoring these, and developing tools which use facilities and conventions as they please will degrade the ability to manage these systems with minimal work. I can always grep, but it will be inefficient and not guaranteed to get everything that I want, and also it will cost me and the computer time to do so. Like algorithms, systems are easier to manage when "n" is small. When "n" gets big, these tasks got really hard, really fast. So, develop tools to ease tasks. Not to show-off. [0] https://www.engineyard.com/blog/pets-vs-cattle https://www.engineyard.com/blog/pets-vs-cattle [1] https://en.wikipedia.org/wiki/Unix_philosophy https://en.wikipedia.org/wiki/Unix_philosophy Edit: Tried to increase readability.
O_H_E 8y ago
> Follow XDG-spec Oh, please guys. Some apps even put visible (non .) folders in my home.
justinrlle 8y ago
> I also suggest sending the version string as the User-Agent so you can debug server-side issues. (Assuming your CLI uses an API of some sort) Isn't it some kind of disguised tracking? I know it doesn't give as much info as the user agent of a browser, but still, you could track the OS, even the linux distribution, and surely more, while still being a reproducible build.
czechdeveloper 8y ago
I have recently created single CLI program to put in all actions I need to automate. It's so fast to make new action, that I make anything that saves me just few seconds a day. Even stuff like "invoice" will open me timescheduling app, invoicing app and creates canned email to clipboard. "Clockout" will open timescheduling app and copy expected date and times to clipboard just to paste to app. I've of course spent some time on automating Help, flags parsing etc, so I essentially just say what data I need and then what to perform. It was best idea in a long time. I'm thinking that I'll publish framework for this as opensource (it's C# project).
superlevure 8y ago
Does someone has the name of the terminal app used in the article ? (with the nice colored path)
- kjaer 8y ago
  It looks like zsh with the Oh My Zsh [1] with the Agnoster theme [2]. 1. https://ohmyz.sh/ https://ohmyz.sh/ 2. https://github.com/robbyrussell/oh-my-zsh/wiki/Themes#agnoster https://github.com/robbyrussell/oh-my-zsh/wiki/Themes#agnost...
  - superlevure 8y ago
    Thank you !
phoe-krk 8y ago
I clicked this in hope that these was an article describing 12 CLI apps written in the Factor language. http://factorcode.org/ http://factorcode.org/ My hopes were crushed.
JepZ 8y ago
> 7. Prompt if you can Please don't. There is nothing wrong with interactive tools, but by default, they should not be. So instead of making non-interactive session possible via flags, the default should be to be non-interactive. If there is an option to start an interactive session, everything is fine. Otherwise, you would never know when your script could run into some kind of interactive session (and therefore break; possibly after an update).
- jessaustin 8y ago
  My interpretation was that the prompt is for required information. In the example graphic, "run demo" really does require that "stage" be specified. This is considered more user-friendly than simply crashing. If you don't want to see the prompt, provide that information as a flag or in a config file or whatever.
  - dickeytk 8y ago
    Correct, and if stdin is not a tty it should error out instead of prompting
    - iainmerrick 8y ago
      The one thing missing from the example is a little more contextual help. Not only should it prompt you for the stage, it should say “use --stage [development|staging|production] on the command line to skip this prompt” or some such. (Could be as terse as the prompt reading “please specify --stage”.)
      - dickeytk 8y ago
        Like for the confirmation example? I agree. It would help clear this up. The points people are raising here with prompting are definitely not issues, they're just misunderstanding my point.
        jessaustin 8y ago
jessaustin 8y ago
...all of these must show help. $ mycli Some commands have an unambiguous meaning and don't need arguments. For example, it would be weird if a bare "make" command returned help information. Great post, though. I'm looking forward to digging into oclif.
stuaxo 8y ago
"12 factor" anything seems to be a symptom of the over-complexity of modern apps, go back and rethink.
- shoo 8y ago
  i found the original 12 factor website had a number of pretty reasonable suggestions based on experience of people who ran a business doing operations for other people's web apps. sure, web apps are themselves probably over complicated, but given that you're doing a web app, the recommendations arent bad. compare to where things have gone since, with containerisation.
  - subway 8y ago
    In a lot of ways the "12 factors" have overly simplistic views of the world. For instance, storing your config in the environment is fantastic -- until you remember a great many frameworks will dump their environment to the browser in a number of failure scenarios. There go all your secrets.
gtramont 8y ago
Related: http://docopt.org/ http://docopt.org/ – There are implementations for various languages. Whenever I need to write something that has a CLI, this is my default option…
fiatjaf 8y ago
Is it just me or every now and then Medium opens with what looks like to be a snapshot of the article instead of the HTML? I can't select text or scroll the page. If I refresh the page everything is normal, though.
rdsubhas 8y ago
Really great advice here. But missing one key area: Continuous Delivery / Change Management. I believe any policy without change management principles isn't really complete, especially when its about 12factor which is considered a gold standard for production. * CLIs are notoriously difficult to update because you have to convince every single consumer to update it manually, otherwise you just have scattered logic everywhere. Having an update workflow is essential before releasing the first version in production. * Closely tied, a clear Backwards compatibility policy. Apart from those two major items, I have also found one optionally nice pattern to reason about CLIs: Design CLIs like APIs wherever possible. Treat subcommands as paths, arguments as identifiers, and flags as query/post parameters. It's not always applicable, but doing that for large internal tools helps against the "kitchen sink" syndrome.
kurtisc 8y ago
>Still, you need to be able to fall back and know when to fall back to more basic behavior. If the user’s stdout isn’t connected to a tty (usually this means their piping to a file), then don’t display colors on stdout. (likewise with stderr) GCC does this, leading to no colour output where it would be useful if you're building with Google's Ninja-build. Maybe there are some people who do pipe GCC output to a file - I've never had to. If you do this with your app, I'd appreciate being able to re-enable the colour.
twic 8y ago
> 8. Use tables > By keeping each row to a single entry, you can do things like pipe to wc to get the count of lines, or grep to filter each line > Allow output in csv or json. Yes please. Default to readable-but-shellable tabular output, and support other formats. libxo from the BSD world is a really smart idea - it provides an API that programs can use to emit data, with implementations for text, XML, JSON, and HTML: http://juniper.github.io/libxo/libxo-manual.html http://juniper.github.io/libxo/libxo-manual.html I personally love CSV output. Something like libxo means that CSV output could be added to every program in the system in one fell swoop.
lousyd 8y ago
Only a web programmer could believe that man pages "just aren't used that often". It drives me nuts when compiled cli programs don't have a man page. It says to me that the author of the program doesn't know Unix conventions or doesn't care enough to put the effort into meeting his or her users where they are, and so I'm gonna have to be careful about how I use the program lest it do something unexpected. Use man pages. The awscli is just terrible in this respect. There's no man page for 'aws' so I say "aws --help". It then literally tells me "To see help text, you can run: aws help". OpenShift's 'oc' sucks at this only a little less, with no man pages and for some inexplicable reason you can only get a list of global options in a dedicated global options help subcommand instead of at the bottom of every help page. The documentation system for 'git' on the other hand is a work of art. Pure beauty.

524 points by dickeytk 8y ago

Just the help factor alone is a big one.

In regards to 7, prompts are great for teaching new users how the program should be used. Instead of failing then spitting out --help or manpage style info, the program just ask the user enter the needed argument or flag to continue. Having more ways to learn usage is always good IMO.

dickeytk 8y ago
yep +1. A lot of times users are only ever going to run a command once. Better to ask for the right information than bailing out because it's not perfect syntax.

woodruffw 8y ago

> The user may have reasons for just not wanting this fancy output. Respect this if TERM=dumb or if they specify --no-color. Or if they specify `NO_COLOR`[1]! [1]: https://no-color.org/ https://no-color.org/

dickeytk 8y ago
yes, adding this
- deleted 8y ago
  [deleted]

drewmassey 8y ago

I’m a huge admirer or well crafted cli apps, they can massively boost the effectiveness of a team. An oldie but goodie here: https://eng.localytics.com/exploring-cli-best-practices/ https://eng.localytics.com/exploring-cli-best-practices/

stepvhen 8y ago

> I would skip man pages are they just aren’t used that often anymore. I understand that man pages might represent a minority, but I cannot express enough how wonderful it is to get the full manual of a program without interfacing with the web. Not to mention how powerful that is, since most apps have short names that are difficult to search for, but how accessible that makes the application.

norova 8y ago
I agree that man pages are wonderful. This point in the article really irked me.
dickeytk 8y ago
The fact they don't run on windows means some subset of your users cannot even use them if they wanted to. Better to spend your time on something they can all read. I'm not saying they're not useful. If you've got plenty of time to write up docs, go ahead, but the reality is we only have so much time and I think we should spend our time writing in-CLI docs and web docs before we start man pages. Also, you don't need web access to use in-CLI docs either, and that works on all platforms. Having said this, I do plan on having man pages be an export type of the oclif docs (which is currently in-CLI and markdown). I intentionally made the output very similar to man pages already so it should be relatively easy to do.
- deleted 8y ago
  [deleted]
- therealjumbo 8y ago
  For man pages you could suggest writing markdown and using a build process to automatically generate man pages [1] in the event they aren't using oclif. EDIT: I think we've all been in areas without network access like on a plane and not having a man page in that scenario is very annoying. Also, you briefly say a few things about CLI apps using a remote API, you may want to add to that and say a few things about the proxy environment variables [2]. These are indispensible for corporate users. I think some early, early version of npm didn't respect the no_proxy environment variable, and for the http_proxy and https_proxy it required some arcane combination of: proxy in a flag, proxy in a config file, proxy environment variable set. It really should be an OR not an AND... Last but not least, another annoying thing was tools changing their config format or location. I think it was docker that changed their config file format and/or location like two or three times. Absolutely infuriating. 1. https://rtomayko.github.io/ronn/ronn.1.html https://rtomayko.github.io/ronn/ronn.1.html 2. https://wiki.archlinux.org/index.php/proxy_settings https://wiki.archlinux.org/index.php/proxy_settings

013a 8y ago

This is all great advice. The one thing this does miss is distribution, which is a HUGE part of offering a great CLI app. Specifically, I'd say: 1. Make your OFFICIAL distribution channel the primary package manager on each platform (ex: on Mac, homebrew. Ubuntu, apt/snap). Beyond that, support as many as you have capacity to. 2. Also offer an official docker image which fully encapsulates the CLI tool and all of its dependencies. This can be a great way to get a CLI tool loaded into a bespoke CI environment.

dickeytk 8y ago
yeah I agree 100%. Distribution of CLIs is a headache for the maintainer, but it's really important to get right if you want users to have a seamless experience.
curun1r 8y ago
Homebrew is NOT the primary package manager on Mac and I wish people would stop perpetuating that falsehood. Apple includes pkgutil/pkgbuild in the OS and that official package management strategy plays much better with corporate IT control of managed machines. In my experience, Homebrew always eventually results in pain and complex debugging and it's almost impossible to audit software it installs to prevent the installation of prohibited or dangerous software. It's really not that hard to build a .pkg file and developers that want to properly support the Mac platform should go down that path before offering Homebrew support.
- deathanatos 8y ago
  Well, I'll admit this is the first I'm learning of this tool. That said… The top hit on Google for "os x pkgbuild" is a link into Apple's documentation that 404s. (Further Googling turns up some blog posts, and man pages, so that's good.) Does this support dependencies? How do I get updates to users? How does a user receive updates?
  - curun1r 8y ago
    Wow...I'm not sure why Apple has taken down it's documentation for the command-line tools. It's been so long since I looked at anything beyond the man pages, I didn't realize the web docs weren't up. If I had to hazard a guess, I'd bet they're trying to move developers towards an XCode-centric approach. But there are plenty of other tools that can generate that format too. And it's the only format that's natively understood by the OS, so I still stand by my assertion that Homebrew is not the official package manager for Mac.

sigjuice 8y ago

Nitpicks: Replace “pipe content” with “redirect content”

dickeytk 8y ago
nice catch, ty

breckuh 8y ago

> 1. Great help is essential I like how this is their #1. In my opinion the best way to do this is with tldr. https://github.com/tldr-pages/tldr https://github.com/tldr-pages/tldr I'd highly recommend folks create a tldr page for their CLI app. Add 4-8 examples to cover 80%+ of the most common use cases. -h flags, readmes & man pages can cover the other 20%.

dickeytk 8y ago
I almost want to rewrite the help section to encourage examples even more. They're incredibly valuable. I hadn't considered this before you mentioned it, but oclif CLIs could integrate to tldr pretty well. It already supports arrays of strings for examples.
- tetha 8y ago
  Yup. On CPAN, it is encouraged that the first part of your documentation after the table of contents is the synopsis[1]. The synopsis should clearly show how to do the common tricks with the library. From there you can link and refer to the more detailed documentation. We're doing that for our internal CLI applications and it's great to be able to just copy-paste the common use case from the top of the documentation without searching much. 1: https://metacpan.org/pod/Carp https://metacpan.org/pod/Carp
  - dickeytk 8y ago
    I feel the synopsis section of man pages often just becomes a bunch of useless garbage above the fold (for instance, look at `man git`). Using it less as a complete docopts kind of thing and more of multiple common usages (like `man tar` and what you linked) is far more useful. I think there is something here I hadn't really considered before. It's not an example, but also not a useless dump of flags. Food for thought I suppose.
- Riverheart 8y ago
  Agreed on making examples more important. Powershell, for example, let's you "get-help command -examples" to just retrieve those.

kpcyrd 8y ago

I disagree on the 2nd point. Flags prevent globbing by the shell and make the help text less clear. Consider a usage line like this: prog <user> [password] This tells you which argument is mandatory and which argument is optional in a second, without searching for the help text of --user and --password. Also, an example like this: git add <pathspec>... Tells you that git add accepts multiple paths and you can invoke it with: git add src/* What's more important in my opinion is making sure your argument parser can handle values that start with a dash and respects a double dash to stop the option parser. Consider an interface like this: prog [--rm] [--name <name>] [args...] And you invoke it like this: prog --name --rm -- --name foo This should result in: { "name": "--rm", "args: ["--name", "foo"] } Getting things like this wrong can result in security issues.

dickeytk 8y ago
I may try to expand on this in the article, but it's in there if you read between the lines. There is a difference between something that takes in multiple args of the same type and multiple TYPES of args. I'm arguing against multiple args of different types, not the same. By definition any CLI that accepts variable args is fine here as it's all the same type. The -- is a great point as well. It solves a lot of problems users have but a lot of time people don't even know about it. It solves issues with `heroku run` for example. EDIT: updated to clarify my point
chriswarbo 8y ago
I don't like using arguments for key/value data, e.g. `prog --name foo`. Much better to use env vars like `NAME=foo prog`, since the OS handles the mapping automatically, reading them out is trivial in every language, passing args to subprocesses is easier, etc.

ISO-morphism 8y ago

These are all good points, and I wish more clis were like this. My own pet peeve is un-disablable stdout logging. > It’s important that each row of your output is a single ‘entry’ of data. It felt weird to me to use `ls` as an example as it's not immediately obvious it adheres to the advice from the printed output. I suppose they were also trying to highlight the earlier point of differing output format depending on whether output is a tty/pipe. Unrelated, but I didn't know `ls` was that smart about isatty. Once upon a time I read the '-1' option to print one name per line in the man page and assumed it was necessary for that functionality. Thanks!

dickeytk 8y ago
I just picked `ls` as it's a common utility everyone understands and isn't some contrived example using `cat`. I am going to add a note about `ls`'s behavior with isatty. It's sort of conflating a couple of things, but I think it's interesting enough to leave it in.
- ISO-morphism 8y ago
  I agree, it is a good example, and hey I learned something. I really appreciate that you're taking the time to respond to all the feedback in this thread and wade through everyone's nitpicks. Looking forward to more articles.
  - dickeytk 8y ago
    Thank you for the great feedback!
dotancohen 8y ago
> My own pet peeve is un-disablable stdout logging. I really think that we need a stdmeta file descriptor. See my comments at the link below, I would appreciate your feedback: https://unix.stackexchange.com/questions/197809/propose-additional-file-descriptor-stdmeta https://unix.stackexchange.com/questions/197809/propose-addi...

nimish 8y ago

#6 is great, except if it causes performance issues: https://github.com/npm/npm/issues/11283 https://github.com/npm/npm/issues/11283 Speed is the ultimate fancy enhancement ;)

roylez 8y ago
Even if it does not cause any performance issues, I dislike it. If the thing runs in terminal, it should be expect to be used in a script thus it would be better to make no assumptions of terminal capabilities and leave the fancy part to external tools, if one is interested. I always hate systemctl's piping to a pager by default. "Do one thing, do it well", don't try to surprise users with fanciness because at work we don't like surprises.
- dickeytk 8y ago
  In practice I've had overwhelming feedback praising our use of spinners in the Heroku CLI and not a single complaint I can think of. In fact, I've had more praise for adding spinners and color than any other change we've put in over 4+ years of development. That said, you need to be careful. Don't use a spinner if it's not a tty or TERM=dumb. Do use it in some CI environments that support it (Travis, CircleCI) That handles all the issues we've seen and everyone seems to be happy.
OptionX 8y ago
And more than that can cause compatibility issues. For example, its a bit of a pain to use utf-8 in the normal windows cmd.
AgentME 8y ago
Wasn't that issue fixed?

emmanueloga_ 8y ago

Don't get me wrong! I love command line apps. But I wonder if we all have a bit of an Stockholm syndrome... there are several things that suck about them... While writing this I'm thinking on my experience trying to do anything with ffmpeg or imagemagick... or even find. * For any sufficiently complicated cmd line app, the list of arguments can be huge and the --help so terse as to be become useless. For man pages, the problem is the opposite... the forest hides the tree! I'm sure we all end up using google to look for example invocations. * Very often completion doesn't work, since custom per-app machinery is needed. For instance: git-completion for with bash-completion. * Sometimes I end up passing the help output through grep, then copy-pasting the flags from the output, and then hoping I got the right flag. * ...how about things like regular expressions parameters... always so hard to remember the escaping rules! (and the regex flavor accepted by each different app). * Not to talk about more complicated setups involved -print0 parameters or anything involving xargs, tee, and formatting with sed and cut, etc. Is there a better way? Not sure. I like powershell a bit but some of the things I mention above still apply. I think we may be able to get a workflow that is a bit closer to the tooling we use for writing programs while not being perceived as verbose and heavy (I'm thinking, the kind of workflow I get with a Clojure repl).

NeedMoreTea 8y ago
For a quick args reminder, without having to dig through man pages, try tldr: https://tldr.sh/ https://tldr.sh/
forkerenok 8y ago
Speaking for myself of ffmpeg and imagemagick, I don't find the CLI itself difficult, I find that their application domain is a bit complicated for a layman. Meaning I usually struggle to comprehend what particular options mean rather than how they bind to CLI. Maybe indeed a good intuitive UI would give more intuition about the more obscure options. But then again, a good intuitive UI is a quest of its own.
- tajen 8y ago
  What is difficult with ffmpeg is the sub-language in each argument, which depends on the filter you’re using. I can learn to lookup the manual, but if I have to go to the internet to learn the syntax of the filter, I’m better off googling my usecas.

henkdevries 8y ago

Man I wish OpenVMS was still a thing. All the commands worked the same due to the DCL enforcing it. https://en.m.wikipedia.org/wiki/DIGITAL_Command_Language https://en.m.wikipedia.org/wiki/DIGITAL_Command_Language

jwilk 8y ago
Non-mobile link: https://en.wikipedia.org/wiki/DIGITAL_Command_Language https://en.wikipedia.org/wiki/DIGITAL_Command_Language
stunthamsterio 8y ago
Me and you both. OpenVMS got so much right back in the day. Good error conventions, versioned file system... I still fondly run an OpenVMS workstation under my desk so I can revisit the good old days sometimes. Then realise my skills have atrophied to the point I can barely remember how to use edit.
bigpicture 8y ago
A) PowerShell was inspired by OpenVMS DCL. B) OpenVMS on x86 is due to be released in 2019 (it's in private beta right now).

OJFord 8y ago

> 12. Follow XDG-spec I'm so glad to see this included. I don't like $HOME being cluttered with .<app> config directories, but worse than that, far too many when releasing on macOS say Oh Library/Application\ Support/<app>/vom/something is the standard config location on Mac, so I'll respect XDG on Linux but on Mac it should go there. No! Such an unfriendly location for editable config files.

dickeytk 8y ago
I am having an impossible time verifying this, but I recall reading that MacOS deletes unaccessed files eventually from `~/Library/Caches/*`. Which would be a compelling reason to use that for cache. (Not being able to verify this I didn't add it to the article) If anyone can verify that I'm either right or wrong here that would be helpful.
- zbentley 8y ago
  OSX does not automatically delete from that directory. See: https://developer.apple.com/library/archive/documentation/General/Conceptual/MOSXAppProgrammingGuide/AppRuntime/AppRuntime.html https://developer.apple.com/library/archive/documentation/Ge... > Your app is responsible for cleaning out cache data files when they are no longer needed. The system does not delete files from this directory. However, many third-party tools delete from that directory with minimal caution if any, so it's a good idea to consider it ephemeral.
  - dickeytk 8y ago
    appreciate the clarification!
andreareina 8y ago
I agree that seeing a bunch of ~/.<app> directories is annoying, but at the same time I do think that it makes sense for each application to manage its own hierarchy, rooted under e.g. ~/.apps/<app> instead of splitting it into ~/.config/<app>, ~/.local/share/<app>, etc. Regardless, I think it probably makes sense to have a uniform interface for getting said directories, so that however the OS decides things should be laid out, the developer just needs to `local_config_dir(app_name)`. If the user (or at least administrator) can decide between <app>/<function> and <function>/<app>, all the better.

keithnz 8y ago

While a bit quirky, I really like powershells approach where you aren't limited to text streams. All the same advice applies.

jstanley 8y ago

> Error: EPERM - Invalid permissions on myfile.out > Cannot write to myfile.out, file does not have write permissions > Fix with: chmod +w myfile.out I actually much prefer: "can't write myfile.out: Permission denied" This shows the same information as the first 2 lines combined from the example, and the 3rd line is not necessarily the correct way to fix the problem anyway (e.g. you might be running it as your user when it should be root, chmod +w would not help). If you are so convinced that chmod +w is the way to fix the problem, why not just do that and carry on without bugging the user? And having each error confined to one line also means it's much less likely that some of the lines are missed, e.g. when grepping a log. EDIT: And to add to this: it's sometimes useful to prefix the error messages with the name of the program that generated them, so that when you're looking at a combined log from different places, you know which of the programs actually wrote the error, e.g. "mycli: can't write myfile.out: Permission denied".

Beowolve 8y ago
So, I understand the basis of your comment. You have the knowledge to know that there are other things that may be "the right way" given your situation. I think what the author is getting at is that there are users who don't have that knowledge. Giving them a hint that is verbose and non arcane can make a world of difference. Speaking from personal experience, there are many developers that I have met who don't have basic *nix knowledge, much less knowledge of a terminal. The reality of the situation is that a business is going to hire people regardless of that ability. They want someone who can move the features out the door. Whether this is good or bad is probably beyond this conversation. I think, for those users, these sorts of helpful hints are extremely important because it makes them feel like they aren't stuck and helpless. I think that, to your point, it may be useful to have the CLI offer a "pro" mode in which you could set a config to not give you as verbose error messages. Annoying? Yes. However, it would strike a balance and serve both needs.
- jstanley 8y ago
  I understand what you're saying, but this conversation is about the best way to design CLIs. I agree that you can do lots of stuff suboptimally and still have a usable tool, but I disagree with the author on what the ideal error output looks like.

hornetblack 8y ago

Also on Color. Don't got go all pschadelic. I've found that some programs using 256-color is unreadable with my terminal colors. Also unix commands tend to have illegible colors in Powershell on Windows. (Ripgrep for example). Powershell defaults to a blue background.

dickeytk 8y ago
Yeah if I get some time I should expand on that. There are some colors that don't work well at all for common scenarios (solarized, windows). Some of them we've blacklisted and do not use. Also for colors it's good to try to pick a decent palette of a couple-three colors and stick with it rather than try to categorize a bunch of different bits of output on a single command. Still, colors are awesome and make your CLI look and feel 10x better than it is, so it's worth the extra effort.
- gvalkov 8y ago
  A problem with color is that people end up optimizing the aesthetics for their own terminal setup. There is a wild number of different color schemes out there and it's really hard to make something that looks good on all of them. In my experience the only safe choice is bold text (i.e. \033[1m) since it stands out in all cases.
  - dickeytk 8y ago
    red, yellow, green, cyan, and magenta are pretty safe. We've had complaints about others, but these are pretty reliable in my experience. dim is safe too, but doesn't work all the time. (Not working meaning just not dimmed)
    - roryokane 8y ago
      I would definitely not call yellow a safe color. I use a terminal with a white background, and yellow text is unreadable unless the text is also given a custom background color.
8y ago

gnomewascool 8y ago

I almost stopped reading at "I would skip man pages", but the rest of the article was mostly great advice. I disagree about 11 (using "main_command sub_command:sub-sub_command" rather than "main sub sub-sub" syntax), but it's mostly a matter of taste. Seriously, though, if you've already taken the time to write documentation, then there's no reason not to also generate a manpage. Just using pandoc to convert your, say, README.md gives good-enough results: pandoc -s -f markdown_github -t man -o your_cli.1 README.md (There probably are other good conversion methods.) Why I like man: Advantage over online docs: It's offline and available directly in the terminal, without having to open a browser and it has a distraction-free, clean look. The only slight disadvantage is the lack of support for images, which are occasionally helpful, but in a pinch, for some use-cases, you can have ascii diagrams. Advantages over "--help": 1. Conventionally, "--help" just provides a brief rundown/reminder of the options, so having full documentation is valuable. 2. If "--help" provides the full docs then: a) You lose the option of having the brief rundown, which is also very valuable. b) "man command" is slightly faster than "command --help" :p (yes, it is a slight pity that accessing the full docs is faster than accessing the brief version, if you use convention). c) man deals with things like having nice output, with proper margins, at different terminal widths. d) man deals with the formatting for you, providing consistency with all other applications. FWIW I think that texinfo is (mostly) even better than man, as it considerably improves on the navigation, but it's been crippled by the FSF-Debian GFDL feud, which meant that the info pages weren't actually installed on many systems, and it's mostly a lost cause now.

enriquto 8y ago
> Advantages of man over "--help": You can have the best both worlds if the manpages are built automatically from the "--help" output (e.g., using help2man). Then you can have "-h" give a brief rundown and "--help" give the full docs. > FWIW I think that texinfo is (mostly) even better than man, as it considerably improves on the navigation I am curious about that. Do you really like texinfo navigation? I find it completely unusable, to the point of prefering to download and print a pdf from the web instead of opening (gasp!) the dreaded "info" program.
- 8y ago

sytelus 8y ago

Does these CLI features like tables, OS notifications etc work cross-platform (Linux, Windows, OSX)? IS there any good library to develop such CLI apps?

dickeytk 8y ago
we use https://github.com/mikaelbr/node-notifier https://github.com/mikaelbr/node-notifier

sudofail 8y ago

I'm probably being picky, but I would also include that the CLI be a self-contained binary. I'm tired of managing Python / Ruby / Node versions.

nhumrich 8y ago
Its bad both ways. The advantage to python/ruby for example is you can simply pip/gem install, or update. With a binary, you have to download, move, and change permissions every update. For experienced linux users, the binary is fine, but for newer users, its much more "friction"
- Sean1708 8y ago
  IMO using a language's package manager to install applications is a massive anti-pattern, that should be handled by your OS package manager.
  - sudofail 8y ago
    This is what I prefer as well. Let me use my OS's package manager for managing my packages.
    - mixmastamyk 8y ago
      That would still be the case if the packages weren't 1-3 years out of date.
      - jetblackio 8y ago
        That is true. But there are ways around that. Including documentation on how to build it locally is pretty standard. And hosting prebuilt binaries with package installation for targeted platforms is also pretty common as well. With interpreted languages with language-specific package managers, you have to: 1) Install the language 1a) Possibly have to install a language version manager (rbenv, pyenv, etc) 2) Install the language's package manager 3) Install the CLI utility via the language's package manager Here's the order I think CLI maintainers should strive to making their utilities available: 1) Install via OS package manager 2) Install via prebuilt release with OS-specific package, from hosting site (GitHub, etc). 3) Install from source 4) Install via language-specific package manager 5) Install via curl | sh :)

Walkman 8y ago

Unless you already know your users will want man pages, I wouldn’t bother also outputting them as they just aren’t used often enough anymore. I don't know where this is coming from, me and my colleagues are reading man pages every day. I would be interested how much others read them.

kolme 8y ago
I strongly disagree on this one too. That's the first place I look for help and it annoys me to no end when a CLI program that doesn't come with one. I stopped taking seriously the article at that point and quickly skimmed through the rest of it. Man pages are a great unix culture heritage, please new developers don't give up on them!
- davidhcefx 8y ago
  I myself also love reading man pages, but speaking of compatibility, I have to say that “--help” is a more universal way of showing help pages. Of course it’s better to have both of them though.
  - larkeith 8y ago
    --help is fine, but almost never a substitute for a full man page, except for the most trivial of applications (unless your --help is as complete as a man page, in which case... good on you for providing full documentation, but I'll hate you a bit every time I unthinkingly drop two hundred lines of text in my terminal.)
    - smarinov 8y ago
      Unless it just opens the man page if it is so long. Like git.
  - JdeBP 8y ago
    I strongly suspect that --help is on the contrary less universal, given that there exist entire toolsets lacking the --help convention but having manual pages. There was almost a quarter of a century's worth of Unix tools that developed before --help was invented. * https://unix.stackexchange.com/a/416796/5132 https://unix.stackexchange.com/a/416796/5132 * https://unix.stackexchange.com/questions/207136/ https://unix.stackexchange.com/questions/207136/

bayindirh 8y ago

I don't agree with 7 and 8. I like silent apps while working, and actually I'm used to applications saying nothing if everything is correct. Also, "outputting something to stdout just because I can" kills scriptability a lot. Using tables, colors and other stuff requires a lot of terminal support. MacOS terminal, iTerm, Linux terminals supports a lot of stuff, but not always (our team is generally using XTerm for example). Implementing these are acceptable if there's a robust code detecting terminal capabilities and falling back gracefully and without treating these more streamlined terminals as lesser citizens, and this requires a lot of development, head banging and maintenance. If you're accepting the challenge, then go on. BTW, That unicode spinner is nice. Very nice.

oblio 8y ago
You can just have a "quiet" mode for scripting. Or even better, detect if you're connected to a TTY.
- bayindirh 8y ago
  Checking whether you're connected to a TTY is a good practice, but there's also cognitive advantages of silent applications too. Also, similarly you can have a -v --verbose set to make the application talkative. It's also an option.
- scbrg 8y ago
  I run scripts from a tty all the time...
Sean1708 8y ago
> Also, "outputting something to stdout just because I can" kills scriptability a lot. The article does specifically recommend using stderr for messages to the user, is there any reason that would kill scriptability?
- bayindirh 8y ago
  I missed that. It's actually a much bigger problem. stdout and stderr(or) is two output paths to explicitly diverge error logs and problems from standard output. stdout / stdin is reserved for user interaction, informational messages (md5sum), actual output (ls, less, etc.), and the like. OTOH, stderr is explicitly for error messages only, and it's very useful (and necessary). A real use case from my daily job: I'm an administrator of a large system (approx ~1K servers), and I have substantial amount of cattle, and a lot of pets [0]. All of these servers have cron jobs, and other automated tasks on them. All servers can mail to a local mail server to report us problems, so our cron automatically mails any outputs to us. All the tools we use, and scripts we write have the following properties: - If everything is OK, they are silent by default. - They output to stderr, if anything notable happens. - Also we copy (think tee) all stderr to their respective logs (both local and on a centralized server). Now consider: - The (e-mail, log) noise if all the tools were writing their outputs to stdout. - The work required to silence all tools. What if they don't have any --quiet switches? - Furthermore, they wrote everything to stderr. How can I know whether thing has worked as it should? - How can I find the problem quickly if everything is written to "Error" log? - Furthermore how can I revise the errors happened quickly? Info & Error on the same file. grep galore! - How can I silent a tool (by redirecting to /dev/null) if both normal and error logs are written via stderr? These are the simple problems that I can come up on a real, big production system in five minutes. I can find more problematic scenarios which will happen on a daily basis, if I think more. All these conventions, folklore, recommended usage and facilities are in place because of the needs and the experience acquired in the history of these systems. Running around them amok, just because they enable color, pretty spinners, or justify some narrow usage scenario is not correct. These conventions and philosophy [1] allowed *NIX systems to scale without needing excessive administrative elbow grease. Ignoring these, and developing tools which use facilities and conventions as they please will degrade the ability to manage these systems with minimal work. I can always grep, but it will be inefficient and not guaranteed to get everything that I want, and also it will cost me and the computer time to do so. Like algorithms, systems are easier to manage when "n" is small. When "n" gets big, these tasks got really hard, really fast. So, develop tools to ease tasks. Not to show-off. [0] https://www.engineyard.com/blog/pets-vs-cattle https://www.engineyard.com/blog/pets-vs-cattle [1] https://en.wikipedia.org/wiki/Unix_philosophy https://en.wikipedia.org/wiki/Unix_philosophy Edit: Tried to increase readability.

O_H_E 8y ago

> Follow XDG-spec Oh, please guys. Some apps even put visible (non .) folders in my home.

justinrlle 8y ago

> I also suggest sending the version string as the User-Agent so you can debug server-side issues. (Assuming your CLI uses an API of some sort) Isn't it some kind of disguised tracking? I know it doesn't give as much info as the user agent of a browser, but still, you could track the OS, even the linux distribution, and surely more, while still being a reproducible build.

czechdeveloper 8y ago

I have recently created single CLI program to put in all actions I need to automate. It's so fast to make new action, that I make anything that saves me just few seconds a day. Even stuff like "invoice" will open me timescheduling app, invoicing app and creates canned email to clipboard. "Clockout" will open timescheduling app and copy expected date and times to clipboard just to paste to app. I've of course spent some time on automating Help, flags parsing etc, so I essentially just say what data I need and then what to perform. It was best idea in a long time. I'm thinking that I'll publish framework for this as opensource (it's C# project).

superlevure 8y ago

Does someone has the name of the terminal app used in the article ? (with the nice colored path)

kjaer 8y ago
It looks like zsh with the Oh My Zsh [1] with the Agnoster theme [2]. 1. https://ohmyz.sh/ https://ohmyz.sh/ 2. https://github.com/robbyrussell/oh-my-zsh/wiki/Themes#agnoster https://github.com/robbyrussell/oh-my-zsh/wiki/Themes#agnost...
- superlevure 8y ago
  Thank you !

phoe-krk 8y ago

I clicked this in hope that these was an article describing 12 CLI apps written in the Factor language. http://factorcode.org/ http://factorcode.org/ My hopes were crushed.

JepZ 8y ago

> 7. Prompt if you can Please don't. There is nothing wrong with interactive tools, but by default, they should not be. So instead of making non-interactive session possible via flags, the default should be to be non-interactive. If there is an option to start an interactive session, everything is fine. Otherwise, you would never know when your script could run into some kind of interactive session (and therefore break; possibly after an update).

jessaustin 8y ago
My interpretation was that the prompt is for required information. In the example graphic, "run demo" really does require that "stage" be specified. This is considered more user-friendly than simply crashing. If you don't want to see the prompt, provide that information as a flag or in a config file or whatever.
- dickeytk 8y ago
  Correct, and if stdin is not a tty it should error out instead of prompting
  - iainmerrick 8y ago
    The one thing missing from the example is a little more contextual help. Not only should it prompt you for the stage, it should say “use --stage [development|staging|production] on the command line to skip this prompt” or some such. (Could be as terse as the prompt reading “please specify --stage”.)
    - dickeytk 8y ago
      Like for the confirmation example? I agree. It would help clear this up. The points people are raising here with prompting are definitely not issues, they're just misunderstanding my point.
      - jessaustin 8y ago

jessaustin 8y ago

...all of these must show help. $ mycli Some commands have an unambiguous meaning and don't need arguments. For example, it would be weird if a bare "make" command returned help information. Great post, though. I'm looking forward to digging into oclif.

stuaxo 8y ago

"12 factor" anything seems to be a symptom of the over-complexity of modern apps, go back and rethink.

shoo 8y ago
i found the original 12 factor website had a number of pretty reasonable suggestions based on experience of people who ran a business doing operations for other people's web apps. sure, web apps are themselves probably over complicated, but given that you're doing a web app, the recommendations arent bad. compare to where things have gone since, with containerisation.
- subway 8y ago
  In a lot of ways the "12 factors" have overly simplistic views of the world. For instance, storing your config in the environment is fantastic -- until you remember a great many frameworks will dump their environment to the browser in a number of failure scenarios. There go all your secrets.

gtramont 8y ago

Related: http://docopt.org/ http://docopt.org/ – There are implementations for various languages. Whenever I need to write something that has a CLI, this is my default option…

fiatjaf 8y ago

Is it just me or every now and then Medium opens with what looks like to be a snapshot of the article instead of the HTML? I can't select text or scroll the page. If I refresh the page everything is normal, though.

rdsubhas 8y ago

Really great advice here. But missing one key area: Continuous Delivery / Change Management. I believe any policy without change management principles isn't really complete, especially when its about 12factor which is considered a gold standard for production. * CLIs are notoriously difficult to update because you have to convince every single consumer to update it manually, otherwise you just have scattered logic everywhere. Having an update workflow is essential before releasing the first version in production. * Closely tied, a clear Backwards compatibility policy. Apart from those two major items, I have also found one optionally nice pattern to reason about CLIs: Design CLIs like APIs wherever possible. Treat subcommands as paths, arguments as identifiers, and flags as query/post parameters. It's not always applicable, but doing that for large internal tools helps against the "kitchen sink" syndrome.

kurtisc 8y ago

>Still, you need to be able to fall back and know when to fall back to more basic behavior. If the user’s stdout isn’t connected to a tty (usually this means their piping to a file), then don’t display colors on stdout. (likewise with stderr) GCC does this, leading to no colour output where it would be useful if you're building with Google's Ninja-build. Maybe there are some people who do pipe GCC output to a file - I've never had to. If you do this with your app, I'd appreciate being able to re-enable the colour.

twic 8y ago

> 8. Use tables > By keeping each row to a single entry, you can do things like pipe to wc to get the count of lines, or grep to filter each line > Allow output in csv or json. Yes please. Default to readable-but-shellable tabular output, and support other formats. libxo from the BSD world is a really smart idea - it provides an API that programs can use to emit data, with implementations for text, XML, JSON, and HTML: http://juniper.github.io/libxo/libxo-manual.html http://juniper.github.io/libxo/libxo-manual.html I personally love CSV output. Something like libxo means that CSV output could be added to every program in the system in one fell swoop.

lousyd 8y ago

Only a web programmer could believe that man pages "just aren't used that often". It drives me nuts when compiled cli programs don't have a man page. It says to me that the author of the program doesn't know Unix conventions or doesn't care enough to put the effort into meeting his or her users where they are, and so I'm gonna have to be careful about how I use the program lest it do something unexpected. Use man pages. The awscli is just terrible in this respect. There's no man page for 'aws' so I say "aws --help". It then literally tells me "To see help text, you can run: aws help". OpenShift's 'oc' sucks at this only a little less, with no man pages and for some inexplicable reason you can only get a list of global options in a dedicated global options help subcommand instead of at the bottom of every help page. The documentation system for 'git' on the other hand is a work of art. Pure beauty.