5 ms·

The go flag package has a nice API, but the user interface it creates is straight from its Rob Pike's Plan9 roots, and Linux users familiar with getopt command

The go flag package has a nice API, but the user interface it creates is straight from its Rob Pike's Plan9 roots, and Linux users familiar with getopt command line interfaces will find it strange and perhaps even ugly. I wanted to create command line programs that look like traditional GNU/Unix utilities, so I wrote the https://github.com/ogier/pflag https://github.com/ogier/pflag package. It's a drop-in replacement for the flag package, just change the import line. It also adds some basic features, such as the common pattern of a flag with a long-hand form usable with a double --dash and a shorthand character with a single -d.

rsc 12y ago
The Go flag package has nothing to do with Plan 9. Like Unix, Plan 9 had single-dash single-letter options ONLY. The long option syntax of getopt is a BSD/GNU-ism, not from Unix. The Go flag package's command lines adopt the observation made by the Google C++ command line flag parser (https://code.google.com/p/gflags https://code.google.com/p/gflags): the long/short --/- two-names-for-every-flag dichotomy was useful for backwards compatibility when recreating original Unix commands, but for new programs, it doesn't pull its weight. At the cost of requiring that each option be specified with a separate command-line argument, you can drop the distinction between - and -- entirely. (This is similar to what X11 programs do, although I believe they do not admit --x as a synonym for -x.) It's not clear to me what is "ugly" about not having to keep alternating between - and -- as you spell out options on the command line.
- burke 12y ago
  > It's not clear to me what is "ugly" about not having to keep alternating between - and -- as you spell out options on the command line. It's about the principle of least surprise. Regardless of whether double-dash opts came from UNIX or GNU (and obviously you are correct in that they came from BSD/GNU), they're the convention nowadays and it's less confusing for end users to just embrace the convention. Go has bucked the (recent) tradition on a lot of different fronts, which I love it for, but the flag package -- the interface to the users of the software, not the authors -- is one place I don't really think it's quite as appropriate. Certainly, Google is able to make these sort of changes in its internal world, and reach a high enough percentage of software that the new choice becomes normal, but Go will probably never have enough mindshare in the extraGoogle for its CLI parsing to not be "a little weird". --- As an addendum, I happily use the builtin flag package for software that is consumed primarily by my team. For software I expect other teams to use, who don't necessarily care about the fact that it was written in Go, I prefer pflag.
- jsmthrowaway 12y ago
  There is a difference between writing systems in an environment where you have a centralized flag parser and flag inheritance across google3, and shipping a binary to users who expect things to behave like getopt. Also, let's be honest, flags in google3 are a nightmare. They are certainly well-engineered but lost any semblance of sanity a long time before you and me got there. The kernel patch springs to mind. Having to rethink my entire career of something simple like "how to pass flags to a binary" had better bring significant improvement, and flag does not. flag regresses a lot of convention about the operator interface to a program. This: binary -vexd data output Is encouraged by flag to become: binary -verbose -extra -one-file-system -data-dir=data -output-dir=output I understand the argument for self-explanatory command lines but at the cost of unexpected (and far more verbose) behavior is a tough trade off. One thing I haven't seen discussed is how "required" options are encouraged by flag because of how difficult positional arguments are. That's in addition to everything in the article, which I agree with. Python argparse gets this nearly perfect, subcommands and all. flag is near the top of what I hate about Go as an SRE. Java too.
  - quotemstr 12y ago
    The kernel patch?
    - jsnell 12y ago
      The maximum command line size allowed by the Linux kernel used to be 128kB. This might sound crazy, but that limit was a real constraint for some Google services. My guess is that this is referring to a kernel patch to increase the maximum size. This is a somewhat educated guess. A long time ago at Google I used to work on the program that was used to launch most jobs on the clusters. To help mitigate the command line length problems, the launcher preferred generating command lines with -nofoo instead of --foo=false, and -foo instead of -foo=true. This saved a few characters per boolean flag. Of course it also caused no end of trouble for people who'd defined their own "three-state boolean" flags that could e.g. be true/false/autodetect. Not the finest hour of engineering, in retrospect.
      - 12y ago
- bborud 12y ago
  People insisting that "XYZ is not UNIX", Unix or however you want to capitalize it, is kinda pointless. For all practical purposes Linux is Unix, FreeBSD, OpenBSD and OSX are Unix. One would think that if any OS would foster the idea of sameness it would be those belonging to the Unix family. As for old vs new style command line options: that's now Unix in much the same way that Dave Gilmour was a proper part of Pink Floyd and not merely "the new guy". The original OS we call Unix has not been relevant to the discussion for 20-25 years.
  - slimsag 12y ago
    That's silly. Can I then say that the Cocoa GUI from OSX is therefore the "Unix GUI" ?
    - talideon 12y ago
      It's a Unix GUI, not the Unix GUI.
    - bborud 12y ago
      Unix now refers to a family of operating systems -- not a single OS. Thinking of Unix as a specific OS and a specific trademark in 2015 is just odd since the majority of people using various forms of Unix today were not even born when the distinction was even relevant. And just as with other families, the peculiarities of a single member does not automatically apply to the entire family.
    - GFK_of_xmaspast 12y ago
      Why not? It's not as if there hasn't been incredible diversity in the x windows experience over the last 30 years.
  - enneff 12y ago
    The GP's point was that the origins of single-dash single-character flags is the original Unix, not Plan 9 as the GGP had claimed.
- digi_owl 12y ago
  Thing is that you can compress - flags as long as they don't take a argument. Meaning that you can do things like ls -ltr to get ls in long format, time sorted and reversed.
- bobbyi_settv 12y ago
  There are so many existing scripts, books and guides that use "rm -rf", "du -hs", "ls -la" and so on that those will have to be supported forever. For many people, those sorts of commands are 80% of what they run on the command line. Having it so that the other 20% consisting of new programs works different is confusing and annoying.
- munificent 12y ago
  > the long/short --/- two-names-for-every-flag dichotomy was useful for backwards compatibility when recreating original Unix commands, but for new programs, it doesn't pull its weight. Short is more usable for users on the command line. Long is more maintainable in scripts. That's why it makes sense to have both. > At the cost of requiring that each option be specified with a separate command-line argument, you can drop the distinction between - and -- entirely. Again, that cost is high for users on the command line, one of the few places where every keystroke does matter. It's a lot more pleasant to be able to do: $ rm -rf Than: $ rm -r -f Or, worse: $ rm --recursive --force Orthogonality is definitely an important principle, but when it comes to usability, it often makes sense to offer multiple ways to accomplish the same goal for users in different circumstances. Ultimately, you're interfacing with a greater primate. Even with all of our nice features, we still don't support USB or web standards. Instead, it's all lossy OCR and weird muscles. You should expect a certain amount of... squishiness... when interfacing with something like that.
  - codebeaker 12y ago
    I would argue (at the risk of being down-voted) that if you are scripting tools then it is important to use the long option, not just because future-you might forget what the options are (relevant XKCD http://xkcd.com/1168/ http://xkcd.com/1168/), but also for searchability, it's rather difficult in almost any engine (except the wonderful new code-orientated ones that are cropping) up to search for `-xvzf`, but rather simpler to search for `extract verbose zip filename +tar`, for example.
    - burke 12y ago
      That's really what munificent was arguing too though -- that the long-form options are preferable for scripting, but that no one prefers "tar --extract --verbose --compressed --file" to "tar -xzvf" when you just have to extract a file on the command line (in fact, many people don't even bother with the "-", "v", or "z", opting for "tar xf"). This optional brevity is important for utilities.
  - rsc 12y ago
    Re command line vs scripts, I am not convinced that the mental overhead of knowing two names for each flag and flipping back and forth when you context switch between the command line and a shell script really makes that much sense. Certainly common flags should be short and less common flags need not be, but personally I'd rather have one way to spell each. Certainly having just one name per flag is more in keeping with Go trying to keep things simple. My officemate is a long-time Git user but was surprised the other day to see me type "git pull -r". He didn't know that -r was short for --rebase, and in fact I didn't know that --rebase was long for -r. This kind of mutual incomprehensibility is a tiny variant of the observation that every C++ programmer uses only a small subset of C++, but the problem is that each programmer chooses a different subset. Not having two names for a flag can be a feature. Re typing, I am also not convinced that the overhead of rm -r -f vs rm -rf is really that high. Of course, rm is a bad example: if you are implementing rm, you MUST use the Unix conventions and accept -rf. But for new commands not burned into decades-old muscle memory, it's not clear that it's needed. I think there are multiple valid design decisions that could be made here, and we've tried to tilt the balance toward simplicity, but of course other balances could be struck, and maybe in other contexts it might make sense to strike a different one. I tried to make sure it was possible that someone could write a package (say, flg) with the following API (and nothing else): // Package flg implements a getopt-compatible command line flag parser. package flg // Parse parses the command-line flags defined in package flag. // In contrast to flag.Parse, Parse imposes getopt semantics: // - single letter flag names must be specified with a single dash: -x // - longer names must be specified with a double dash: --long // - the argument to a single-letter flag can follow it immediately: // -xfoo means -x foo when -x takes an argument. // - multiple short flags can be combined: -xyz means -x -y -z, // when neither -x nor -y takes an argument. // - name aliases can be introduced by calling Alias before Parse func Parse() // Alias records new as an alias for the flag named old. // Typically old is a long name and new is a single-letter name or vice versa. // For example, Alias("r", "recursive"). func Alias(new, old string) I believe it is, although I haven't seen one. That would let programs still use package flag as the data definition, so that they can reuse any other "plug ins" for the package (like custom flag.Values) but still provide a getopt-style parser if desired.
deleted 12y ago
[deleted]
MoOmer 12y ago
Packages like this are where I find the debate between idiomatic Go usage and general use cases difficult to side on. I would err on the idiomatic side by using stdlib flag.
- jerf 12y ago
  "Use stdlib unless you can name a reason good enough to incur the cost of importing another package, which shouldn't be underestimated" is probably the short version of the idiomatic Go position. If you've only got three flags, there probably isn't any reason to use anything other than stdlib; you can't be winning big enough to pull in another package. If you're solely being driven by external scripts, it's harder to win than if you expect interactive usage. Of course, if you're writing a grep replacement and you've got half the alphabet in both capitalizations being used and half of them take arguments and a couple of things switch "modes"... well, by all means don't write that yourself, go get a package that does it well. Most of the stdlib is designed to hit the 95-99% use case, but if you're really pushing something hard you may need something else. Probably the only truly controversial bits of the Go standard answers are A: pulling in external packages isn't free and extremely frequently underestimated over the life of the product, they better be bringing some real value beyond merely preventing the developer's nose from crinkling a bit [1] and B: if you've only got three flags right now, don't go leaping to the conclusion you're in the process of writing a grep replacement; wait until you actually have the situation I described before dropping another package in (YAGNI). [1]: Contrast this with something like Node or Ruby that seems to encourage pulling in dozens of libraries, many of which may literally be single-digit lines of payload code, and it is considered totally worth it for developer tastes to alter significant elements of the language/runtime/stdlib. And I really do mean contrast this with... I am currently expressing no opinion as to which philosophy is "better".
  - deleted 12y ago
    [deleted]
  - XorNot 12y ago
    Of course if the Go stdlib produced something which looked more like almost every other Unix-like tool out there, then we wouldn't be having this discussion - but it doesn't, and leaving your app's commandline parsing until it becomes a catastrophe to manage (or your users complain) is technical debt whichever way you spin it. For a language like Go I also strongly question why keeping with the stdlib is necessary. There's value in a language like Python where you the environment is dynamic - Go produces statically linked binaries. Who cares what libraries you build them with? That's kind of the point.

rsc 12y ago

The Go flag package has nothing to do with Plan 9. Like Unix, Plan 9 had single-dash single-letter options ONLY. The long option syntax of getopt is a BSD/GNU-ism, not from Unix. The Go flag package's command lines adopt the observation made by the Google C++ command line flag parser (https://code.google.com/p/gflags https://code.google.com/p/gflags): the long/short --/- two-names-for-every-flag dichotomy was useful for backwards compatibility when recreating original Unix commands, but for new programs, it doesn't pull its weight. At the cost of requiring that each option be specified with a separate command-line argument, you can drop the distinction between - and -- entirely. (This is similar to what X11 programs do, although I believe they do not admit --x as a synonym for -x.) It's not clear to me what is "ugly" about not having to keep alternating between - and -- as you spell out options on the command line.

burke 12y ago
> It's not clear to me what is "ugly" about not having to keep alternating between - and -- as you spell out options on the command line. It's about the principle of least surprise. Regardless of whether double-dash opts came from UNIX or GNU (and obviously you are correct in that they came from BSD/GNU), they're the convention nowadays and it's less confusing for end users to just embrace the convention. Go has bucked the (recent) tradition on a lot of different fronts, which I love it for, but the flag package -- the interface to the users of the software, not the authors -- is one place I don't really think it's quite as appropriate. Certainly, Google is able to make these sort of changes in its internal world, and reach a high enough percentage of software that the new choice becomes normal, but Go will probably never have enough mindshare in the extraGoogle for its CLI parsing to not be "a little weird". --- As an addendum, I happily use the builtin flag package for software that is consumed primarily by my team. For software I expect other teams to use, who don't necessarily care about the fact that it was written in Go, I prefer pflag.
jsmthrowaway 12y ago
There is a difference between writing systems in an environment where you have a centralized flag parser and flag inheritance across google3, and shipping a binary to users who expect things to behave like getopt. Also, let's be honest, flags in google3 are a nightmare. They are certainly well-engineered but lost any semblance of sanity a long time before you and me got there. The kernel patch springs to mind. Having to rethink my entire career of something simple like "how to pass flags to a binary" had better bring significant improvement, and flag does not. flag regresses a lot of convention about the operator interface to a program. This: binary -vexd data output Is encouraged by flag to become: binary -verbose -extra -one-file-system -data-dir=data -output-dir=output I understand the argument for self-explanatory command lines but at the cost of unexpected (and far more verbose) behavior is a tough trade off. One thing I haven't seen discussed is how "required" options are encouraged by flag because of how difficult positional arguments are. That's in addition to everything in the article, which I agree with. Python argparse gets this nearly perfect, subcommands and all. flag is near the top of what I hate about Go as an SRE. Java too.
- quotemstr 12y ago
  The kernel patch?
  - jsnell 12y ago
    The maximum command line size allowed by the Linux kernel used to be 128kB. This might sound crazy, but that limit was a real constraint for some Google services. My guess is that this is referring to a kernel patch to increase the maximum size. This is a somewhat educated guess. A long time ago at Google I used to work on the program that was used to launch most jobs on the clusters. To help mitigate the command line length problems, the launcher preferred generating command lines with -nofoo instead of --foo=false, and -foo instead of -foo=true. This saved a few characters per boolean flag. Of course it also caused no end of trouble for people who'd defined their own "three-state boolean" flags that could e.g. be true/false/autodetect. Not the finest hour of engineering, in retrospect.
    - 12y ago
bborud 12y ago
People insisting that "XYZ is not UNIX", Unix or however you want to capitalize it, is kinda pointless. For all practical purposes Linux is Unix, FreeBSD, OpenBSD and OSX are Unix. One would think that if any OS would foster the idea of sameness it would be those belonging to the Unix family. As for old vs new style command line options: that's now Unix in much the same way that Dave Gilmour was a proper part of Pink Floyd and not merely "the new guy". The original OS we call Unix has not been relevant to the discussion for 20-25 years.
- slimsag 12y ago
  That's silly. Can I then say that the Cocoa GUI from OSX is therefore the "Unix GUI" ?
  - talideon 12y ago
    It's a Unix GUI, not the Unix GUI.
  - bborud 12y ago
    Unix now refers to a family of operating systems -- not a single OS. Thinking of Unix as a specific OS and a specific trademark in 2015 is just odd since the majority of people using various forms of Unix today were not even born when the distinction was even relevant. And just as with other families, the peculiarities of a single member does not automatically apply to the entire family.
  - GFK_of_xmaspast 12y ago
    Why not? It's not as if there hasn't been incredible diversity in the x windows experience over the last 30 years.
- enneff 12y ago
  The GP's point was that the origins of single-dash single-character flags is the original Unix, not Plan 9 as the GGP had claimed.
digi_owl 12y ago
Thing is that you can compress - flags as long as they don't take a argument. Meaning that you can do things like ls -ltr to get ls in long format, time sorted and reversed.
bobbyi_settv 12y ago
There are so many existing scripts, books and guides that use "rm -rf", "du -hs", "ls -la" and so on that those will have to be supported forever. For many people, those sorts of commands are 80% of what they run on the command line. Having it so that the other 20% consisting of new programs works different is confusing and annoying.
munificent 12y ago
> the long/short --/- two-names-for-every-flag dichotomy was useful for backwards compatibility when recreating original Unix commands, but for new programs, it doesn't pull its weight. Short is more usable for users on the command line. Long is more maintainable in scripts. That's why it makes sense to have both. > At the cost of requiring that each option be specified with a separate command-line argument, you can drop the distinction between - and -- entirely. Again, that cost is high for users on the command line, one of the few places where every keystroke does matter. It's a lot more pleasant to be able to do: $ rm -rf Than: $ rm -r -f Or, worse: $ rm --recursive --force Orthogonality is definitely an important principle, but when it comes to usability, it often makes sense to offer multiple ways to accomplish the same goal for users in different circumstances. Ultimately, you're interfacing with a greater primate. Even with all of our nice features, we still don't support USB or web standards. Instead, it's all lossy OCR and weird muscles. You should expect a certain amount of... squishiness... when interfacing with something like that.
- codebeaker 12y ago
  I would argue (at the risk of being down-voted) that if you are scripting tools then it is important to use the long option, not just because future-you might forget what the options are (relevant XKCD http://xkcd.com/1168/ http://xkcd.com/1168/), but also for searchability, it's rather difficult in almost any engine (except the wonderful new code-orientated ones that are cropping) up to search for `-xvzf`, but rather simpler to search for `extract verbose zip filename +tar`, for example.
  - burke 12y ago
    That's really what munificent was arguing too though -- that the long-form options are preferable for scripting, but that no one prefers "tar --extract --verbose --compressed --file" to "tar -xzvf" when you just have to extract a file on the command line (in fact, many people don't even bother with the "-", "v", or "z", opting for "tar xf"). This optional brevity is important for utilities.
- rsc 12y ago
  Re command line vs scripts, I am not convinced that the mental overhead of knowing two names for each flag and flipping back and forth when you context switch between the command line and a shell script really makes that much sense. Certainly common flags should be short and less common flags need not be, but personally I'd rather have one way to spell each. Certainly having just one name per flag is more in keeping with Go trying to keep things simple. My officemate is a long-time Git user but was surprised the other day to see me type "git pull -r". He didn't know that -r was short for --rebase, and in fact I didn't know that --rebase was long for -r. This kind of mutual incomprehensibility is a tiny variant of the observation that every C++ programmer uses only a small subset of C++, but the problem is that each programmer chooses a different subset. Not having two names for a flag can be a feature. Re typing, I am also not convinced that the overhead of rm -r -f vs rm -rf is really that high. Of course, rm is a bad example: if you are implementing rm, you MUST use the Unix conventions and accept -rf. But for new commands not burned into decades-old muscle memory, it's not clear that it's needed. I think there are multiple valid design decisions that could be made here, and we've tried to tilt the balance toward simplicity, but of course other balances could be struck, and maybe in other contexts it might make sense to strike a different one. I tried to make sure it was possible that someone could write a package (say, flg) with the following API (and nothing else): // Package flg implements a getopt-compatible command line flag parser. package flg // Parse parses the command-line flags defined in package flag. // In contrast to flag.Parse, Parse imposes getopt semantics: // - single letter flag names must be specified with a single dash: -x // - longer names must be specified with a double dash: --long // - the argument to a single-letter flag can follow it immediately: // -xfoo means -x foo when -x takes an argument. // - multiple short flags can be combined: -xyz means -x -y -z, // when neither -x nor -y takes an argument. // - name aliases can be introduced by calling Alias before Parse func Parse() // Alias records new as an alias for the flag named old. // Typically old is a long name and new is a single-letter name or vice versa. // For example, Alias("r", "recursive"). func Alias(new, old string) I believe it is, although I haven't seen one. That would let programs still use package flag as the data definition, so that they can reuse any other "plug ins" for the package (like custom flag.Values) but still provide a getopt-style parser if desired.

deleted 12y ago

[deleted]

MoOmer 12y ago

Packages like this are where I find the debate between idiomatic Go usage and general use cases difficult to side on. I would err on the idiomatic side by using stdlib flag.

jerf 12y ago
"Use stdlib unless you can name a reason good enough to incur the cost of importing another package, which shouldn't be underestimated" is probably the short version of the idiomatic Go position. If you've only got three flags, there probably isn't any reason to use anything other than stdlib; you can't be winning big enough to pull in another package. If you're solely being driven by external scripts, it's harder to win than if you expect interactive usage. Of course, if you're writing a grep replacement and you've got half the alphabet in both capitalizations being used and half of them take arguments and a couple of things switch "modes"... well, by all means don't write that yourself, go get a package that does it well. Most of the stdlib is designed to hit the 95-99% use case, but if you're really pushing something hard you may need something else. Probably the only truly controversial bits of the Go standard answers are A: pulling in external packages isn't free and extremely frequently underestimated over the life of the product, they better be bringing some real value beyond merely preventing the developer's nose from crinkling a bit [1] and B: if you've only got three flags right now, don't go leaping to the conclusion you're in the process of writing a grep replacement; wait until you actually have the situation I described before dropping another package in (YAGNI). [1]: Contrast this with something like Node or Ruby that seems to encourage pulling in dozens of libraries, many of which may literally be single-digit lines of payload code, and it is considered totally worth it for developer tastes to alter significant elements of the language/runtime/stdlib. And I really do mean contrast this with... I am currently expressing no opinion as to which philosophy is "better".
- deleted 12y ago
  [deleted]
- XorNot 12y ago
  Of course if the Go stdlib produced something which looked more like almost every other Unix-like tool out there, then we wouldn't be having this discussion - but it doesn't, and leaving your app's commandline parsing until it becomes a catastrophe to manage (or your users complain) is technical debt whichever way you spin it. For a language like Go I also strongly question why keeping with the stdlib is necessary. There's value in a language like Python where you the environment is dynamic - Go produces statically linked binaries. Who cares what libraries you build them with? That's kind of the point.