9 ms·
/usr/bin/time: not the command you think you know
- jimrandomh 10y agoThe reason 'time' is a builtin is because it can time a complete shell pipeline, not just a single command. If you type time foo |bar Then the result is the total time taken by foo and bar together. This requires it to have special-case syntax. Whereas /usr/bin/time foo |bar Would run foo and give its time statistics as input to bar.
- chubot 10y agoYes, as a consequence, it also lets you do this: time { sleep 0.1 sleep 0.2 } # result is 0.3 seconds which you obviously can't do with an external command. I wrote about that here: https://lobste.rs/s/hwgt2h/usr_bin_time_not_command_you_think_you_know https://lobste.rs/s/hwgt2h/usr_bin_time_not_command_you_thin...
- arthur2e5 10y agoThe `time` syntax in combination with for loops or just shell functions is pretty useful as well.
- ramshorns 10y agoIt looks like /usr/bin/time foo |bar would actually give the output of foo as input to bar, since the time statistics go to stderr.
- dotancohen 10y agoSince it is not an error, what we really need is a stdmeta file descriptor: http://unix.stackexchange.com/questions/197809/propose-additional-file-descriptor-stdmeta http://unix.stackexchange.com/questions/197809/propose-addit...
- easytiger 10y agoJust think of it as std dbg, not err
- eriknstr 10y agoI've been thinking a bit about this before and I think that not just one but a whole lot of additional streams would be useful. By default all commands would still use stdout and stderr only but you would be able to tell them to use other streams as well for their various data. Furthermore, each stream should be possible to pipe to a different process if you wish independently of the other streams. To a small degree this might be possible already with named pipes.
- MrQuincle 10y agoI just started playing around with IPC ideas with processes tellin each other which flags they would like: https://github.com/mrquincle/bipipes https://github.com/mrquincle/bipipes I think socket pairs would be great for it. Nothing implemented yet!
- Klathmon 10y agoHonestly this is one area where PowerShell kind of nailed it. I still don't use it enough, but when it comes to scripting PS's objects beat the hell out of these
- ncw33 10y agoIt is possible already - you can do `command 2>&4` to separate the output from one command in a pipeline into its own stream, then do `command2 <&4` to pull from the further down the pipeline.
- eriknstr 10y agoI would like more pipes for each process. This would allow many things, for example searching for multiple terms across all files in the whole file system in one go without opening each file more than once, and then grouping the different results together in accordance with the term that they matched.
- jcoffland 10y agoRun these commands for a better demonstration: time echo | sleep 1 vs. /usr/bin/time echo | sleep 1 The former times the entire pipeline whereas the later only times the first command in the pipe.
- fnord123 10y agoThis also gives the time of the entire pipeline (since the commands all begin together and only finish when the last command has completed): echo | /usr/bin/time sleep 1
- roblabla 10y agoI'm pretty sure this is wrong. As an example : sleep 2 | /usr/bin/time sleep 1 This gives an output of "1 second" even though the entire pipeline took 2 seconds.
- exprA 10y agoThe sleep sub-process of time is not part of the pipeline, however. It will exit after the requested time has passed.
- jstimpfle 10y agoThat is merely an optimization though. You can simply time sh -c 'foo | bar'
- therein 10y agoBut now you are timing the startup of `sh` as well.
- aexaey 10y ago...which (surprisingly) is a measurable amount. About 1 ms for `dash` and a whooping 9.5 ms for `bash` on a relatively slow CPU: pi@pi:~ $ time (for x in {1..1000}; do time echo | sleep 0.01; done 2>/dev/null) real 0m24.173s pi@pi:~ $ time (for x in {1..1000}; do time sh -c 'echo | sleep 0.01'; done 2>/dev/null) real 0m25.170s pi@pi:~ $ time (for x in {1..1000}; do time bash -c 'echo | sleep 0.01'; done 2>/dev/null) real 0m33.643s
- jstimpfle 10y ago1ms is on the order of process creation time, so not very relevant: The measured program itself requires process creation. Most programs run considerably longer than that (otherwise timing would not make much sense). It's really not more than that: an optimization.
- inetknght 10y ago`/usr/bin/time -o foo_time.txt foo | bar`
- Merovius 10y agoNo, it wouldn't require special-case syntax. It could just pass it's argument to '/bin/sh -c'. You'd have to shell-escape the pipeline, though.
- eridius 10y agoYeah it does require special-case syntax. When using bash's built-in time, the command is executed exactly the same way as if you weren't using time, which means the command can set variables or define functions in the current shell. For example, you can say time foo=bar echo $foo and it will correctly print out "bar". It can't do this if it executes the command in a subshell. Or similarly you can say `time cd /foo` and it will actually change the current working directory.
- floatboth 10y agoWow — that is extremely surprising. I did not expect that behavior at all. Who the hell thought it was a good idea to let a builtin command absorb the pipe symbol?!
- d4l3k 10y ago/usr/bin/time doesn't seem to be a thing on my Arch Linux install. ¯\_(ツ)_/¯
- fbernier 10y agoSame here. I had to fetch it via pacman. Also, the -l option doesn't exist on the non-bsd version.
- wkz 10y ago`-v` (--verbose) Seems to give the same (although differently formatted) output on GNU's implementation.
- tomsmeding 10y agoI love arch. Sudo doesn't exist until you install it (directly or indirectly).
- jcoffland 10y agoThat's the same on Debian.
- ajsalminen 10y agoYeah, I don't think it's all that rare for sudo to be an optional package not installed by default in distributions. At least that used to be the case back when I used try various ones a lot more.
- majewsky 10y ago$ pacman -Fo /usr/bin/time usr/bin/time is owned by extra/time 1.7-8 If that command doesn't work for you, try adding "-y". `man pacman` for details.
- hobarrera 10y ago$ which time time: shell reserved word $ ls /usr/bin/time /bin/time ls: cannot access '/usr/bin/time': No such file or directory ls: cannot access '/bin/time': No such file or directory Looks like something specific to the author's distribution.
- myrandomcomment 10y agoHe is on MacOS.
- pmontra 10y agoIt's time -l on BSD derivatives and time -v on GNU userlands.
- ams6110 10y agoIt's a BSD thing.
- ramshorns 10y agoThere's a GNU time as well, but it appears not to always be installed by default.
- voltagex_ 10y agoYep. GNU source: http://git.savannah.gnu.org/cgit/time.git/tree/ http://git.savannah.gnu.org/cgit/time.git/tree/ BSD source: https://github.com/freebsd/freebsd/blob/f5d95e1f8d32db4ccccfd5ad9cecb21ed07a695d/usr.bin/time/time.c https://github.com/freebsd/freebsd/blob/f5d95e1f8d32db4ccccf... Busybox source: https://github.com/mirror/busybox/blob/f88e3bfa8ded4f1c7e7d2143a1cfcbbdfe5e8541/miscutils/time.c https://github.com/mirror/busybox/blob/f88e3bfa8ded4f1c7e7d2... Toybox source: https://github.com/landley/toybox/blob/master/toys/posix/time.c https://github.com/landley/toybox/blob/master/toys/posix/tim... This would have been a lot easier if I'd known GitHub has a `filename:` search
- kylek 10y ago
- brendangregg 10y agoNo, /usr/bin/time is indeed what I know, and its extended stats is why I suggested using it in my last perf book (time -v). "/usr/bin/time: not the command you think you know" -> "/usr/bin/time: may not be the command you think you know" There, I fixed the title.
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- an_account 10y ago/r/iamverysmart
- deleted 10y ago[deleted]
- inopinatus 10y agoOr even "may not be the command I think you know".
- halostatue 10y agoIn both bash and zsh, you can force the shell to use $PATH for lookup (bypassing functions and shell builtins) by calling a builtin name with 'command' ('command time -l ls'). You can equivalently force a builtin with 'builtin', but that does not work with reserved words (and 'time' is a shell reserved word).
- ramshorns 10y agoTIL that a shell reserved word is different from a shell builtin. http://unix.stackexchange.com/questions/267761/differences-between-keyword-reserved-word-and-builtin http://unix.stackexchange.com/questions/267761/differences-b...
- halostatue 10y agoMe, too. I was surprised because I was trying to use `builtin time`. :)
- xelxebar 10y agoYeah. Shell semantics can be pretty unintuitive sometimes. I often find it helpful to translate these ideas to standard programming language terms. * Commands are like functions * Commands in /bin etc. are like library functions * Builtins are like a language's primitive functions * Keywords are keywords
- LeoPanthera 10y agoYou can also just prefix the command with a backslash. "\time" will run the version in the path.
- dredmorbius 10y agoSince the article doesn't clarify on shell command priority, commands are search in order: 1. Alias expansion. 2. Defined shell functions. 3. Built-in shell functions. 4. Command path. If you provide an unqualified command that matches more than one of these elements, the first match wins. Prepending a backstroke: "\command", will inhibit alias expansion. E.g.: alias date="echo no date" date \date Should return "no date", and your current system date, respectively. To invoke a system command directly, call the full path. If you don't feel like running the fish shell (not that there's anything wrong with that). Some simple shells (e.g., dash, and IIRC the original Bourne shell, though that is not what you'll find as /bin/sh on most modern systems) don't include a time builtin, and can invoke the system time command directly.
- yrro 10y agoYou can also use the 'command' built-in to skip aliases and shell functions. $ command time --version GNU time 1.7
- knome 10y agoPerhaps, but time is a syntactic form, like `if` or `case`. https://www.gnu.org/software/bash/manual/html_node/Pipelines.html#Pipelines https://www.gnu.org/software/bash/manual/html_node/Pipelines...
- dredmorbius 10y agoI'm sorry, that's not clear to me. What's the significance exactly?
- knome 10y agoA command takes a series of arguments, stdin/out/err pipes and returns an error code on completion. time prefixes a pipeline ( the most basic case being a single command without a pipe into another ), a command block { ... } a subshell block ( ... ), a for statement, an if statement, just whatever really. This "time" would output how long "a" took to run: /usr/bin/time a b c | { d e ; f g ; } | h i ; This "time" outputs how long the pipeline "a", "d", "f", and "h" took to run: time a b c | { d e ; f g ; } | h i ; This is a syntax error: /usr/bin/time { d e ; f g ; } This returns how long the command group takes to run: time { d e ; f g ; } Further examples: $ time sleep 10 | sleep 1 ; real 0m10.073s user 0m0.000s sys 0m0.000s $ time sleep 1 | sleep 10 ; real 0m10.003s user 0m0.000s sys 0m0.000s $ /usr/bin/time sleep 10 | sleep 1 ; 0.00user 0.00system 0:10.00elapsed 0%CPU (0avgtext+0avgdata 1796maxresident)k 0inputs+0outputs (0major+80minor)pagefaults 0swaps $ $ /usr/bin/time sleep 1 | sleep 10 ; 0.00user 0.00system 0:01.00elapsed 0%CPU (0avgtext+0avgdata 1808maxresident)k 0inputs+0outputs (0major+82minor)pagefaults 0swaps Note the two /usr/bin/time's output their timing information as soon as the first command is done, but the pipeline doesn't return until both commands have exited. > hope this rant helps in some way
- eccstartup 10y ago/usr/bin/time: not the command you think you want
- graton 10y agoOn bash you can do this to have it not use an alias or built-in command: $ \time echo 0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 1764maxresident)k 0inputs+0outputs (0major+66minor)pagefaults 0swaps The built-in version: $ time echo real 0m0.000s user 0m0.000s sys 0m0.000s
- yrro 10y agoIs this behaviour documented anywhere? The closest I can find is https://www.gnu.org/software/bash/manual/bashref.html#Aliases https://www.gnu.org/software/bash/manual/bashref.html#Aliase... which remarks, > The first word of each simple command, if unquoted, is checked to see if it has an alias. I observe that t\ime works just as well as \time.
- ptx 10y agoJust quoting it normally also works, apparently, as your excerpt from the man page suggests: $ help|head -1 GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu) $ "time" --version GNU time 1.7 $ time --version bash: --version: command not found real 0m0.001s user 0m0.000s sys 0m0.000s
- sigjuice 10y agot\ime is rather surprising. Is there any documentation or explanation on why it does what it does?
- yrro 10y agohttps://www.gnu.org/software/bash/manual/bashref.html#Escape-Character https://www.gnu.org/software/bash/manual/bashref.html#Escape... I think it's because using the backslash to escape counts as quoting. So the following are all equivalent: $ \time foo $ t\ime foo $ 'time' foo
- bobbyi_settv 10y agoThe conventional way of forcing bash not to use a built-in is by using the full path (/usr/bin/time in this case).
- saagarjha 10y agoSo it's just like /usr/bin/cd–a builtin that also has a binary.
- tyingq 10y agoHow would that work? A forked/execed subprocess somehow forcing chdir() in it's parent? Guessing it isn't terribly useful.
- zwp 10y agoI too don't see how it is useful but it's certainly a thing. The Solaris implementation looks like this: #!/bin/ksh -p # ... cmd=`basename $0` $cmd "$@" I just noticed that the what(1)-string (I haven't seen on of those for a long time) references "alias.sh", perhaps this is a clue? #ident "@(#)alias.sh 1.2 00/02/15 SMI" Were builtins actually aliases in an early shell? I still don't understand how this works though.
- tyingq 10y agoThat's a funny implementation. It would end up being an infinite loop for any command that wasn't a builtin, or didn't have an identically named thing higher up the PATH. Like if you renamed it from /usr/bin/cd to /usr/bin/mycd.
- saagarjha 10y agoOn my Mac, it's #!/bin/sh # $FreeBSD: src/usr.bin/alias/generic.sh,v 1.2 2005/10/24 22:32:19 cperciva Exp $ # This file is in the public domain. builtin `echo ${0##*/} | tr \[:upper:] \[:lower:]` ${1+"$@"}
- cperciva 10y agoGuessing it isn't terribly useful. You guess correctly. The issue is that POSIX states that ... all of the standard utilities [...] shall be implemented in a manner so that they can be accessed via the exec family of functions as defined in the System Interfaces volume of POSIX.1-2008 and can be invoked directly by those standard utilities that require it (env, find, nice, nohup, time, xargs). Note however that /usr/bin/cd isn't completely useless: It has the same diagnostics (error message and exit code) as the command, so `env cd $FOO` is a way to check if you can change to that directory without actually doing it.
- deleted 10y ago[deleted]
- 7171u 10y agoI had to use "--verbose" instead of "-l" in my RHEL7 \time --verbose echo
- metafunctor 10y agoA more interesting command is /bin/[. Yep, it's the same as /bin/test, but basically requires the last argument to be a ]. Typically /bin/test and bin/[ are hard links, that is, they are physically the same file and share the same inode(s). You know, so you can type if [ $foo = $bar ]; then yes else exit fi This actually runs the command /bin/[, except most of the time it doesn't, because it's built-in to the shell. As as side-note, an article like this should at least mention which OS and shell the author is using.
- Elrac 10y ago> As as side-note, an article like this should at least mention which OS and shell the author is using. Very much agree! I just tried this on my 2.6.32 RHEL system, and it's never heard of "-l". It outputs very similar-looking information as in the article, though, when given "-v" .
- devnonymous 10y agoSo also are /usr/bin/{cd,[,echo,pwd,fg,..etc} a lot of them having subtle differences with their corresponding shell builtins. Most of the time the differences are not worth the hassle to remember, unless you somehow end up on a system with a broken filesystem (for example where /lib or /usr/lib is destroyed) and need to rescue stuff.
- cmurf 10y agoOn Fedora Linux, '/usr/bin/time -v <command>' rather than -l.