5 ms·
Pipe Viewer
- bsg75 14y agoFiled under "How did I not know this?!", with credit to @hmason on Twitter.
- pyre 14y agoIt's been here before: http://news.ycombinator.com/item?id=3626020 http://news.ycombinator.com/item?id=3626020 http://news.ycombinator.com/item?id=3198572 http://news.ycombinator.com/item?id=3198572 I'm sure there were older stories too. IIRC I saw it on HN longer than 6 months ago, but I couldn't find that submission in a cursory search.
- crazy_eye 14y agolove pv, mostly use it for percentage bar on db restores: pv -f *.sql.gz | zcat | mysql db_instance_name
- deleted 14y ago[deleted]
- mibbitier 14y agoHow does it know the progress / ETA? I was under the assumption that pipes do not know how much total data is on its way?
- aidenn0 14y agoIf you give it a file, or specify the amount on the commandline it knows, otherwise it doesn't
- mbreese 14y agoRight. I usually use pv in place of 'cat' at the start of a long set of piped commands. Anywhere you could use 'cat' you can use pv. If I'm not mistaken, I think you can also use multiple 'pv' commands at a time too.
- pronoiac 14y ago> If I'm not mistaken, I think you can also use multiple 'pv' commands at a time too. I use multiple pv instances to track compression/decompression rates, and overall progress, for tarballs. Using the "-c" code for console codes keeps the instances from stomping over each other's output.
- lloeki 14y agoThis is indeed visible in the example, only source has progress while bzcat and gzip only have flow.
- bsg75 14y agoYou have to supply it the stream sizes if you want completion percentages, otherwise it displays rates only. Example: https://gist.github.com/2783910 https://gist.github.com/2783910
- dbuthay 14y agoIt's amazingly useful to have realtime throughput rates tail -f query.log | pv -tri 1 > /dev/null
- pramodbiligiri 14y agoNifty command. Been using this for a month or so. One use case is measuring if a downstream process can consume data as fast the previous process is producing it (by alternating second process with /dev/null or some such).
- spudlyo 14y agoIt's also great that pv supports rate limiting, so you can choose if you want to saturate your network when you use it with netcat.
- there 14y agoSomewhat related, trickle is a good user-space bandwidth shaper if you aren't sending data over a pipe: http://monkey.org/~marius/pages/?page=trickle http://monkey.org/~marius/pages/?page=trickle
- ta12121 14y agohttp://news.ycombinator.com/item?id=3198572 http://news.ycombinator.com/item?id=3198572 http://news.ycombinator.com/item?id=2567186 http://news.ycombinator.com/item?id=2567186
- gouranga 14y agoNice - that'll spice up my rather boring svnadmin dump sessions - thanks for posting!
- ralph 14y agoNote, a plain dd will report progress on stderr if sent a SIGUSR1. That can sometimes be enough to avoid the overhead of pv(1) and sending huge amounts of data through another pipe.
- protopete 14y agoAbout the pv overhead, I sent a note to the pv author saying that on Linux at least the transfer function could take advantage of the splice system call. Splice "copies" data from one file descriptor to another file descriptor, avoiding having to copy the data from the kernel to userspace and back.
- deleted 14y ago[deleted]
- Erwin 14y agoOn FreeBSD (and maybe Mac OS X) you can press Control-T to send a SIGINFO signal (non-standard extension. This prints out how far dd has gotten (and perhaps other FreeBSD utilities too).
- there 14y agoYou may need to setup your terminal to do that with: stty status '^T' fsck also responds to SIGINFO by printing its status. I had to hack up rsync to do it (http://jcs.org/patches/rsync-show_current_file.diff http://jcs.org/patches/rsync-show_current_file.diff), and I wish more utilities supported it by default.
- sneak 14y agoN.B. also that dd on osx will EXIT IMMEDIATELY if sent a SIGUSR1. It double-sucks because one tends to do this on long-running dd processes that you're wondering are done yet...
- kilovoltaire 14y agoyes | pv > /dev/null
- sp332 14y agopv /dev/zero > /dev/null
- kilovoltaire 14y agoawesome, i was actually wondering if there was a "yes-esque" (yesque?) /dev/ file. my throughput from yes is 49MB/s whereas from /dev/zero it's 12GB/s
- potatohead00 14y agoFound out about pv ~year ago when working with large log files. Very useful. I also like it for testing how quickly a script can consume data.
- b0b0b0b 14y agoIf I'm dealing with [compressed] files on disk or across a network, what's the practical overhead of pv? A few extra percent CPU, or some material decrease in through put?
- blibble 14y agovery very little, it's just doing read(), counting the bytes, then write(), with some screen output every few seconds. (hell it could even use splice() and never even see the bytes)