4 ms·

the cult of cat? the cult of pv? Really, we just need a progress meter that says "don't bother getting up up", "get lunch now", or "come back tomorrow", but I d

by jepler 4y ago

the cult of cat? the cult of pv? Really, we just need a progress meter that says "don't bother getting up up", "get lunch now", or "come back tomorrow", but I don't know that there is one. For bulk copying of /dev/zero to /dev/null, `dd` is fastest (block sizes of 256kB up to 4MB are about the same speed), then `pv` (block sizes of 256kB up to 4MB are about the same speed), then `head`. But it's true, unless you're dealing with an actual tape drive, there aren't a lot of things in the world anymore that depend on a specific block size; the idea that you'll get a faster transfer if you (say) exactly match the erase block size of your SD card doesn't seem to hold much water anymore. And debian's bash-completion of `dd` has been broken for several releases, I despair of it ever working right again.

askiiart 4y ago
> Really, we just need a progress meter that says "don't bother getting up up", "get lunch now", or "come back tomorrow", but don't know that there is one. It feel like this is gonna lead to another thing like thefuck...
Andys 4y ago
mbuffer does the job
yjftsjthsd-h 4y ago
> we just need a progress meter that says "don't bother getting up up", "get lunch now", or "come back tomorrow", but I don't know that there is one. Doesn't `status=progress` work? (The "obscure option to GNU dd" that the article mentions)
- Beltalowda 4y ago
  On BSD systems you can press ^T for SIGINFO and that prints some progress information, at least for tools like dd and cp, not sure about cat and I don't have a BSD system to test. Linux doesn't have this unfortunately. GNU tools often support it (when run on BSD), it's just that the kernel doesn't.
  - BenjiWiebe 4y ago
    On Linux you can do a 'kill -USR1 $(pidof dd)' to have dd print a progress report.
GekkePrutser 4y ago
> The idea that you'll get a faster transfer if you (say) exactly match the erase block size of your SD card doesn't seem to hold much water anymore. No but if you use a much lower block size with cheap flash it does get horribly slow. Makes sense because cheap eMMC does not have cache RAM to combine write commands and no NCQ (native command queueing) either. So it has to execute each write as it gets it. I bet you can kill flash pretty quickly this way with a block size of 1. The write amplification will be huge. The problem with the cat method is that you don't really know what it's doing under the hood in terms of write sizes. Probably it will pick something smart but it depends on the OS there and possibly the shell.
knowaveragejoe 4y ago
Assuming you're piping between programs, you could try pipeviewer: https://www.ivarch.com/programs/pv.shtml https://www.ivarch.com/programs/pv.shtml

by jepler 4y ago

askiiart 4y ago

> Really, we just need a progress meter that says "don't bother getting up up", "get lunch now", or "come back tomorrow", but don't know that there is one. It feel like this is gonna lead to another thing like thefuck...

Andys 4y ago

mbuffer does the job

yjftsjthsd-h 4y ago

> we just need a progress meter that says "don't bother getting up up", "get lunch now", or "come back tomorrow", but I don't know that there is one. Doesn't `status=progress` work? (The "obscure option to GNU dd" that the article mentions)

Beltalowda 4y ago
On BSD systems you can press ^T for SIGINFO and that prints some progress information, at least for tools like dd and cp, not sure about cat and I don't have a BSD system to test. Linux doesn't have this unfortunately. GNU tools often support it (when run on BSD), it's just that the kernel doesn't.
- BenjiWiebe 4y ago
  On Linux you can do a 'kill -USR1 $(pidof dd)' to have dd print a progress report.

GekkePrutser 4y ago

> The idea that you'll get a faster transfer if you (say) exactly match the erase block size of your SD card doesn't seem to hold much water anymore. No but if you use a much lower block size with cheap flash it does get horribly slow. Makes sense because cheap eMMC does not have cache RAM to combine write commands and no NCQ (native command queueing) either. So it has to execute each write as it gets it. I bet you can kill flash pretty quickly this way with a block size of 1. The write amplification will be huge. The problem with the cat method is that you don't really know what it's doing under the hood in terms of write sizes. Probably it will pick something smart but it depends on the OS there and possibly the shell.

knowaveragejoe 4y ago

Assuming you're piping between programs, you could try pipeviewer: https://www.ivarch.com/programs/pv.shtml https://www.ivarch.com/programs/pv.shtml