4 ms·

As a method of obscuring the data slightly (although really it doesn't add much), I sometimes like to add "conv=swab" to the dd. But for the CPU that's being s

As a method of obscuring the data slightly (although really it doesn't add much), I sometimes like to add "conv=swab" to the dd. But for the CPU that's being spent on the bzip2, you could do it slightly more easily with a single command on one machine: dd bs=16M if=/dev/sda | ssh dest.host.com 'bzip2 -d >/dev/sda' Of course, that presupposes a running sshd on dest.host.com. You could secure it from being observed (although not from being interfered with) under Linux using loopback devices: ## On the destination machine: LOD=`losetup -f` losetup -e aes $LOD /dev/sda # You'll be prompted for a password for the encryption. nc -l 31337 | bzip2 -d | dd bs=16M of=$LOD ## On the source machine: LOD=`losetup -f` losetup -e aes $LOD /dev/sda # You'll be prompted for a password; use the same one. dd bs=16M if=$LOD | bzip2 -c |nc dest.host.com 31337 Most people just use encrypted loopback devices for things like disk encryption, but losetup (like netcat) has so many other fun potential applications. (Edited twice due to formatting sadness.)

iuguy 16y ago
That second one's really interesting. I've tried the SSH route and while it works (and works well) it really knocks up the time to send an image over the wire. We've used this for forensic imaging in the past and have relied upon digital signatures as integrity checks. It's not great I know, but it's the best way to get disk images from a big server onto a NAS (when you can't plug the NAS into the server).
- 1337p337 16y ago
  The second one is way more fun, using the real device as the plaintext and the loopback as the cyphered version. Have you tried using the -C (compression) option to ssh instead of the bzip2 pipe? I don't often have occasion to do this sort of thing, so if you've tried that already, it would be interesting to know relative timing. pbzip2 ( compression.ca/pbzip2/ ) might be helpful sometimes, but I don't suspect that it's carried on most live CDs (unless you guys are rolling your own) and the CPU probably isn't the bottleneck anyway. 9P2000 has recently made it into the Linux kernel and it gets way better throughput than NFS, but (and I'm not too familiar with NAS hardware or software, so I may be saying something foolish) I don't suspect it's a supported protocol on those things. Either way, I also don't suspect that adding the overhead of an FS protocol could be helpful in cases where netcat would do, but it's a nice tool to have in the box.
  - iuguy 16y ago
    I haven't tried -C on SSH, but I'll make a note to try it next time I get the chance. We use QNAP NAS so we're more or less stuck to specific kernel versions, but the userland is fairly complete (as you have basically got a ton of disk space for userland apps!)

iuguy 16y ago

That second one's really interesting. I've tried the SSH route and while it works (and works well) it really knocks up the time to send an image over the wire. We've used this for forensic imaging in the past and have relied upon digital signatures as integrity checks. It's not great I know, but it's the best way to get disk images from a big server onto a NAS (when you can't plug the NAS into the server).

1337p337 16y ago
The second one is way more fun, using the real device as the plaintext and the loopback as the cyphered version. Have you tried using the -C (compression) option to ssh instead of the bzip2 pipe? I don't often have occasion to do this sort of thing, so if you've tried that already, it would be interesting to know relative timing. pbzip2 ( compression.ca/pbzip2/ ) might be helpful sometimes, but I don't suspect that it's carried on most live CDs (unless you guys are rolling your own) and the CPU probably isn't the bottleneck anyway. 9P2000 has recently made it into the Linux kernel and it gets way better throughput than NFS, but (and I'm not too familiar with NAS hardware or software, so I may be saying something foolish) I don't suspect it's a supported protocol on those things. Either way, I also don't suspect that adding the overhead of an FS protocol could be helpful in cases where netcat would do, but it's a nice tool to have in the box.
- iuguy 16y ago
  I haven't tried -C on SSH, but I'll make a note to try it next time I get the chance. We use QNAP NAS so we're more or less stuck to specific kernel versions, but the userland is fairly complete (as you have basically got a ton of disk space for userland apps!)