5 ms·

What's the simplest way to rewrite the data without actually copying the data? Like in place rewrite - you write what you read.

fhdkweig 7mo ago
I've seen "dd if=/dev/removable of=/dev/removable" suggested. I don't know if it actually works or if the OS optimizes it to a no-op.
- piyh 7mo ago
  the risk of catastrophic data loss from misuse of `dd` makes my hackles rise just looking at this. I will never forget when I mixed up `if` and `of` during a routine backup. `cat /dev/sda > /mnt/myDisk2` is so much safer, explicit, and in unix norms. It's also faster because you don't have to tune block size parameters. Plus you can also do `pv /dev/sda > /mnt/myDisk2` to get transfer speed details. Friends don't let friends use `dd` where `cat` can do the same job.
  - jmb99 7mo ago
    I stopped getting scared of `if` and `of` about a decade ago when I started explicitly saying (in my head) "input file" and "output file" rather than "if" and "of." You still can mess up the order, but imo no more easily than you can swap `cat in > out` for `cat out > in`. > Friends don't let friends use `dd` where `cat` can do the same job. Technically yes... but I like being able to explicitly set block sizes and force sync writes.
    - ogurechny 7mo ago
      I think you both are arguing about how to fight a bear with your bare hands. To win in that, you simply need to not fight with a bear. Let's say someone made an expansion board with a cool feature: there are 5 documented I/O addresses, but accessing any other address fries the stored firmware. What would you do? No, not leaving a lot of comments in code in CAPS LOCK. No, not printing the correct hexadecimal values in red to put the message on the wall. You make a driver that only allows access to the correct addresses, and configure the rest of the system to make sure that it can only work through that driver. Let's say there's a loading bay at the chemical plant with multiple flanges. If strong acid from the tanker is pumped into the main acid tank, everything is fine. If it is pumped into any other tank, the whole plant may explode and burn. What should be done? No, not promising that drivers will be fired, then shot by the firing squad if they make a mistake. Each connection is independently locked, and the driver only gets a single matching key. You have wonderful programmable devices that allow you to solve non-standard problems with non-standard tools. What should be done is making a wrapper for dd that just does not allow you to do anything you don't want to happen. Even the most basic script with checks and confirmation is enough.
- valleyer 7mo ago
  Certainly the OS can't optimize it to a no-op, since `dd` makes separate read and write syscalls. I suppose your `dd` implementation itself could do so, but I don't know why it would.
hpb42 7mo ago
Wouldn't a ZFS Scrub get the job done?
- toast0 7mo ago
  zpool scrub is what I would do, but it only rewrites data that needs it. The OP wanted to rewrite everything.

fhdkweig 7mo ago

I've seen "dd if=/dev/removable of=/dev/removable" suggested. I don't know if it actually works or if the OS optimizes it to a no-op.

piyh 7mo ago
the risk of catastrophic data loss from misuse of `dd` makes my hackles rise just looking at this. I will never forget when I mixed up `if` and `of` during a routine backup. `cat /dev/sda > /mnt/myDisk2` is so much safer, explicit, and in unix norms. It's also faster because you don't have to tune block size parameters. Plus you can also do `pv /dev/sda > /mnt/myDisk2` to get transfer speed details. Friends don't let friends use `dd` where `cat` can do the same job.
- jmb99 7mo ago
  I stopped getting scared of `if` and `of` about a decade ago when I started explicitly saying (in my head) "input file" and "output file" rather than "if" and "of." You still can mess up the order, but imo no more easily than you can swap `cat in > out` for `cat out > in`. > Friends don't let friends use `dd` where `cat` can do the same job. Technically yes... but I like being able to explicitly set block sizes and force sync writes.
  - ogurechny 7mo ago
    I think you both are arguing about how to fight a bear with your bare hands. To win in that, you simply need to not fight with a bear. Let's say someone made an expansion board with a cool feature: there are 5 documented I/O addresses, but accessing any other address fries the stored firmware. What would you do? No, not leaving a lot of comments in code in CAPS LOCK. No, not printing the correct hexadecimal values in red to put the message on the wall. You make a driver that only allows access to the correct addresses, and configure the rest of the system to make sure that it can only work through that driver. Let's say there's a loading bay at the chemical plant with multiple flanges. If strong acid from the tanker is pumped into the main acid tank, everything is fine. If it is pumped into any other tank, the whole plant may explode and burn. What should be done? No, not promising that drivers will be fired, then shot by the firing squad if they make a mistake. Each connection is independently locked, and the driver only gets a single matching key. You have wonderful programmable devices that allow you to solve non-standard problems with non-standard tools. What should be done is making a wrapper for dd that just does not allow you to do anything you don't want to happen. Even the most basic script with checks and confirmation is enough.
valleyer 7mo ago
Certainly the OS can't optimize it to a no-op, since `dd` makes separate read and write syscalls. I suppose your `dd` implementation itself could do so, but I don't know why it would.

hpb42 7mo ago

Wouldn't a ZFS Scrub get the job done?

toast0 7mo ago
zpool scrub is what I would do, but it only rewrites data that needs it. The OP wanted to rewrite everything.