4 ms·

The author appears to be mistaken about what's going on here, I think. It's not that ZFS takes a long time to update free space - it's that it actually doesn't

The author appears to be mistaken about what's going on here, I think. It's not that ZFS takes a long time to update free space - it's that it actually doesn't necessarily free the space immediately. Deletes over a certain size (by default 20480 blocks, so 10/80 MB for 512b/4k sectors, respectively, I believe) dump the thing to be deleted (assuming it's to be freed, dedup refcounting/snapshots/etc might mean it's not actually freed) on an async queue that gets worked through in the background, rather than blocking rm on it. (Otherwise, if the files are smaller than that, I think it's as the author says, with bundling things into a txg and only updating once they're flushed out. But the example was ~40 files totalling several GB, so I think it's as I describe above.)

tbrownaw 4y ago
Does this queue have a way to check the age (timestamp, txg id, whatever) of the oldest thing in it?
- toast0 4y ago
  There's zfs wait, which can wait on the delete queue. Otoh, files that are deleted and open will be on the delete queue until they're closed, so that queue doesn't have any guarantee of progress while the system remains running. A process to delete less important files to maintain free space that waited on the delete queue to drain before reassessing the situation would need something more sophisticated.
- rincebrain 4y ago
  I'm not aware of one, but don't think it would be terribly hard to implement. I don't know of anyone who has, probably because it's not really a thing that's broken for anyone in a way where that data point would be interesting...that I know of. Could be wrong. Maybe I'll remember and go try it in a day or two, when I'm not at an event[1]... [1] - https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022 https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022
cryptonector 4y ago
Yes, there is this and then too the fact that it's hard to predict how much space you'll get back by deleting files because of snapshots and compression. Combine the two things and managing free space is hard for users.

tbrownaw 4y ago

Does this queue have a way to check the age (timestamp, txg id, whatever) of the oldest thing in it?

toast0 4y ago
There's zfs wait, which can wait on the delete queue. Otoh, files that are deleted and open will be on the delete queue until they're closed, so that queue doesn't have any guarantee of progress while the system remains running. A process to delete less important files to maintain free space that waited on the delete queue to drain before reassessing the situation would need something more sophisticated.
rincebrain 4y ago
I'm not aware of one, but don't think it would be terribly hard to implement. I don't know of anyone who has, probably because it's not really a thing that's broken for anyone in a way where that data point would be interesting...that I know of. Could be wrong. Maybe I'll remember and go try it in a day or two, when I'm not at an event[1]... [1] - https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022 https://openzfs.org/wiki/OpenZFS_Developer_Summit_2022

cryptonector 4y ago

Yes, there is this and then too the fact that it's hard to predict how much space you'll get back by deleting files because of snapshots and compression. Combine the two things and managing free space is hard for users.