3 ms·

The proposed change to free-disk is a hack. Using --track-bytes-deleted on ZFS wont really track the bytes deleted, but just the size of the file that was dele

The proposed change to free-disk is a hack. Using --track-bytes-deleted on ZFS wont really track the bytes deleted, but just the size of the file that was deleted -- as the author points out, that may not be the number of bytes actually freed. For my money, "sleep 10 before checking free-space" is a more robust solution.

Andys 4y ago
Yes, and file_size != space_consumed is pretty much the main point of using ZFS!
drpixie 4y ago
It is a hack, but given the complexity and non-timeliness of what's going on behind the scenes, and the lack of control that the user has, perhaps a good hack is the right solution. My choice would be a cron job to run every few hours (or every day) that calculates space required and deletes logs files to that size. As long as the desired free space leaves an adequate margin, this would be robust and work in the presence of compressed files, snapshots, etc. "Sleep 10" should be fine, but it's also a hack, and the author would need to consider how long it takes for cleared space to become available on a heavily loaded system. I would expect that under severe load, the clear-up lag could be very long.
mustache_kimono 4y ago
> For my money, "sleep 10 before checking free-space" is a more robust solution. And running more often. 50GB(?) at a time can back up any filesystem. Not to mention, there is another option which perhaps better matches this use case. And author seems to be using the zfs quota property when the refquota property might better account for descendants and perhaps make a better initial calculation of what free space is available easier? ZFS is great but absolutely has tradeoffs. It makes really good choices. Maybe this is one?
mnw21cam 4y ago
The "sleep 10 seconds" solution isn't robust enough. Consider if you have created a snapshot on the filesystem. That snapshot has to keep hold of all the data, so when you delete files, the amount of free space doesn't change. A loop with "sleep 10 seconds" is likely to delete every single file in the filesystem in an attempt to get that desired amount of free space. A very common thing to do with ZFS is to have automatically rotating snapshots, so that you can go and retrieve a file that you accidentally deleted or changed, without having to go to the backups. In this case, the space is freed when the last snapshot containing those files is rotated away, which could be day or a week later, depending on how it is set up.

Andys 4y ago

Yes, and file_size != space_consumed is pretty much the main point of using ZFS!

drpixie 4y ago

It is a hack, but given the complexity and non-timeliness of what's going on behind the scenes, and the lack of control that the user has, perhaps a good hack is the right solution. My choice would be a cron job to run every few hours (or every day) that calculates space required and deletes logs files to that size. As long as the desired free space leaves an adequate margin, this would be robust and work in the presence of compressed files, snapshots, etc. "Sleep 10" should be fine, but it's also a hack, and the author would need to consider how long it takes for cleared space to become available on a heavily loaded system. I would expect that under severe load, the clear-up lag could be very long.

mustache_kimono 4y ago

> For my money, "sleep 10 before checking free-space" is a more robust solution. And running more often. 50GB(?) at a time can back up any filesystem. Not to mention, there is another option which perhaps better matches this use case. And author seems to be using the zfs quota property when the refquota property might better account for descendants and perhaps make a better initial calculation of what free space is available easier? ZFS is great but absolutely has tradeoffs. It makes really good choices. Maybe this is one?

mnw21cam 4y ago

The "sleep 10 seconds" solution isn't robust enough. Consider if you have created a snapshot on the filesystem. That snapshot has to keep hold of all the data, so when you delete files, the amount of free space doesn't change. A loop with "sleep 10 seconds" is likely to delete every single file in the filesystem in an attempt to get that desired amount of free space. A very common thing to do with ZFS is to have automatically rotating snapshots, so that you can go and retrieve a file that you accidentally deleted or changed, without having to go to the backups. In this case, the space is freed when the last snapshot containing those files is rotated away, which could be day or a week later, depending on how it is set up.