3 ms·

Pretty much by the time you get to 100% full on ZFS, the latency is going to get atrocious anyway, but from my understanding there are multiple steps (from simp

by p_l 3y ago

Pretty much by the time you get to 100% full on ZFS, the latency is going to get atrocious anyway, but from my understanding there are multiple steps (from simplest to worst case) that ZFS permits in case you do hit the error: 1. Just remove some files - ZFS will attempt to do the right thing 2. Remove old snapshots 3. Mount the drive from another system (so nothing tries writing to it), then remove some files, reboot back to normal 4. Use `zfs send` to copy the data you want to keep to another bigger drive temporarily, then either prune the data or if you already filtered out any old snapshots, zero the original pool and reload it by `zfs send` from before.

Shorel 3y ago
Modern defrag seems very cumbersome xD
- p_l 3y ago
  Defragmentation and ability to do it are not free. You can have cheap defrag but comparatively brittle filesystems by making things modifiable in place. You can have filesystem that has as its primary value "never lose your data", but in exchange defragmentation is expensive.
  - dataflow 3y ago
    I don't buy this? What does defragmentation have to do with snapshotting? Defragmentation is just a rearrangement of the underlying blocks. Wouldn't snapshots just get moved around?
    - p_l 3y ago
      The problem is that you have to track down all pointers pointing to specific block. With snapshotting, especially with filesystems that can only write data through snapshots (like ZFS), blocks can be referred to by many pointers. It's similar to evaluating liveness of object in a GC, except you're now operating on possibly gigantic heap with very... pointer-ful objects, that you have to rewrite - which goes against core principle of ZFS which is data safety. You're doing essentially a huge history rewrite on something like git repo, with billions of small objects, and doing it safely means you have to rewrite every metadata block that in any way refers to given data block - and rewrite every metadata block pointing to those metadata blocks.
      - dataflow 3y ago
        But more pointers is just more cost, not outright inability to do it. The debate wasn't over whether defragmentation itself is costly. The question was whether merely making defragmentation possible would impose a cost on the rest of the system. So far you've only explained why defragmentation on a snapshotting volume would be expensive with typical schemes, which is entirely uncontroversial. But you neither explain why you believe defragmentation would be impossible (no "ability to do it") with your scheme, nor why you believe it's impossible for other schemes to make it possible "for free"? In fact, the main difficulty with garbage collectors is maintaining real-time performance. Throw that constraint out, and the game changes entirely.
        p_l 3y ago
        I never claimed it's impossible - I claimed it's expensive. Prohibitively expensive, as the team at Sun found out when they attempted to do so, and offline defrag becomes easy with two-space approach which is essentially "zfs send to separate device". You can attempt to add an extra indirection layer, but it does not really reduce fragmentation, it just lets you remap existing blocks to another location at a cost of extra lookup. This is in fact implemented in ZFS as solution for erroneous addition of a vdev, allowing device removal though due to performance cost its oriented mostly at "oops, I added the device wrongly, let me quickly revert".
        dataflow 3y ago
        If by "not able to" you meant "prohibitively expensive" - well, I also don't see why it's prohibitively expensive even without indirection. Moving blocks would seem to be a matter of (a) copy the data, (b) back up the old pointers, (c) update the pointers in-place, (d) mark the block move as committed, (e) and delete the old data/backups. If you crash in the middle you have the backup metadata journaled there to restore from. No indirection. What am I missing? I feel like you might have unstated assumptions somewhere?
        p_l 3y ago

by p_l 3y ago

Shorel 3y ago

Modern defrag seems very cumbersome xD

p_l 3y ago
Defragmentation and ability to do it are not free. You can have cheap defrag but comparatively brittle filesystems by making things modifiable in place. You can have filesystem that has as its primary value "never lose your data", but in exchange defragmentation is expensive.
- dataflow 3y ago
  I don't buy this? What does defragmentation have to do with snapshotting? Defragmentation is just a rearrangement of the underlying blocks. Wouldn't snapshots just get moved around?
  - p_l 3y ago
    The problem is that you have to track down all pointers pointing to specific block. With snapshotting, especially with filesystems that can only write data through snapshots (like ZFS), blocks can be referred to by many pointers. It's similar to evaluating liveness of object in a GC, except you're now operating on possibly gigantic heap with very... pointer-ful objects, that you have to rewrite - which goes against core principle of ZFS which is data safety. You're doing essentially a huge history rewrite on something like git repo, with billions of small objects, and doing it safely means you have to rewrite every metadata block that in any way refers to given data block - and rewrite every metadata block pointing to those metadata blocks.
    - dataflow 3y ago
      But more pointers is just more cost, not outright inability to do it. The debate wasn't over whether defragmentation itself is costly. The question was whether merely making defragmentation possible would impose a cost on the rest of the system. So far you've only explained why defragmentation on a snapshotting volume would be expensive with typical schemes, which is entirely uncontroversial. But you neither explain why you believe defragmentation would be impossible (no "ability to do it") with your scheme, nor why you believe it's impossible for other schemes to make it possible "for free"? In fact, the main difficulty with garbage collectors is maintaining real-time performance. Throw that constraint out, and the game changes entirely.
      - p_l 3y ago
        I never claimed it's impossible - I claimed it's expensive. Prohibitively expensive, as the team at Sun found out when they attempted to do so, and offline defrag becomes easy with two-space approach which is essentially "zfs send to separate device". You can attempt to add an extra indirection layer, but it does not really reduce fragmentation, it just lets you remap existing blocks to another location at a cost of extra lookup. This is in fact implemented in ZFS as solution for erroneous addition of a vdev, allowing device removal though due to performance cost its oriented mostly at "oops, I added the device wrongly, let me quickly revert".
        dataflow 3y ago
        If by "not able to" you meant "prohibitively expensive" - well, I also don't see why it's prohibitively expensive even without indirection. Moving blocks would seem to be a matter of (a) copy the data, (b) back up the old pointers, (c) update the pointers in-place, (d) mark the block move as committed, (e) and delete the old data/backups. If you crash in the middle you have the backup metadata journaled there to restore from. No indirection. What am I missing? I feel like you might have unstated assumptions somewhere?
        p_l 3y ago