5 ms·

Beautiful. Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It mi

Beautiful. Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years... But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.

doublepg23 1mo ago
only HDD drives I use are in a ZFS pool which has a very usable fragmentation value from zpool.
- Modified3019 1mo ago
  In case you (or others aren’t aware) the FRAG value ZFS shows, is an arbitrary calculation to help give a value to free space fragmentation. The purpose is to have a sense of when ZFS may developing trouble quickly finding contiguous blocks of free space to dump new data into. Mostly only a concern for high active and highly full pools. When you start watching the number climb past double digits, you may start finding performance issues but it’s a very contextual thing, rather than “at x value it’s bad”. Basically it has nothing to do with written file fragmentation, or potential read speed. Because ZFS is a copy on write filsystem, attempts at defragmentation an active pool are generally not effective. An exception would be a pool that is full of large files that was written in a non contiguous way (like torrents), in which case copying the data to a new dataset and deleting the old roughly achieves the purpose. Or you could use something like https://github.com/salesforce/zfs_defrag https://github.com/salesforce/zfs_defrag
  - doublepg23 1mo ago
    Wow! I did not know that, thank you for the info. Indeed torrents were the most obvious cause of the fragmentation I saw.
throwaway17_17 1mo ago
I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives. I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.
- prmoustache 1mo ago
  > My partner complains about me closing apps on my phone consistently. I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?
  - snailmailman 1mo ago
    At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that. Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
    - charcircuit 1mo ago
      iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.
      - snailmailman 1mo ago
        Well, it closes the apps somewhat arbitrarily, but they definitely arent all open. If the most recent app is a resource-intensive game or something it will more aggressively close things. but you can have more open if they are all lightweight apps. But the switcher shows every app ever opened and tries to pretend these apps aren't being closed. Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months
socalgal2 1mo ago
Except for some backups, all of my machines use solid state storage so I don't think there is a point for most people, is there?
- charcircuit 1mo ago
  Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.
  - __d 1mo ago
    Is it possible to determine which sectors are physically sequential given remapping for wear-leveling? Otherwise the claimed defragmentation here is not actually resulting in sequential data.
    - wtallis 1mo ago
      You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
      - kvemkon 1mo ago
        > to a newly-allocated part of the drive's LBA space Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD). https://www.usenix.org/conference/fast24/presentation/jun https://www.usenix.org/conference/fast24/presentation/jun
ErroneousBosh 1mo ago
If it makes it feel faster, it makes it feel faster. It's like those copper bangles with magnets that people swear makes their joints less creaky. My example is a daith piercing, that's when you pierce a ring through a fold of cartilage in a certain spot inside your ear. It's supposed to stop you getting migraines, but there's no sensible mechanism for this to work. It's all woo and bunkum, apparently. But I've had four migraines in eight years as opposed to four every month. So, if it feels faster, it might well be.
gbin 1mo ago
I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs in copper so this is not helping at all. This is on a slow device but it might be worse on a fast one as the extra allocation of small extents start to hit harder on the host side: (fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%) Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.

doublepg23 1mo ago

only HDD drives I use are in a ZFS pool which has a very usable fragmentation value from zpool.

Modified3019 1mo ago
In case you (or others aren’t aware) the FRAG value ZFS shows, is an arbitrary calculation to help give a value to free space fragmentation. The purpose is to have a sense of when ZFS may developing trouble quickly finding contiguous blocks of free space to dump new data into. Mostly only a concern for high active and highly full pools. When you start watching the number climb past double digits, you may start finding performance issues but it’s a very contextual thing, rather than “at x value it’s bad”. Basically it has nothing to do with written file fragmentation, or potential read speed. Because ZFS is a copy on write filsystem, attempts at defragmentation an active pool are generally not effective. An exception would be a pool that is full of large files that was written in a non contiguous way (like torrents), in which case copying the data to a new dataset and deleting the old roughly achieves the purpose. Or you could use something like https://github.com/salesforce/zfs_defrag https://github.com/salesforce/zfs_defrag
- doublepg23 1mo ago
  Wow! I did not know that, thank you for the info. Indeed torrents were the most obvious cause of the fragmentation I saw.

throwaway17_17 1mo ago

I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives. I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.

prmoustache 1mo ago
> My partner complains about me closing apps on my phone consistently. I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?
- snailmailman 1mo ago
  At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that. Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
  - charcircuit 1mo ago
    iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.
    - snailmailman 1mo ago
      Well, it closes the apps somewhat arbitrarily, but they definitely arent all open. If the most recent app is a resource-intensive game or something it will more aggressively close things. but you can have more open if they are all lightweight apps. But the switcher shows every app ever opened and tries to pretend these apps aren't being closed. Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months

socalgal2 1mo ago

Except for some backups, all of my machines use solid state storage so I don't think there is a point for most people, is there?

charcircuit 1mo ago
Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.
- __d 1mo ago
  Is it possible to determine which sectors are physically sequential given remapping for wear-leveling? Otherwise the claimed defragmentation here is not actually resulting in sequential data.
  - wtallis 1mo ago
    You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
    - kvemkon 1mo ago
      > to a newly-allocated part of the drive's LBA space Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD). https://www.usenix.org/conference/fast24/presentation/jun https://www.usenix.org/conference/fast24/presentation/jun

ErroneousBosh 1mo ago

If it makes it feel faster, it makes it feel faster. It's like those copper bangles with magnets that people swear makes their joints less creaky. My example is a daith piercing, that's when you pierce a ring through a fold of cartilage in a certain spot inside your ear. It's supposed to stop you getting migraines, but there's no sensible mechanism for this to work. It's all woo and bunkum, apparently. But I've had four migraines in eight years as opposed to four every month. So, if it feels faster, it might well be.

gbin 1mo ago

I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs in copper so this is not helping at all. This is on a slow device but it might be worse on a fast one as the extra allocation of small extents start to hit harder on the host side: (fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%) Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.