3 ms·

Except for some backups, all of my machines use solid state storage so I don't think there is a point for most people, is there?

charcircuit 1mo ago
Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.
- __d 1mo ago
  Is it possible to determine which sectors are physically sequential given remapping for wear-leveling? Otherwise the claimed defragmentation here is not actually resulting in sequential data.
  - wtallis 1mo ago
    You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
    - kvemkon 1mo ago
      > to a newly-allocated part of the drive's LBA space Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD). https://www.usenix.org/conference/fast24/presentation/jun https://www.usenix.org/conference/fast24/presentation/jun
  - charcircuit 1mo ago
    >Otherwise the claimed defragmentation here is not actually resulting in sequential data. I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
- Groxx 1mo ago
  Prefetching applies to SSDs too, and that's generally sequential in some sense.
- socalgal2 1mo ago
  OH! TIL! Apparently contiguous fragmented ------------------------------------------------------------------- PCIe 4.0 NVMe Read Speed ~5,000 – 7,000 MB/s ~80 – 250 MB/s SATA 3.0 SSD Read Speed ~500 – 550 MB/s ~30 – 60 MB/s Still, given SSDs have a lifespan and given few of my files are that large, I think I'm personally okay without defragmenting. Those numbers are impressive but, most of 10s of thousands of files are source code text files. Asking online what the real-world loss is from not defragmenting > You are giving up virtually 0% to 3% of real-world performance. In daily development, media editing, and standard system use, defragmenting your SSD will yield no human-noticeable speedup.
- shuwix 1mo ago
  [flagged]
  - dspillett 1mo ago
    SSDs often interface with their storage in different (larger) granularities than the OS meaning that even though the data is effectively more random at large scale due to wear leveling there are still benefits to keeping blocks that are sequential in your file sequential in the filesystem also. As well as a potential read boost, the larger blocks introduce a read-before-write issue that can allow multiple close updates. The difference is very small when compared to the differences between solid state and spinning rust based solutions, but definitely measurable, particularly on drives with DRAM cache, and in some cases human-noticable. Actually read the article and you'll see the author specifically talks about this begin part of why this side project started (though I suspect nostalgia is the true driving force as there are more efficient ways of addressing that matter!).
    - shuwix 1mo ago
      [flagged]
      - dspillett 1mo ago
        You pretty being sure of that and not understanding the relevance of the reply, makes me very sure that you are the one without a clue. And you sound like a bit of dick on top. I wish you luck in your future endeavours.
        shuwix 1mo ago
        [flagged]
        dspillett 1mo ago
        > You really don't wish me luck. Oh, but I do. Please note that I do not specify what variety of luck I am wishing. > I really don't need your wishes. You have them anyway. Enjoy. > But you really excel at being butthurt My butt is just fine, thanks. I can send a picture as proof if you are really interested. > Your inability to notice T.R.I.M. I know about TRIM. I also know it is irrelevant to the performance issue being discussed. TRIM works over the SSD's native block size which is much larger (usually at least 512KB, could be several MB depending on controller and memory layout) than the filesystem's granularity (usually 4K or of similar order). The OS can ask the drive to TRIM smaller parts all it likes, and nothing will happen because to do that the drive would need to relocate the rest of the block causing a full (ssd native sized) block READ+WRITE. The issue defragmenting in this instance deals with is how filesystem blocks are arranged within the SSD's blocks: keeping those close by in the same block. The order within the block won't matter, but if due to fragmentation the filesystem blocks of a file are spread through many SSD blocks then reading that file will take longer than if they aren't. TRIM does nothing here, TRIM is not aware of the filesystem layout. > "bit of a dick" Sorry, am I underestimating? Should that have been a lot of a dick?

charcircuit 1mo ago

Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.

__d 1mo ago
Is it possible to determine which sectors are physically sequential given remapping for wear-leveling? Otherwise the claimed defragmentation here is not actually resulting in sequential data.
- wtallis 1mo ago
  You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
  - kvemkon 1mo ago
    > to a newly-allocated part of the drive's LBA space Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD). https://www.usenix.org/conference/fast24/presentation/jun https://www.usenix.org/conference/fast24/presentation/jun
- charcircuit 1mo ago
  >Otherwise the claimed defragmentation here is not actually resulting in sequential data. I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
Groxx 1mo ago
Prefetching applies to SSDs too, and that's generally sequential in some sense.
socalgal2 1mo ago
OH! TIL! Apparently contiguous fragmented ------------------------------------------------------------------- PCIe 4.0 NVMe Read Speed ~5,000 – 7,000 MB/s ~80 – 250 MB/s SATA 3.0 SSD Read Speed ~500 – 550 MB/s ~30 – 60 MB/s Still, given SSDs have a lifespan and given few of my files are that large, I think I'm personally okay without defragmenting. Those numbers are impressive but, most of 10s of thousands of files are source code text files. Asking online what the real-world loss is from not defragmenting > You are giving up virtually 0% to 3% of real-world performance. In daily development, media editing, and standard system use, defragmenting your SSD will yield no human-noticeable speedup.
shuwix 1mo ago
[flagged]
- dspillett 1mo ago
  SSDs often interface with their storage in different (larger) granularities than the OS meaning that even though the data is effectively more random at large scale due to wear leveling there are still benefits to keeping blocks that are sequential in your file sequential in the filesystem also. As well as a potential read boost, the larger blocks introduce a read-before-write issue that can allow multiple close updates. The difference is very small when compared to the differences between solid state and spinning rust based solutions, but definitely measurable, particularly on drives with DRAM cache, and in some cases human-noticable. Actually read the article and you'll see the author specifically talks about this begin part of why this side project started (though I suspect nostalgia is the true driving force as there are more efficient ways of addressing that matter!).
  - shuwix 1mo ago
    [flagged]
    - dspillett 1mo ago
      You pretty being sure of that and not understanding the relevance of the reply, makes me very sure that you are the one without a clue. And you sound like a bit of dick on top. I wish you luck in your future endeavours.
      - shuwix 1mo ago
        [flagged]
        dspillett 1mo ago
        > You really don't wish me luck. Oh, but I do. Please note that I do not specify what variety of luck I am wishing. > I really don't need your wishes. You have them anyway. Enjoy. > But you really excel at being butthurt My butt is just fine, thanks. I can send a picture as proof if you are really interested. > Your inability to notice T.R.I.M. I know about TRIM. I also know it is irrelevant to the performance issue being discussed. TRIM works over the SSD's native block size which is much larger (usually at least 512KB, could be several MB depending on controller and memory layout) than the filesystem's granularity (usually 4K or of similar order). The OS can ask the drive to TRIM smaller parts all it likes, and nothing will happen because to do that the drive would need to relocate the rest of the block causing a full (ssd native sized) block READ+WRITE. The issue defragmenting in this instance deals with is how filesystem blocks are arranged within the SSD's blocks: keeping those close by in the same block. The order within the block won't matter, but if due to fragmentation the filesystem blocks of a file are spread through many SSD blocks then reading that file will take longer than if they aren't. TRIM does nothing here, TRIM is not aware of the filesystem layout. > "bit of a dick" Sorry, am I underestimating? Should that have been a lot of a dick?