4 ms·

5 gigabit/s = 0.625 gigabyte/s = 625 megabyte/s so about 20% overhead... I would not be dissatisfied with 500 MB/s result on a theoretical 625 MB/s bus.

masklinn 5y ago
FWIW it's actually 500MB theoretical (ideal), because USB uses a 8/10 encoding[0], until 3.1 Gen 2 which switches to a 128/132 encoding[1]. So you have a 5Gb/s physical signal, then a 25% encoding overhead, for a raw data throughput of 537MB/s. Once framing and protocol overhead are taken in account, mid-400s effective is probably a good result. [0] https://en.wikipedia.org/wiki/8b/10b_encoding https://en.wikipedia.org/wiki/8b/10b_encoding [1] https://en.wikipedia.org/wiki/64b/66b_encoding https://en.wikipedia.org/wiki/64b/66b_encoding
- amelius 5y ago
  What sucks, however, is that you can transfer 1TB to a USB drive correctly, and then finally have a bit-error that spoils the entire transfer and might even corrupt your drive. Happened to me several times during testing on different machines and with different cables and drives (luckily no real data was lost, but USB is now sort of banned from my office for use with external harddrives). Any sane protocol would be able to deal with such a low error rate, and finish a data transfer without problems.
  - RedShift1 5y ago
    Are you sure it's a USB problem and not something else? Did you reduce all possible variables - ie how scientific were you in determining USB was the problem and not something else?
    - amelius 5y ago
      I think I tested quite thoroughly on very recent Linux versions with Rsync. Of course it is difficult to pinpoint the real culprit with certainty, but USB has no error correction, and that makes it the prime suspect here.
      - amelius 5y ago
        Another bad thing about it is that there is no good way to debug it (afaik).
  - matja 5y ago
    Use a filesystem that does end-to-end error detection and correction. I use ZFS on all of my USB drives (even "real" SATA and M.2 SSDs via adaptors) and it frequently finds checksum errors (and thanks to the design of ZFS, is able to completely recover without any risk of further corruption).
    - amelius 5y ago
      Isn't any filesystem that runs on top of a SSD or flash drive at risk from the SSD/flash drive controller corrupting the disk or making it inaccessible, when the volume is not cleanly unmounted? I can remember losing a USB drive that way, even though it was using a journaling filesystem.
      - paulmd 5y ago
        Potentially yes, realistically no. This is the old "if the storage controller is lying about flushing to disk, there's not a lot you can do about it" problem. ZFS does have defensive mechanisms, like doing a read after write to try and be sure that what is written is actually committed to disk, but if the storage controller chooses to serve that out of cache then that could be a lie too. It's the old "trusting trust" predicament, there's no way for hardware higher in the stack to prove that lower levels aren't simply a tower of lies, only instead of viruses it's flush. That said, in practice very little hardware is actually a giant tower of lies. Flash drives typically do not have enough of a controller to actually cache anything, thus no real capability to lie about writes/etc. SSDs do, but they also generally obey the expected behavior around flushing actually flushing and not just lying about it. RAID controllers and similar are the danger zone, because they may have cache and may lie to the CPU about it on the assumption that it actually will eventually be flushed, and that's the dangerous thing.
        amelius 5y ago
        The question remains: how did I lose an entire ext4 volume after pulling the USB drive from the machine without unmounting it. Is anyone testing for these kinds of things?
  - rasz 5y ago
    This doesnt actually happen (single bit errors getting thru). USB 3.1 data is protected with https://en.wikipedia.org/wiki/Cyclic_redundancy_check#CRC-32_algorithm https://en.wikipedia.org/wiki/Cyclic_redundancy_check#CRC-32... , same CRC as the one used in Ethernet. https://www.rroij.com/open-access/implementation-of-link-layer-of-usb-31.pdf https://www.rroij.com/open-access/implementation-of-link-lay... USB 3 uses CRC-16, still more than good enough for single bit flips.
    - amelius 5y ago
      Interesting. Perhaps it was more than one bit-flip then ... Anyway, it still sucks that I couldn't find the exact reason for the error in the logs.

masklinn 5y ago

FWIW it's actually 500MB theoretical (ideal), because USB uses a 8/10 encoding[0], until 3.1 Gen 2 which switches to a 128/132 encoding[1]. So you have a 5Gb/s physical signal, then a 25% encoding overhead, for a raw data throughput of 537MB/s. Once framing and protocol overhead are taken in account, mid-400s effective is probably a good result. [0] https://en.wikipedia.org/wiki/8b/10b_encoding https://en.wikipedia.org/wiki/8b/10b_encoding [1] https://en.wikipedia.org/wiki/64b/66b_encoding https://en.wikipedia.org/wiki/64b/66b_encoding

amelius 5y ago
What sucks, however, is that you can transfer 1TB to a USB drive correctly, and then finally have a bit-error that spoils the entire transfer and might even corrupt your drive. Happened to me several times during testing on different machines and with different cables and drives (luckily no real data was lost, but USB is now sort of banned from my office for use with external harddrives). Any sane protocol would be able to deal with such a low error rate, and finish a data transfer without problems.
- RedShift1 5y ago
  Are you sure it's a USB problem and not something else? Did you reduce all possible variables - ie how scientific were you in determining USB was the problem and not something else?
  - amelius 5y ago
    I think I tested quite thoroughly on very recent Linux versions with Rsync. Of course it is difficult to pinpoint the real culprit with certainty, but USB has no error correction, and that makes it the prime suspect here.
    - amelius 5y ago
      Another bad thing about it is that there is no good way to debug it (afaik).
- matja 5y ago
  Use a filesystem that does end-to-end error detection and correction. I use ZFS on all of my USB drives (even "real" SATA and M.2 SSDs via adaptors) and it frequently finds checksum errors (and thanks to the design of ZFS, is able to completely recover without any risk of further corruption).
  - amelius 5y ago
    Isn't any filesystem that runs on top of a SSD or flash drive at risk from the SSD/flash drive controller corrupting the disk or making it inaccessible, when the volume is not cleanly unmounted? I can remember losing a USB drive that way, even though it was using a journaling filesystem.
    - paulmd 5y ago
      Potentially yes, realistically no. This is the old "if the storage controller is lying about flushing to disk, there's not a lot you can do about it" problem. ZFS does have defensive mechanisms, like doing a read after write to try and be sure that what is written is actually committed to disk, but if the storage controller chooses to serve that out of cache then that could be a lie too. It's the old "trusting trust" predicament, there's no way for hardware higher in the stack to prove that lower levels aren't simply a tower of lies, only instead of viruses it's flush. That said, in practice very little hardware is actually a giant tower of lies. Flash drives typically do not have enough of a controller to actually cache anything, thus no real capability to lie about writes/etc. SSDs do, but they also generally obey the expected behavior around flushing actually flushing and not just lying about it. RAID controllers and similar are the danger zone, because they may have cache and may lie to the CPU about it on the assumption that it actually will eventually be flushed, and that's the dangerous thing.
      - amelius 5y ago
        The question remains: how did I lose an entire ext4 volume after pulling the USB drive from the machine without unmounting it. Is anyone testing for these kinds of things?
- rasz 5y ago
  This doesnt actually happen (single bit errors getting thru). USB 3.1 data is protected with https://en.wikipedia.org/wiki/Cyclic_redundancy_check#CRC-32_algorithm https://en.wikipedia.org/wiki/Cyclic_redundancy_check#CRC-32... , same CRC as the one used in Ethernet. https://www.rroij.com/open-access/implementation-of-link-layer-of-usb-31.pdf https://www.rroij.com/open-access/implementation-of-link-lay... USB 3 uses CRC-16, still more than good enough for single bit flips.
  - amelius 5y ago
    Interesting. Perhaps it was more than one bit-flip then ... Anyway, it still sucks that I couldn't find the exact reason for the error in the logs.