5 ms·

I wish there was some kind of builtin tool in mainstream operating system to validate the real capacity of an external storage. Right now I use h2testw on all

by m-p-3 4y ago

I wish there was some kind of builtin tool in mainstream operating system to validate the real capacity of an external storage. Right now I use h2testw on all storage I purchase, but you kinda need to know this is a thing.

mort96 4y ago
I suppose the issue is that the only real way of doing that is to write as much random garbage as the drive supports and then verify that you can read it all back again. That should probably be a thing, but it would be incredibly slow and waste your limited write cycles. The combination of regulations and buying from reputable sellers should be enough IMO... though most people would probably view Walmart as a reputable seller, so that clearly doesn't work.
- unwind 4y ago
  Yeah, and it's not enough to do e.g. d = random_block() for block in capacity: block.write(d) if block.read() != d: fail("fake disk") Since that would be fooled by a device mapping all reported blocks to the same actual storage. I guess you basically have to write unique data to all reported blocks first, then read them all back to verify. Since storage devices are often bigger than RAM, the data is probably best to generate procedurally (but deterministically) from the block number. That sounds like fun!
  - pronoiac 4y ago
    Ooh, I just did something with this, and I should finish the draft blog post for it: * write encrypted zeroes * decrypt it and count the zeroes I think cbc mode is enough. I see a possible computational loophole, which I want to check before I post.
    - mort96 4y ago
      You could do that, but doesn't it seem easier to: 1) initialize a CSPRNG to a known seed; 2) generate random numbers and write them sequentially; 3) initialize a new RNG to the same seed and compare numbers you generate to numbers you read from the storage? That way, you even get to point at the exact byte where the first divergence happens, which might let you say, "this drive which claims to be 30TB seems to be 1GB".
      - pronoiac 4y ago
        I think that would avoid the potential loophole I was worried about! I'd be careful with the periodicity of the RNG though. I was on a machine with limited user space. I used OpenSSL, no new binaries, reading and writing at close-to-wire speeds. "Counting zeroes" would track down where they don't match, doesn't it?
        NavinF 4y ago
        CSPRNGs don’t have that issue.
      - ThatPlayer 4y ago
        Once you find the exact byte where you know the storage ends, you can actually format the drive it's completely usable. F3 can do this with f3probe and f3fix. https://fight-flash-fraud.readthedocs.io/en/latest/usage.html#how-to-fix-a-fake-drive https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm... I've used it on a 2GB microSD that came with a 3d printer that turned out to be 100MB. I continued to use it to transport files to my 3d printer for a bit with this.
  - ThatPlayer 4y ago
    I believe f3 (Fight Flash Fraud) already does random numbers with a preset seed: https://github.com/AltraMayor/f3/blob/master/f3read.c#L144 https://github.com/AltraMayor/f3/blob/master/f3read.c#L144
    - robocat 4y ago
      So they put that code on the drive controller, detect when f3 is being used, and generate the pattern dynamically in the drive controller.
      - ThatPlayer 4y ago
        Yes, that would be possible, but that would require effort on the counterfeiter's part. Any additional hardware in the drive controller is going to reduce the counterfeiter's profits. So why would they bother? H2testw, which f3's algorithm is based upon, is more than a decade old at this point with the same RNG generator and no one has made such a drive controller to trick it.
- ThatPlayer 4y ago
  I've been using f3probe rather than full f3: https://fight-flash-fraud.readthedocs.io/en/latest/usage.html#f3probe-the-fastest-drive-test https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm... The documentation does mention it is possible for more advanced fake firmware to bypass this, but I don't think anyone has bothered yet. It's labelled experimental because of that, but I believe that was at least 5 years ago.
PeterisP 4y ago
Yesterday I read another analysis of what seems to be the same drive which tried to verify the real capacity, but the issue was that the drive was throttled to a low-speed connection so that validating it by writing and re-reading the full volume of data would take something like 250 days. So it's kind of obvious that it was bad, but you can't easily prove it in a reasonable time.

by m-p-3 4y ago

mort96 4y ago

I suppose the issue is that the only real way of doing that is to write as much random garbage as the drive supports and then verify that you can read it all back again. That should probably be a thing, but it would be incredibly slow and waste your limited write cycles. The combination of regulations and buying from reputable sellers should be enough IMO... though most people would probably view Walmart as a reputable seller, so that clearly doesn't work.

unwind 4y ago
Yeah, and it's not enough to do e.g. d = random_block() for block in capacity: block.write(d) if block.read() != d: fail("fake disk") Since that would be fooled by a device mapping all reported blocks to the same actual storage. I guess you basically have to write unique data to all reported blocks first, then read them all back to verify. Since storage devices are often bigger than RAM, the data is probably best to generate procedurally (but deterministically) from the block number. That sounds like fun!
- pronoiac 4y ago
  Ooh, I just did something with this, and I should finish the draft blog post for it: * write encrypted zeroes * decrypt it and count the zeroes I think cbc mode is enough. I see a possible computational loophole, which I want to check before I post.
  - mort96 4y ago
    You could do that, but doesn't it seem easier to: 1) initialize a CSPRNG to a known seed; 2) generate random numbers and write them sequentially; 3) initialize a new RNG to the same seed and compare numbers you generate to numbers you read from the storage? That way, you even get to point at the exact byte where the first divergence happens, which might let you say, "this drive which claims to be 30TB seems to be 1GB".
    - pronoiac 4y ago
      I think that would avoid the potential loophole I was worried about! I'd be careful with the periodicity of the RNG though. I was on a machine with limited user space. I used OpenSSL, no new binaries, reading and writing at close-to-wire speeds. "Counting zeroes" would track down where they don't match, doesn't it?
      - NavinF 4y ago
        CSPRNGs don’t have that issue.
    - ThatPlayer 4y ago
      Once you find the exact byte where you know the storage ends, you can actually format the drive it's completely usable. F3 can do this with f3probe and f3fix. https://fight-flash-fraud.readthedocs.io/en/latest/usage.html#how-to-fix-a-fake-drive https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm... I've used it on a 2GB microSD that came with a 3d printer that turned out to be 100MB. I continued to use it to transport files to my 3d printer for a bit with this.
- ThatPlayer 4y ago
  I believe f3 (Fight Flash Fraud) already does random numbers with a preset seed: https://github.com/AltraMayor/f3/blob/master/f3read.c#L144 https://github.com/AltraMayor/f3/blob/master/f3read.c#L144
  - robocat 4y ago
    So they put that code on the drive controller, detect when f3 is being used, and generate the pattern dynamically in the drive controller.
    - ThatPlayer 4y ago
      Yes, that would be possible, but that would require effort on the counterfeiter's part. Any additional hardware in the drive controller is going to reduce the counterfeiter's profits. So why would they bother? H2testw, which f3's algorithm is based upon, is more than a decade old at this point with the same RNG generator and no one has made such a drive controller to trick it.
ThatPlayer 4y ago
I've been using f3probe rather than full f3: https://fight-flash-fraud.readthedocs.io/en/latest/usage.html#f3probe-the-fastest-drive-test https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm... The documentation does mention it is possible for more advanced fake firmware to bypass this, but I don't think anyone has bothered yet. It's labelled experimental because of that, but I believe that was at least 5 years ago.

PeterisP 4y ago

Yesterday I read another analysis of what seems to be the same drive which tried to verify the real capacity, but the issue was that the drive was throttled to a low-speed connection so that validating it by writing and re-reading the full volume of data would take something like 250 days. So it's kind of obvious that it was bad, but you can't easily prove it in a reasonable time.