7 ms·

F3 – Fight Flash Fraud

139 points by CTOSian 2y ago

Animats 2y ago
Somebody should make this up as a handheld device. Something that you can use to inspect new items, and retailers can use to inspect what their suppliers are sending them. Can a Flipper Zero be programmed for this? It connects to Micro SD cards and USB ports.
- RockRobotRock 2y ago
  A retailer wouldn't want to deal with this. They would expect their distributor to weed out the fakes for them. If enough people complain about poor quality drives, you fire them. In my opinion, anyone selling these must be doing it knowingly at this point. Distributors that care probably use something like this: https://www.ureach-inc.com/ https://www.ureach-inc.com/
  - denkmoon 2y ago
    Ah, but we live in the glorious future, where retailers are just fronts for flea-markets filled with third party crap, and claim zero responsibility. You thought you were buying a flash drive from Best Buy? HA, you actually bought it from AHJKHJ Pty Ltd! And no we won't help you!
    - monksy 2y ago
      All while pocketing the listing fee.
  - Wistar 2y ago
    Too bad the ureach fails to provide some means to price or even buy one of their products. At least it doesn’t on iOS Safari.
- cyanydeez 2y ago
  Someone unrmployed should offer to MITM amazon purchases and ensure low priced cards get tested and reshipped or files fraud reports.
  - wakawaka28 2y ago
    This won't work because there are multiple sellers on Amazon. I'd trust the original packaging of the device more than anyone supposedly running these tests for me, especially since they can take hours to complete.
    - bayindirh 2y ago
      Original packaging won't save you. Moreover, dealing with the manufacturer directly won't save you either in some cases[0]. [0]: https://www.bunniestudios.com/blog/on-microsd-problems/ https://www.bunniestudios.com/blog/on-microsd-problems/
      - wakawaka28 2y ago
        Put it this way. I would trust the packaging over someone's word that they opened the packaging and tested it. I would trust it 10x more if I bought from the manufacturer's site.
        bayindirh 2y ago
        Bunnie bought the cards from Kingston directly, in its original packaging (trays). Cards were subpar at best, counterfeit at worst. Kingston exchanged them no questions asked after some pressing. Delidding the cards revealed different components and construction. So no, buying directly from the manufacturer brought no advantages or guarantees. Regardless of the packaging and purchasing channel, I’ll only trust my own test. Nothing else.
        Dalewyn 2y ago
        Despite their good reputation (mostly associated with their RAM), Kingston is just a repackager of third-party parts. They don't actually make any of their products themselves. Therefore, you indeed can't trust their original packaging if they themselves don't vet their supply chain properly. This is in stark contrast to manufacturer-vendors like Samsung, Micron (Crucial), and SanDisk (Western Digital) who manufacture either all or at least the core components of their products and have their own manufacturing reputation on the line.
- pronoiac 2y ago
  > Can a Flipper Zero be programmed for this? It connects to Micro SD cards and USB ports. I wrote something barebones for this on my Synology, using just a shell script and (hardware accelerated) openssl, if memory serves; that acceleration was crucial for handling, say, an 8TB hard drive.
ajb 2y ago
So this always overwrites the device to test it, and I was wondering if that was necessary (say, you had started using it). Presumably you only need to write at most N+1 blocks, where N is the number of blocks the device actually has.l, in order to detect exaggeration. But at that point the fake device will have overwritten all of your files anyway, even if they were theoretically on different blocks (of the exaggerated inventory). So I guess the minimum harm to test a device is to hash all your files, then write at most N+1 (unused) blocks, stopping after each to check if any of your files got harmed. In theory that risks at most one block... Of course, it's better to back up the suspect one.
- ThatPlayer 2y ago
  This also has the option of f3probe: https://fight-flash-fraud.readthedocs.io/en/latest/usage.html#f3probe-the-fastest-drive-test https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm...
- lxgr 2y ago
  > hash all your files, then write at most N+1 (unused) blocks, stopping after each to check if any of your files got harmed That strategy adds O(n^2) reads on top of O(n) writes, though. Even reads don't come for free on modern multi-level cell NAND (due to read disturb), and for just a thousand blocks, you'd end up reading the first block a million times. That's to say nothing of the time this would take.
  - deleted 2y ago
    [deleted]
  - ajb 2y ago
    You're right, this is a non-starter. I should stop posting late at night. The owner of a suspect device just needs to bite the bullet and use a destructive method.
- justinclift 2y ago
  For a new device (ie no existing files on it), wouldn't the simplest approach be to full block 1 (whether 512 or 4k bytes) with a series of "1"'s, block 2 with a series of "2"'s, (etc). ie incrementing the number that gets written as the block number being written to is written. Reading that back (either the full device or a random sample) should pretty quickly identify whether things are still in their expected location.
bagels 2y ago
How long does this tool take to verify a real drive? "it only writes what’s necessary to test the drive" How does that actually work, wouldn't that mean the whole stated capacity would have to be written?
- jtriangle 2y ago
  write 0001, read and confirm, repeat until drive is full. For a fake drive, it'll take awhile, because the underlying storage is much, much slower than it should be, often usb2 speeds. Realistically, this is just a test that satisfies curiosity without opening the drive. It's obvious when you have a fake drive because it won't benchmark anywhere near what it should.
  - CaptainOfCoit 2y ago
    > For a fake drive, it'll take awhile Or, for any mid to large sized real storage. Writing/reading 64GB takes a while, fake or not
  - deleted 2y ago
    [deleted]
  - bagels 2y ago
    From what I understand, some of the fraudulent drives have firmware that just wrap the writes around to the beginning.
    - ungamedplayer 2y ago
      I had or had one in my possession that did exactly this. Another writes above a certain size were simply discarded.
  - sukmaagung 2y ago
    The problem is the fake storage can just overwrite the new data on top existing data. Then you always confirm new data is correct because it broke the old ones. But something like 2TB micro SD when actually it only has 64GB capacity, that will be very long time waiting 2TB to fully written. How about write some file, then verify sometimes the new sometimes the old one, repeat until full. Write(0.h2w) Write(1.h2w) Read(1.h2w) Write(2.h2w) Write(3.h2w) Read(3.h2w) Read(2.h2w) Write(4.h2w) Write(5.h2w) Read(5.h2w) Write(6.h2w) Write(7.h2w) Read(7.h2w) Read(6.h2w) Read(4.h2w) Write(8.h2w) Write(9.h2w) Read(9.h2w) Write(10.h2w) Write(11.h2w) Read(11.h2w) Read(10.h2w) Write(12.h2w) Write(13.h2w) Read(13.h2w) Write(14.h2w) Write(15.h2w) Read(15.h2w) Read(14.h2w) Read(12.h2w) Read(8.h2w) ...Read(0.h2w)
userbinator 2y ago
This appears to be a cross-platform version of h2testw, which is widely recommended for finding the true capacity of a storage device. Another common recommendation is ChipGenius and the various manufacturer-specific tools, which can read the real ID of the NAND ICs, that AFAIK is beyond the ability of nearly all the fakers to change: https://blog.elcomsoft.com/2019/01/identifying-ssd-controller-and-nand-configuration/ https://blog.elcomsoft.com/2019/01/identifying-ssd-controlle... From the research I've done (mainly related to data recovery), the NAND flash industry seems extremely secretive and shady in many ways --- from the near-zero availability of public datasheets, to the many rebrands/"reclaimed"/recycled part sources, to what they're doing to SLC and higher-reliability technologies. There are also ways to determine how worn-out a NAND IC is, but even those may be reversible with the right physical treatments.
- bayindirh 2y ago
  I did some digging in the days of yore just because I was curious about how these things worked, and I found absolutely nothing on how people make these things work. I'm really amazed that unbranded 512GB NVMe drives doesn't randomly eat my data at this point, yet I still can't trust any of these drives w/o file level checksum patrols. So, instead I buy Samsung 9xx drives and use them.
bdcravens 2y ago
Showing my age: based on the title, I assumed it was a ActionScript scanner.
- kazinator 2y ago
  Flash memory was invented by Toshiba in 1980, and marketed starting in 1987. ActionScript dates back to only 1998.
  - Dalewyn 2y ago
    The practical history of flash memory dates back to 1997 with the introduction of MMC by Sandisk. I don't blame anyone for not knowing about Toshiba because they actively detested the technology and shunned its inventor and employee Fujio Masuoka.[1][2] [1]: https://en.wikipedia.org/wiki/Fujio_Masuoka https://en.wikipedia.org/wiki/Fujio_Masuoka [2]: "Toshiba gave Masuoka a few hundred dollar bonus for the invention, and later tried to demote him. But it was American company Intel which made billions of dollars in sales on related technology. Toshiba press department told Forbes that it was Intel that invented flash memory."
notorandit 2y ago
As stated somewhere else, flash fraud has already been committed when you start testing. Once you grab your dubious device, the seller has already got your bucks in exchange of a fake device. You've been already and effectively cheated when those flash devices are being tested against cheats.
- teo_zero 2y ago
  So what? F3 detects frauds, it doesn't guarantee that you can avoid them.
  - derefr 2y ago
    The argument is against the project name: “fighting” flash fraud suggests this project could somehow disincentivize flash fraud. Like how “fighting scammers” means things like taking down scam call centers.
    - schoen 2y ago
      Maybe some consumer protection agency will use the tool in conjunction with random purchases and some kind of enforcement action.
- re 2y ago
  > Once you grab your dubious device, the seller has already got your bucks in exchange of a fake device. Unless you're buying your flash device on the street and paying cash, you likely can return it or initiate a chargeback. And even if you can't undo the purchase, it's better to know whether a device is fraudulent before you start filling it up with real data that you don't want to lose.
- hansvm 2y ago
  There are one or more platforms in the middle. In the US, if you report a fraud your credit card and Amazon will both go out of their way to refund you (Amazon might require a return but will have options to make that free if they do). Other platforms are similar. You, as a consumer, don't have any major problems from the fraud unless you needed a performant device with low delivery times. Whether the fraudster has somebody's dollars varies, but for that kind of a scheme they're able to just hide in plain sight. If 100 people don't test the device (and it works for months or years) and 1 person does, they have a 5-star rating and can just eat the cost of returns. Even if everyone on Hacker News started testing devices it wouldn't make a dent in fraudulent profits.
iam-TJ 2y ago
I wrote a Linux/BASH tester [0] for this recently that includes instructions on how to create simulated fake devices: To create simulated (2GiB) fake devices fallocate -l 1G fff_test.flash DEV=$(losetup --show --find fff_test.flash); echo $DEV DEVNUM=$(stat -c %Hr:%Lr $DEV); echo $DEVNUM With wrap-around sectors: dmsetup create --concise "fff_wrap,,,,0 2097152 linear $DEVNUM 0, 2097152 2097152 linear $DEVNUM 0" With silently dropped writes: dmsetup create --concise "fff_drop,,,,0 2097152 linear $DEVNUM 0, 2097152 2097152 zero" To test: fake_flash_finder.bash /dev/mapper/fff_wrap Capacity mismatch at LBA 2097152, data wrapped around to block 0. Size is most likely really 1073741824 bytes fake_flash_finder.bash /dev/mapper/fff_drop Capacity mismatch at LBA 2097152, data does not match what was written. Size is most likely really 1073741824 bytes To wipe the device if repeating tests: dd if=/dev/zero of=/dev/mapper/fff_wrap bs=64M status=progress conv=fdatasync dd if=/dev/zero of=/dev/mapper/fff_drop bs=64M status=progress conv=fdatasync To remove the device: dmsetup remove fff_wrap dmsetup remove fff_drop losetup --detach "$DEV" rm fff_test.flash [0] https://salsa.debian.org/-/snippets/732 https://salsa.debian.org/-/snippets/732
jwilk 2y ago
Does anyone know how robust it is? Could sophisticated malicious firmware detect that it's being probed by f3 and fake reads such that f3 thinks everything is OK?

139 points by CTOSian 2y ago

Somebody should make this up as a handheld device. Something that you can use to inspect new items, and retailers can use to inspect what their suppliers are sending them. Can a Flipper Zero be programmed for this? It connects to Micro SD cards and USB ports.

RockRobotRock 2y ago
A retailer wouldn't want to deal with this. They would expect their distributor to weed out the fakes for them. If enough people complain about poor quality drives, you fire them. In my opinion, anyone selling these must be doing it knowingly at this point. Distributors that care probably use something like this: https://www.ureach-inc.com/ https://www.ureach-inc.com/
- denkmoon 2y ago
  Ah, but we live in the glorious future, where retailers are just fronts for flea-markets filled with third party crap, and claim zero responsibility. You thought you were buying a flash drive from Best Buy? HA, you actually bought it from AHJKHJ Pty Ltd! And no we won't help you!
  - monksy 2y ago
    All while pocketing the listing fee.
- Wistar 2y ago
  Too bad the ureach fails to provide some means to price or even buy one of their products. At least it doesn’t on iOS Safari.
cyanydeez 2y ago
Someone unrmployed should offer to MITM amazon purchases and ensure low priced cards get tested and reshipped or files fraud reports.
- wakawaka28 2y ago
  This won't work because there are multiple sellers on Amazon. I'd trust the original packaging of the device more than anyone supposedly running these tests for me, especially since they can take hours to complete.
  - bayindirh 2y ago
    Original packaging won't save you. Moreover, dealing with the manufacturer directly won't save you either in some cases[0]. [0]: https://www.bunniestudios.com/blog/on-microsd-problems/ https://www.bunniestudios.com/blog/on-microsd-problems/
    - wakawaka28 2y ago
      Put it this way. I would trust the packaging over someone's word that they opened the packaging and tested it. I would trust it 10x more if I bought from the manufacturer's site.
      - bayindirh 2y ago
        Bunnie bought the cards from Kingston directly, in its original packaging (trays). Cards were subpar at best, counterfeit at worst. Kingston exchanged them no questions asked after some pressing. Delidding the cards revealed different components and construction. So no, buying directly from the manufacturer brought no advantages or guarantees. Regardless of the packaging and purchasing channel, I’ll only trust my own test. Nothing else.
        Dalewyn 2y ago
        Despite their good reputation (mostly associated with their RAM), Kingston is just a repackager of third-party parts. They don't actually make any of their products themselves. Therefore, you indeed can't trust their original packaging if they themselves don't vet their supply chain properly. This is in stark contrast to manufacturer-vendors like Samsung, Micron (Crucial), and SanDisk (Western Digital) who manufacture either all or at least the core components of their products and have their own manufacturing reputation on the line.
pronoiac 2y ago
> Can a Flipper Zero be programmed for this? It connects to Micro SD cards and USB ports. I wrote something barebones for this on my Synology, using just a shell script and (hardware accelerated) openssl, if memory serves; that acceleration was crucial for handling, say, an 8TB hard drive.

ajb 2y ago

So this always overwrites the device to test it, and I was wondering if that was necessary (say, you had started using it). Presumably you only need to write at most N+1 blocks, where N is the number of blocks the device actually has.l, in order to detect exaggeration. But at that point the fake device will have overwritten all of your files anyway, even if they were theoretically on different blocks (of the exaggerated inventory). So I guess the minimum harm to test a device is to hash all your files, then write at most N+1 (unused) blocks, stopping after each to check if any of your files got harmed. In theory that risks at most one block... Of course, it's better to back up the suspect one.

ThatPlayer 2y ago
This also has the option of f3probe: https://fight-flash-fraud.readthedocs.io/en/latest/usage.html#f3probe-the-fastest-drive-test https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm...
lxgr 2y ago
> hash all your files, then write at most N+1 (unused) blocks, stopping after each to check if any of your files got harmed That strategy adds O(n^2) reads on top of O(n) writes, though. Even reads don't come for free on modern multi-level cell NAND (due to read disturb), and for just a thousand blocks, you'd end up reading the first block a million times. That's to say nothing of the time this would take.
- deleted 2y ago
  [deleted]
- ajb 2y ago
  You're right, this is a non-starter. I should stop posting late at night. The owner of a suspect device just needs to bite the bullet and use a destructive method.
justinclift 2y ago
For a new device (ie no existing files on it), wouldn't the simplest approach be to full block 1 (whether 512 or 4k bytes) with a series of "1"'s, block 2 with a series of "2"'s, (etc). ie incrementing the number that gets written as the block number being written to is written. Reading that back (either the full device or a random sample) should pretty quickly identify whether things are still in their expected location.

bagels 2y ago

How long does this tool take to verify a real drive? "it only writes what’s necessary to test the drive" How does that actually work, wouldn't that mean the whole stated capacity would have to be written?

jtriangle 2y ago
write 0001, read and confirm, repeat until drive is full. For a fake drive, it'll take awhile, because the underlying storage is much, much slower than it should be, often usb2 speeds. Realistically, this is just a test that satisfies curiosity without opening the drive. It's obvious when you have a fake drive because it won't benchmark anywhere near what it should.
- CaptainOfCoit 2y ago
  > For a fake drive, it'll take awhile Or, for any mid to large sized real storage. Writing/reading 64GB takes a while, fake or not
- deleted 2y ago
  [deleted]
- bagels 2y ago
  From what I understand, some of the fraudulent drives have firmware that just wrap the writes around to the beginning.
  - ungamedplayer 2y ago
    I had or had one in my possession that did exactly this. Another writes above a certain size were simply discarded.
- sukmaagung 2y ago
  The problem is the fake storage can just overwrite the new data on top existing data. Then you always confirm new data is correct because it broke the old ones. But something like 2TB micro SD when actually it only has 64GB capacity, that will be very long time waiting 2TB to fully written. How about write some file, then verify sometimes the new sometimes the old one, repeat until full. Write(0.h2w) Write(1.h2w) Read(1.h2w) Write(2.h2w) Write(3.h2w) Read(3.h2w) Read(2.h2w) Write(4.h2w) Write(5.h2w) Read(5.h2w) Write(6.h2w) Write(7.h2w) Read(7.h2w) Read(6.h2w) Read(4.h2w) Write(8.h2w) Write(9.h2w) Read(9.h2w) Write(10.h2w) Write(11.h2w) Read(11.h2w) Read(10.h2w) Write(12.h2w) Write(13.h2w) Read(13.h2w) Write(14.h2w) Write(15.h2w) Read(15.h2w) Read(14.h2w) Read(12.h2w) Read(8.h2w) ...Read(0.h2w)

userbinator 2y ago

This appears to be a cross-platform version of h2testw, which is widely recommended for finding the true capacity of a storage device. Another common recommendation is ChipGenius and the various manufacturer-specific tools, which can read the real ID of the NAND ICs, that AFAIK is beyond the ability of nearly all the fakers to change: https://blog.elcomsoft.com/2019/01/identifying-ssd-controller-and-nand-configuration/ https://blog.elcomsoft.com/2019/01/identifying-ssd-controlle... From the research I've done (mainly related to data recovery), the NAND flash industry seems extremely secretive and shady in many ways --- from the near-zero availability of public datasheets, to the many rebrands/"reclaimed"/recycled part sources, to what they're doing to SLC and higher-reliability technologies. There are also ways to determine how worn-out a NAND IC is, but even those may be reversible with the right physical treatments.

bayindirh 2y ago
I did some digging in the days of yore just because I was curious about how these things worked, and I found absolutely nothing on how people make these things work. I'm really amazed that unbranded 512GB NVMe drives doesn't randomly eat my data at this point, yet I still can't trust any of these drives w/o file level checksum patrols. So, instead I buy Samsung 9xx drives and use them.

bdcravens 2y ago

Showing my age: based on the title, I assumed it was a ActionScript scanner.

kazinator 2y ago
Flash memory was invented by Toshiba in 1980, and marketed starting in 1987. ActionScript dates back to only 1998.
- Dalewyn 2y ago
  The practical history of flash memory dates back to 1997 with the introduction of MMC by Sandisk. I don't blame anyone for not knowing about Toshiba because they actively detested the technology and shunned its inventor and employee Fujio Masuoka.[1][2] [1]: https://en.wikipedia.org/wiki/Fujio_Masuoka https://en.wikipedia.org/wiki/Fujio_Masuoka [2]: "Toshiba gave Masuoka a few hundred dollar bonus for the invention, and later tried to demote him. But it was American company Intel which made billions of dollars in sales on related technology. Toshiba press department told Forbes that it was Intel that invented flash memory."

notorandit 2y ago

As stated somewhere else, flash fraud has already been committed when you start testing. Once you grab your dubious device, the seller has already got your bucks in exchange of a fake device. You've been already and effectively cheated when those flash devices are being tested against cheats.

teo_zero 2y ago
So what? F3 detects frauds, it doesn't guarantee that you can avoid them.
- derefr 2y ago
  The argument is against the project name: “fighting” flash fraud suggests this project could somehow disincentivize flash fraud. Like how “fighting scammers” means things like taking down scam call centers.
  - schoen 2y ago
    Maybe some consumer protection agency will use the tool in conjunction with random purchases and some kind of enforcement action.
re 2y ago
> Once you grab your dubious device, the seller has already got your bucks in exchange of a fake device. Unless you're buying your flash device on the street and paying cash, you likely can return it or initiate a chargeback. And even if you can't undo the purchase, it's better to know whether a device is fraudulent before you start filling it up with real data that you don't want to lose.
hansvm 2y ago
There are one or more platforms in the middle. In the US, if you report a fraud your credit card and Amazon will both go out of their way to refund you (Amazon might require a return but will have options to make that free if they do). Other platforms are similar. You, as a consumer, don't have any major problems from the fraud unless you needed a performant device with low delivery times. Whether the fraudster has somebody's dollars varies, but for that kind of a scheme they're able to just hide in plain sight. If 100 people don't test the device (and it works for months or years) and 1 person does, they have a 5-star rating and can just eat the cost of returns. Even if everyone on Hacker News started testing devices it wouldn't make a dent in fraudulent profits.

iam-TJ 2y ago

I wrote a Linux/BASH tester [0] for this recently that includes instructions on how to create simulated fake devices: To create simulated (2GiB) fake devices fallocate -l 1G fff_test.flash DEV=$(losetup --show --find fff_test.flash); echo $DEV DEVNUM=$(stat -c %Hr:%Lr $DEV); echo $DEVNUM With wrap-around sectors: dmsetup create --concise "fff_wrap,,,,0 2097152 linear $DEVNUM 0, 2097152 2097152 linear $DEVNUM 0" With silently dropped writes: dmsetup create --concise "fff_drop,,,,0 2097152 linear $DEVNUM 0, 2097152 2097152 zero" To test: fake_flash_finder.bash /dev/mapper/fff_wrap Capacity mismatch at LBA 2097152, data wrapped around to block 0. Size is most likely really 1073741824 bytes fake_flash_finder.bash /dev/mapper/fff_drop Capacity mismatch at LBA 2097152, data does not match what was written. Size is most likely really 1073741824 bytes To wipe the device if repeating tests: dd if=/dev/zero of=/dev/mapper/fff_wrap bs=64M status=progress conv=fdatasync dd if=/dev/zero of=/dev/mapper/fff_drop bs=64M status=progress conv=fdatasync To remove the device: dmsetup remove fff_wrap dmsetup remove fff_drop losetup --detach "$DEV" rm fff_test.flash [0] https://salsa.debian.org/-/snippets/732 https://salsa.debian.org/-/snippets/732

jwilk 2y ago

Does anyone know how robust it is? Could sophisticated malicious firmware detect that it's being probed by f3 and fake reads such that f3 thinks everything is OK?