6 ms·

Testing an SD Card's true capacity

159 points by alecjw 9y ago

cjnicholls 9y ago
I had this a year or so ago from Amazon on a Black Friday deal. A simple I/O test is usually enough to confirm a fake. The sellers are aware of their stock being fake, full refund without return and reported it to Amazon. Ebay have supporting documentation about finding fakes on their platforms and testing authenticity. [0] http://www.ebay.co.uk/gds/Spotting-Counterfeit-fake-SD-Micro-SD-USB-Memory-Cards-/10000000175807852/g.html http://www.ebay.co.uk/gds/Spotting-Counterfeit-fake-SD-Micro...
- kelchm 9y ago
  This is made a lot worse at Amazon because of the commingling of stock from different sellers.
  - deleted 9y ago
    [deleted]
  - cjnicholls 9y ago
    Thats true but Amazon needs to create a more transparent reporting system that is buyer centered such as seller rating rather than just item. This was also highlighted with Apple chargers so Amazon has a problem with no 'solution' to preventing it. (https://www.forbes.com/sites/amitchowdhry/2016/10/23/apple-nearly-90-of-genuine-iphone-chargers-on-amazon-are-fake/#7cdf7a595b07 https://www.forbes.com/sites/amitchowdhry/2016/10/23/apple-n...)
  - JohnTHaller 9y ago
    And the fact that Amazon will often block reviews that mention receiving a counterfeit or fake product.
amelius 9y ago
Title should be: Testing for a fake SD card
throwanem 9y ago
Stock Wordpress takes it hard when HN comes around! The Internet Archive has it: http://web.archive.org/web/20170711105235/blog.nothinguntoward.eu/?p=238 http://web.archive.org/web/20170711105235/blog.nothinguntowa...
- ship_it 9y ago
  True, thanks for the link. Shame that author uses Wordpress. I'd advice him to use a static page generator for the future.
  - throwanem 9y ago
    I use Wordpress, too, because I'm more interested in publishing content than in dinking around endlessly with the latest fad in content management tooling.
    - ue_ 9y ago
      To be fair, static blog generators have been around for years, and their use is hardly a fad, considering the number of blogs that people host on github etc. It hasn't caught on as much among non-tech types, but it's certainly not a fad, and the solutions are mature, usable and open source. I can understand wanting various features that only a dynamic blog offers (like commenting without Javascript), and I run such a blog myself.
      - y4mi 9y ago
        There are tools for that as well. Staticman for example.
  - 18nleung 9y ago
    Wordpress is still essentially the status quo outside of technical circles and for non-technical people I'd recommend it over most other static site generators.
    - 9y ago
polote 9y ago
What is the percentage of SD cards which are not fake ? Are we sure that popular SD cards on Amazon are not fake ?
- notyourday 9y ago
  I do not know about Amazon but I have some interesting experiences: Over last several years I have ordered a few hundred SD Cards, some from Ebay some from Amazon, some bought at the stores. Store cards were Kingston and PNY branded - bought at Microcenter. Amazon were some unbranded ones and Sandisks. My method of testing is rather simple - for 64G card generate a file of 63864569856 bytes of random data, dd it onto a card dd result off the card, fingerprint both. If they match the card is probably good. This is what I found: * cards with fake sizes will fail to write entire file with IO error * cards with questionable quality will happily spew the following into kernel log: [6135586.258783] sd 7:0:0:2: [sdf] tag#0 CDB: Read(10) 28 00 07 6f 4f 80 00 00 08 00 [6135586.258785] blk_update_request: I/O error, dev sdf, sector 124735360 [6135586.654749] sd 7:0:0:2: [sdf] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [6135586.654752] sd 7:0:0:2: [sdf] tag#0 Sense Key : Hardware Error [current] [6135586.654753] sd 7:0:0:2: [sdf] tag#0 Add. Sense: No additional sense information [6135586.654755] sd 7:0:0:2: [sdf] tag#0 CDB: Read(10) 28 00 07 6f 4f 80 00 00 08 00 [6135586.654757] blk_update_request: I/O error, dev sdf, sector 124735360 [6135586.654760] Buffer I/O error on dev sdf, logical block 15591920, async page read * Sandisk had the lowest number of the failures when bought on ebay - sub 10%, highest rate of failures when bought on Amazon - over 90%. * Kingstons from Micro Center passed the tests. * PNYs failed the test. The only wrinkle that I currently have is that I use Sandisks in a dual card Nikon configured for dual write. So far two of the cards that passed the tests created a "Card error" reported by the camera on separate occasions with separate slots. I do not however know if the issue is caused by the the adapter or by the card. I just ordered a few new adapters to see if I can isolate it. [Edit: Formatting] [Late addition: below] The cards that triggered an error in camera passed the read/write test after the error so I am inclined to believe it could be the adapter.
  - qb45 9y ago
    > My method of testing is rather simple - for 64G card generate a file of 63864569856 bytes of random data, dd it onto a card dd result off the card, fingerprint both. If they match the card is probably good. One optimization: dd bs=1M count=N if=/dev/urandom | tee /dev/mmcblk0 | md5sum md5sum /dev/mmcblk0
woliveirajr 9y ago
There are some good programs already written to test your Sd / flash card / USB / whatever you want: USB Flash Drive Tester: http://www2.vconsole.com/client/?page=page&id=13 http://www2.vconsole.com/client/?page=page&id=13 H2testw: https://www.portablefreeware.com/?id=1519 https://www.portablefreeware.com/?id=1519
londons_explore 9y ago
A pretty big chunk of memory cards are fake. "kind" fakes show write errors when writing data. Evil fakes just return garbage data. I have even witnessed a few fakes which understand a fat32 filesystem and make sure the filesystem data structures and the first and last few kilobytes of every file get mapped to working flash so that a chkdsk won't pick up issues and browsing the files shows no obvious issue. Many cameras store jpeg thumbnails, and the thumbnails continue to show too.
- LoSboccacc 9y ago
  that's not just bad that's intentionally evil
- milankragujevic 9y ago
  Could it be that a fake SD card would make all the pictures in it Black or garbled? I had that happen to me, but it was a 2GB SD card from 2007...
  - londons_explore 9y ago
    Sure. Fakes like that have been around for almost as long as sd cards themselves.
- TazeTSchnitzel 9y ago
  Then there's the fakes that modulo the address. Write beyond the end of the actual flash and you'll write over the start!
  - mikeash 9y ago
    Seems like that would be the simplest way to design one. Just leave off some of the high address lines.
CaptainZapp 9y ago
Why buy an SD card of ebay or, alternatively, find a deal which is too good to be true (and is neither good, nor true, of course)? A branded, well specced SD card is not really that expensive that I would risk - for example - my photos which are certainly much more valuable to me than a few $ saved. Even if it's in the 10s $ range. I honestly just don't get it.
- kawsper 9y ago
  I don't trust any vendor anymore. I have managed to get fake SD-cards from a local retailer, Amazon, a webshop, and big supermarket chain. It seems like the distribution channels are poisoned.
  - otterpro 9y ago
    I bought a SanDisk SD card from BestBuy (at physical store), trusting that it was genuine, so I never ran any test. Only after a year or so, I think it might possibly be a fake, as the speed was incredibly slow - write speed was slower than my 7 year old SD card.
- ptero 9y ago
  How you do you purchase cards to guarantee you are not getting a fake? I believe fakes are pretty much everywhere now, in branded stores as well. Maybe buying those from stores that serve people who care helps (e.g., B&H photo -- as you indicate, a pro photographer losing his photos would be furious). They try to act on problems and work with suppliers quickly, but even this is not a guarantee. A better approach IMO is to test the card first using easily available tools and use backups. This is more work, but can save you a major aggravation. My 2c.
  - CaptainZapp 9y ago
    I reply to you, but mean both of you who replied to my comment. Thanks! I was really not aware that this is a problem and will likely test my cards in trhe future. This is atrocious.
  - 9y ago
ehPReth 9y ago
I've been using F3 (http://oss.digirati.com.br/f3/ http://oss.digirati.com.br/f3/) to test new SD cards. Installable via homebrew and seems to work quite well
- randkyp 9y ago
  Yes. I just run f3probe for a quick capacity & speed test and then run a single-pass, random data badblocks test just to be sure. If there are any 'bad' sectors, I just pass the badblocks output to mkfs. Be careful with your choice of card reader/adapter though--some starts to get too hot and end up breaking the card under heavy testing!
wingi 9y ago
Hmmm, some start writingGB17 on the position of the GB1, so your 1MB read/test will make all green.
lima 9y ago
About the sync: you can bypass the fs cache by specifying iflag=direct/oflag=direct.
- qb45 9y ago
  In fact, I'm quite surprised that it worked at all without direct. Even if sync writes the cached data to disk, it isn't exactly guaranteed that the following read will read them back instead of just returning a copy from the cache.
- alecjw 9y ago
  Nice, thanks for the tip!
dis-sys 9y ago
nothing special here, just a 16GB card got marked as 64GB, this kind of scam has been there for ages. You got hit by dishonest marketing BS from big brands all the time as well, buying from reliable brand and its recommended (overpriced) channel doesn't solve the problem, it seem to become the standard way of doing business in storage (what is the likelihood to get a fake RAM/CPU/GPU to this extent?). I think that is actually the most scary part. I was lately trying to buy a new SSD, surely you want a NVMe SSD, so I picked up the Pro version of this top brand SSD, as the spec is the best of the best you can find on the market. Did a simple test, fsync performance is actually worse than the 2 years old SATA SSD from the same brand and that SATA was a EVO module. Sure, they didn't advertise fsync performance, but do you think it is fair not to mention/disclose/report such very poor fsync performance when it is actually slower than their own 2 years old product? I didn't waste my time to argue with anyone, got it refunded and bought the EVO version of the NVMe SSD as I thought I just forget about the fsync performance and save that $100 difference. Got the fancy EVO NVMe SSD, did a sequential write test, the throughput performance dropped like 40% after certain amount of writes. Carefully checked the product spec, there is some text on the box written in some size 2 font basically telling people to expect such performance drop as there is some cache style design there. Well, at least I got this very good random IOPS spec, right? Did some tests myself, guess what? The advertised IOPS performance is only true when running in situations considered as rare or even abnormal for such consumer grade SSD - you need a very long IO queue depth. In plain English, if you just buy this consumer grade SSD and use it in a consumer setting, you are not going to get the advertised performance. Fake or marketing BS? If the whole idea is to fool me to pay for their over stated stuff, then what is the difference?
- ve7cxz 9y ago
  The issue is that with these cards they're not just branded wrong, they often present themselves as being bigger than they actually are - so your OS sees it as being 64GB when it's only 16GB, for example. Were it just a branding thing it wouldn't - from a technical standpoint - be quite so bad, other than you know, the intentional lying...
deleted 9y ago
[deleted]
TazeTSchnitzel 9y ago
At least for USB flash drives that behave similarly, sometimes there are software tools that let you configure the drive's claimed capacity.
- jaclaz 9y ago
  Yes and no. In the sense that many "manufacturer tools" for USB sticks (and controllers) are available (sometimes released by he actual manufacturers, sometimes released by some "OEM" or "Brands" as a recovery or formatting software, the most part AFAICT leaked), while there are very few (if any at all) correspondent tools for SD cards. Here is a couple known articles by "Bunnie" of BunnieStudios, respectively 2013 and 2009: https://www.bunniestudios.com/blog/?p=3554 https://www.bunniestudios.com/blog/?p=3554 http://www.bunniestudios.com/blog/?page_id=1022 http://www.bunniestudios.com/blog/?page_id=1022
zeep 9y ago
I've received some fake ones but they were unbranded... The ones that I ordered online on Amazon with the whole retail packaging have always been good so far. Sounds like something an evil cloud storage company would do to promote the use of online storage... that and removing SD cards slot from whatever they can.

159 points by alecjw 9y ago

I had this a year or so ago from Amazon on a Black Friday deal. A simple I/O test is usually enough to confirm a fake. The sellers are aware of their stock being fake, full refund without return and reported it to Amazon. Ebay have supporting documentation about finding fakes on their platforms and testing authenticity. [0] http://www.ebay.co.uk/gds/Spotting-Counterfeit-fake-SD-Micro-SD-USB-Memory-Cards-/10000000175807852/g.html http://www.ebay.co.uk/gds/Spotting-Counterfeit-fake-SD-Micro...

kelchm 9y ago
This is made a lot worse at Amazon because of the commingling of stock from different sellers.
- deleted 9y ago
  [deleted]
- cjnicholls 9y ago
  Thats true but Amazon needs to create a more transparent reporting system that is buyer centered such as seller rating rather than just item. This was also highlighted with Apple chargers so Amazon has a problem with no 'solution' to preventing it. (https://www.forbes.com/sites/amitchowdhry/2016/10/23/apple-nearly-90-of-genuine-iphone-chargers-on-amazon-are-fake/#7cdf7a595b07 https://www.forbes.com/sites/amitchowdhry/2016/10/23/apple-n...)
- JohnTHaller 9y ago
  And the fact that Amazon will often block reviews that mention receiving a counterfeit or fake product.

amelius 9y ago

Title should be: Testing for a fake SD card

throwanem 9y ago

Stock Wordpress takes it hard when HN comes around! The Internet Archive has it: http://web.archive.org/web/20170711105235/blog.nothinguntoward.eu/?p=238 http://web.archive.org/web/20170711105235/blog.nothinguntowa...

ship_it 9y ago
True, thanks for the link. Shame that author uses Wordpress. I'd advice him to use a static page generator for the future.
- throwanem 9y ago
  I use Wordpress, too, because I'm more interested in publishing content than in dinking around endlessly with the latest fad in content management tooling.
  - ue_ 9y ago
    To be fair, static blog generators have been around for years, and their use is hardly a fad, considering the number of blogs that people host on github etc. It hasn't caught on as much among non-tech types, but it's certainly not a fad, and the solutions are mature, usable and open source. I can understand wanting various features that only a dynamic blog offers (like commenting without Javascript), and I run such a blog myself.
    - y4mi 9y ago
      There are tools for that as well. Staticman for example.
- 18nleung 9y ago
  Wordpress is still essentially the status quo outside of technical circles and for non-technical people I'd recommend it over most other static site generators.
  - 9y ago

polote 9y ago

What is the percentage of SD cards which are not fake ? Are we sure that popular SD cards on Amazon are not fake ?

notyourday 9y ago
I do not know about Amazon but I have some interesting experiences: Over last several years I have ordered a few hundred SD Cards, some from Ebay some from Amazon, some bought at the stores. Store cards were Kingston and PNY branded - bought at Microcenter. Amazon were some unbranded ones and Sandisks. My method of testing is rather simple - for 64G card generate a file of 63864569856 bytes of random data, dd it onto a card dd result off the card, fingerprint both. If they match the card is probably good. This is what I found: * cards with fake sizes will fail to write entire file with IO error * cards with questionable quality will happily spew the following into kernel log: [6135586.258783] sd 7:0:0:2: [sdf] tag#0 CDB: Read(10) 28 00 07 6f 4f 80 00 00 08 00 [6135586.258785] blk_update_request: I/O error, dev sdf, sector 124735360 [6135586.654749] sd 7:0:0:2: [sdf] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [6135586.654752] sd 7:0:0:2: [sdf] tag#0 Sense Key : Hardware Error [current] [6135586.654753] sd 7:0:0:2: [sdf] tag#0 Add. Sense: No additional sense information [6135586.654755] sd 7:0:0:2: [sdf] tag#0 CDB: Read(10) 28 00 07 6f 4f 80 00 00 08 00 [6135586.654757] blk_update_request: I/O error, dev sdf, sector 124735360 [6135586.654760] Buffer I/O error on dev sdf, logical block 15591920, async page read * Sandisk had the lowest number of the failures when bought on ebay - sub 10%, highest rate of failures when bought on Amazon - over 90%. * Kingstons from Micro Center passed the tests. * PNYs failed the test. The only wrinkle that I currently have is that I use Sandisks in a dual card Nikon configured for dual write. So far two of the cards that passed the tests created a "Card error" reported by the camera on separate occasions with separate slots. I do not however know if the issue is caused by the the adapter or by the card. I just ordered a few new adapters to see if I can isolate it. [Edit: Formatting] [Late addition: below] The cards that triggered an error in camera passed the read/write test after the error so I am inclined to believe it could be the adapter.
- qb45 9y ago
  > My method of testing is rather simple - for 64G card generate a file of 63864569856 bytes of random data, dd it onto a card dd result off the card, fingerprint both. If they match the card is probably good. One optimization: dd bs=1M count=N if=/dev/urandom | tee /dev/mmcblk0 | md5sum md5sum /dev/mmcblk0

woliveirajr 9y ago

There are some good programs already written to test your Sd / flash card / USB / whatever you want: USB Flash Drive Tester: http://www2.vconsole.com/client/?page=page&id=13 http://www2.vconsole.com/client/?page=page&id=13 H2testw: https://www.portablefreeware.com/?id=1519 https://www.portablefreeware.com/?id=1519

londons_explore 9y ago

A pretty big chunk of memory cards are fake. "kind" fakes show write errors when writing data. Evil fakes just return garbage data. I have even witnessed a few fakes which understand a fat32 filesystem and make sure the filesystem data structures and the first and last few kilobytes of every file get mapped to working flash so that a chkdsk won't pick up issues and browsing the files shows no obvious issue. Many cameras store jpeg thumbnails, and the thumbnails continue to show too.

LoSboccacc 9y ago
that's not just bad that's intentionally evil
milankragujevic 9y ago
Could it be that a fake SD card would make all the pictures in it Black or garbled? I had that happen to me, but it was a 2GB SD card from 2007...
- londons_explore 9y ago
  Sure. Fakes like that have been around for almost as long as sd cards themselves.
TazeTSchnitzel 9y ago
Then there's the fakes that modulo the address. Write beyond the end of the actual flash and you'll write over the start!
- mikeash 9y ago
  Seems like that would be the simplest way to design one. Just leave off some of the high address lines.

CaptainZapp 9y ago

Why buy an SD card of ebay or, alternatively, find a deal which is too good to be true (and is neither good, nor true, of course)? A branded, well specced SD card is not really that expensive that I would risk - for example - my photos which are certainly much more valuable to me than a few $ saved. Even if it's in the 10s $ range. I honestly just don't get it.

kawsper 9y ago
I don't trust any vendor anymore. I have managed to get fake SD-cards from a local retailer, Amazon, a webshop, and big supermarket chain. It seems like the distribution channels are poisoned.
- otterpro 9y ago
  I bought a SanDisk SD card from BestBuy (at physical store), trusting that it was genuine, so I never ran any test. Only after a year or so, I think it might possibly be a fake, as the speed was incredibly slow - write speed was slower than my 7 year old SD card.
ptero 9y ago
How you do you purchase cards to guarantee you are not getting a fake? I believe fakes are pretty much everywhere now, in branded stores as well. Maybe buying those from stores that serve people who care helps (e.g., B&H photo -- as you indicate, a pro photographer losing his photos would be furious). They try to act on problems and work with suppliers quickly, but even this is not a guarantee. A better approach IMO is to test the card first using easily available tools and use backups. This is more work, but can save you a major aggravation. My 2c.
- CaptainZapp 9y ago
  I reply to you, but mean both of you who replied to my comment. Thanks! I was really not aware that this is a problem and will likely test my cards in trhe future. This is atrocious.
- 9y ago

ehPReth 9y ago

I've been using F3 (http://oss.digirati.com.br/f3/ http://oss.digirati.com.br/f3/) to test new SD cards. Installable via homebrew and seems to work quite well

randkyp 9y ago
Yes. I just run f3probe for a quick capacity & speed test and then run a single-pass, random data badblocks test just to be sure. If there are any 'bad' sectors, I just pass the badblocks output to mkfs. Be careful with your choice of card reader/adapter though--some starts to get too hot and end up breaking the card under heavy testing!

wingi 9y ago

Hmmm, some start writingGB17 on the position of the GB1, so your 1MB read/test will make all green.

lima 9y ago

About the sync: you can bypass the fs cache by specifying iflag=direct/oflag=direct.

qb45 9y ago
In fact, I'm quite surprised that it worked at all without direct. Even if sync writes the cached data to disk, it isn't exactly guaranteed that the following read will read them back instead of just returning a copy from the cache.
alecjw 9y ago
Nice, thanks for the tip!

dis-sys 9y ago

nothing special here, just a 16GB card got marked as 64GB, this kind of scam has been there for ages. You got hit by dishonest marketing BS from big brands all the time as well, buying from reliable brand and its recommended (overpriced) channel doesn't solve the problem, it seem to become the standard way of doing business in storage (what is the likelihood to get a fake RAM/CPU/GPU to this extent?). I think that is actually the most scary part. I was lately trying to buy a new SSD, surely you want a NVMe SSD, so I picked up the Pro version of this top brand SSD, as the spec is the best of the best you can find on the market. Did a simple test, fsync performance is actually worse than the 2 years old SATA SSD from the same brand and that SATA was a EVO module. Sure, they didn't advertise fsync performance, but do you think it is fair not to mention/disclose/report such very poor fsync performance when it is actually slower than their own 2 years old product? I didn't waste my time to argue with anyone, got it refunded and bought the EVO version of the NVMe SSD as I thought I just forget about the fsync performance and save that $100 difference. Got the fancy EVO NVMe SSD, did a sequential write test, the throughput performance dropped like 40% after certain amount of writes. Carefully checked the product spec, there is some text on the box written in some size 2 font basically telling people to expect such performance drop as there is some cache style design there. Well, at least I got this very good random IOPS spec, right? Did some tests myself, guess what? The advertised IOPS performance is only true when running in situations considered as rare or even abnormal for such consumer grade SSD - you need a very long IO queue depth. In plain English, if you just buy this consumer grade SSD and use it in a consumer setting, you are not going to get the advertised performance. Fake or marketing BS? If the whole idea is to fool me to pay for their over stated stuff, then what is the difference?

ve7cxz 9y ago
The issue is that with these cards they're not just branded wrong, they often present themselves as being bigger than they actually are - so your OS sees it as being 64GB when it's only 16GB, for example. Were it just a branding thing it wouldn't - from a technical standpoint - be quite so bad, other than you know, the intentional lying...

deleted 9y ago

[deleted]

TazeTSchnitzel 9y ago

At least for USB flash drives that behave similarly, sometimes there are software tools that let you configure the drive's claimed capacity.

jaclaz 9y ago
Yes and no. In the sense that many "manufacturer tools" for USB sticks (and controllers) are available (sometimes released by he actual manufacturers, sometimes released by some "OEM" or "Brands" as a recovery or formatting software, the most part AFAICT leaked), while there are very few (if any at all) correspondent tools for SD cards. Here is a couple known articles by "Bunnie" of BunnieStudios, respectively 2013 and 2009: https://www.bunniestudios.com/blog/?p=3554 https://www.bunniestudios.com/blog/?p=3554 http://www.bunniestudios.com/blog/?page_id=1022 http://www.bunniestudios.com/blog/?page_id=1022

zeep 9y ago

I've received some fake ones but they were unbranded... The ones that I ordered online on Amazon with the whole retail packaging have always been good so far. Sounds like something an evil cloud storage company would do to promote the use of online storage... that and removing SD cards slot from whatever they can.