3 ms·
Yup, been there too, way, way back. We wrote simulation tools for typical access patterns and ran them heavily for testing out new cards, both for performance
by amag 9y ago
Yup, been there too, way, way back.
We wrote simulation tools for typical access patterns and ran them heavily for testing out new cards, both for performance and failure rate. Luckily there was enough margin on the devices that we (R&D) could select the more expensive industrial graded cards by good manufacturers, but I did get to see some really shitty cards that purchasing preferred (because they had gotten a great price on them).
However in our case the corruption was due to a buggy FAT-driver for the obscure RTOS we used. In the end though I learned a lot about FAT which was fun (<sarcasm>and is a highly marketable skill these days</sarcasm>).
- monocasa 9y agoHaha, I ended up tapping all read and writes to the card, intending to catch our crappy FAT driver in the act of corrupting. Turns out the card itself was shit and replaying the trace would just kill arbitrary cards.
- michaelt 9y agoHow does one tap all the reads and writes to a card?
- hazeii 9y agoJust intercept the read and write calls! On linux you can just use 'strace $prog'; try it on a minimsal program that reads and writes a file. For a custom approach, this can be done with a library (using LD_PRELOAD). Plenty of articles about how to do this, e.g. http://www.linuxjournal.com/node/7795 http://www.linuxjournal.com/node/7795 (essentially the same technique on Windows, or other OS with shared libs). Slightly more hardcore is to hook the kernel syscalls; not normally necessary (on linux this is effectively hooking the other side of glibc, where it talks to the kernel) but if all you've got a static binary it's one approach. Note that all these techniques only show you the read and write calls made to the library/OS and - importantly - not what actually happens to the card. To see that, the next level down is to instrument the card driver to track the actual I/O operations (i.e. so you see what the card is really being asked to do, sans all the caching and buffering). Note that's not the end of the story; there's what the hardware controller decides to do and when the hardware actually reads/writes the flash array. That's the level where the quality of the firmware in the controller(s) matters.
- michaelt 9y agoDoesn't that mean you then have to write an event to disk for every write-to-disk event?
- sowbug 9y agoI would log by RPC or whatever to a remote computer.
- monocasa 9y agoI suspected the filesystem driver itself, so I had to modify the driver rather than strace (also this was WinCE quite unfortunately).
- monocasa 9y agoModify the SD card driver to open a socket to a computer in the lab, and just log it's commands and responses over TCP with timestamps. The network was orders of magnitude faster than the crappy SDIO interface, so it was easy this time. I've had to do a similar thing with SAS, and you basically just have to pay tens of thousands for a special purpose protocol analyzer, or build your own with an FPGA.
- amag 9y agoHaha, I spent so much time debugging their crappy FAT-driver. Another time the whole system would just hang when you created a new file in a directory that already contained some files. Turns out that their long-to-short filename algo was: For the first 4 files with identical prefix use the ~N scheme as in LongFile.txt -> LONGFI~1.TXT For the following files it was: 1) "LONG" + hex_str(hash(long_file_name)) + ".TXT" 2) if there's a name collision repeat step 1. Compound that with a the fact that hash() was implemented something like: int h = 0; for(int i = 0; i < strlen(long_file_name); ++i) { h = (h + long_file_name[i]) ^ 0x42424242; } It's pretty easy to see where this falls apart...