12 ms·

Results of 500 MicroSD Benchmarks on SBCs

168 points by bmlw 5y ago

dtx1 5y ago
I'm suprised by the strong results of the amazon basic cards, though what is missing from this dataset is the "how fast does it die" measurement
- bmlw 5y ago
  I started that testing around a week ago! Will hopefully have enough data for a post in the coming weeks but it's a bit easier to test for speed first and before you kill cards :D
von_lohengramm 5y ago
I really dislike how the only the biggest number from each category is bolded. I really doubt that 5 tests was enough to make 21.21MB/s significantly (statistically speaking) faster than 21.16MB/s. Other than that, I'm really surprised by just how much faster the IO on the RPi4 is. Quite the difference!
- gompertz 5y ago
  My thought too. Appreciate the authors effort; but given how tight all the numbers are along with the 5 tests, if I were the experimenter I'd feel like this was a big waste of my time and inconclusive.
  - von_lohengramm 5y ago
    As a reader, I found this more interesting as an IO benchmark of various SBCs. While it may not have been the first benchmark of its kind, I don't really seek out that information, so this was interesting to me at least.
  - nerdponx 5y ago
    Not at all IMO. Concluding that they all perform about the same is informative in and of itself.
  - bmlw 5y ago
    No need to worry, I often feel like what I'm doing is a waste of time. I do agree with what someone else said though as I feel like whilst the sequential reads and writes on most boards are largely the same, the random reads/writes are a little more varied and this may be what people are looking for. In either case, if a £5 card performs largely the same as a £15 card, I'd say it was worth doing for the money savings. Whether that £5 card will last as long as the £15 one is a different matter but hopefully with some of the other tests I'm doing, we'll find out!
    - von_lohengramm 5y ago
      Haha I know what you mean! It was just a nitpicky thing, I found this interesting nevertheless.
- bmlw 5y ago
  Thanks for your comment. I'm not sure if just leaving them as is and letting the reader decide how they want to interpret/read the results would be better? I'm doing these mainly out of my own curiosity in my endless free time at the moment and I'm new to creating content like this so I'm open to feedback!
- gruez 5y ago
  Having data bars like in excel[1] solves this issue. The ability to easily visualize any difference is a nice bonus as well. For instance 2.16 MB/s vs 2.55 MB/s is statistically significant, but it's probably not something that you'll notice in regular use unless you have a stopwatch. However 2.65 MB/s vs 4.49 MB/s is easily noticeable. [1] https://support.microsoft.com/en-us/office/use-data-bars-color-scales-and-icon-sets-to-highlight-data-f118d0a6-5921-4e2e-905b-fe00f3378fb9 https://support.microsoft.com/en-us/office/use-data-bars-col...
cyounkins 5y ago
Here's a massive dataset: https://pibenchmarks.com/ https://pibenchmarks.com/
- m463 5y ago
  That's an odd database for SD cards. A lot of the images do not match the SD card description (in size).
walrus01 5y ago
I am far less concerned about read/write speeds than I am about write endurance and overall reliability/lifespan of microSD cards in raspberry pi 3/4. Would be interesting to see a write torture test until failure comparing the samsung "PRO ENDURANCE" microsd cards to regular u1/u3 class cards.
- icameron 5y ago
  This is a good point, but there is a larger problem anytime you are relying on SD cards for a R/W root partition. Especially when making a product out of one of these boards. It's so easy to follow a tutorial and start an idea from a Bone or Pi or whatever and have some cool functionality and think it's ready to sell. Unfortunately, many of these boards don't have on board flash and by default are booting from SD card with a Linux distribution. It will fail a lot more than a real hard drive does. A cheap SD card is not the same as an SSD when you get down to it, no way around it. Absolutely use the SD card to log data, but don't make booting dependent on cheap removable flash media. Choose an SBC that has high quality flash with good EEC. Source: Experience datalogging with SBCs for 3 companies for 10 years of my career.
  - koz1000 5y ago
    I'm curious about the quality of the uSD card holders on all of these units. Do they all use quality non-corrosive fingers with enough force on the contacts? I once had an intern that told me all about his RPi project to do model rocket telemetry, then after some interrogation sheepishly admitted that the project never quite worked because the g forces and vibration at launch were enough to make the card holder lose contact and the kernel panicked. Bummer.
    - colechristensen 5y ago
      Rockets are difficult environments for connectors, the spring loaded compliant mechanism of an sd card holder just isn’t appropriate for that application. You could get it to work but it would involve a lot of vibration analysis and design to be gentle enough to the connector and you would have to keep track of every time a card was connected and removed.
kup0 5y ago
I know USB booting on pi is still not perfect, but it almost seems to make more sense at this point to boot off of a decent USB flash drive, I would think? If you get a decent brand with good benchmarks, I would think that reads/writes would blow all SD cards away. What I'm not sure about is latency or lifespan, though I would think _good_ USB flash drives are likely to be better quality flash than SD, without really being significantly costly in comparison? I say this as someone that uses SD cards currently, and I understand if others do also- it's the easiest/default option, but I've definitely given thought to switching
- StillBored 5y ago
  USB flash drives are mostly just as bad. Many advertise large read/write numbers but they overwhelmingly all hit a wall once the MB or two of cache is full. And they seem to be as unreliable as SD these days. The solution is USB->SATA SSD converters with SATA SSDs. Those will easily sustain a few hundred MB/sec read/write. The problem then becomes the USB inefficiencies, but for the most part wear leveling/etc make them pretty bullet proof. The PFTF UEFI firmware will reliably boot off just about everything you can plug in (USB DVD/etc included).
- dikei 5y ago
  I've had a USB flash drive failed after only 3 months in the Raspberry Pi before, so now I'm using a SATA-USB cable with a SSD drive, looking good so far.
dsr_ 5y ago
I have to wonder how lucky the Amazon card is: is the manufacturer consistent, or is it just a rebrand of whoever has made them a deal this quarter? Or several rebrands?
- bmlw 5y ago
  This is actually part of my worry too and I bought 2 units (of the 2x64GB) from the UK and Swedish sites and both are showing up as the same manufacturer etc. Would definitely like to hear from others that have them though to see how consistent things are over in North America.
- m463 5y ago
  I wonder the same thing. I had a Kingston SATA SSD that failed. It didn't just become read-only, it failed completely and catastrophically. One day everything was there, the next day the drive couldn't access anything. I delved deeper after that and now stick to major manufacturers.
deleted 5y ago
[deleted]
Seattle3503 5y ago
My understanding is that the SD card standard is often the bottleneck here. It would be nice if devices (like the Pi) could move to the faster UFS standard.
- rsaxvc 5y ago
  SD6 adds class A2 cards that allow queueing.
  - Seattle3503 5y ago
    Presumably the cards they tested have that?
    - rsaxvc 5y ago
      Some, but I'm not sure Linux supports it yet. I had also seen some patches for turning on the DDR SD mode, again not sure if they're used today though.
masyukun 5y ago
This is great data! I sliced it 2 different ways in Google Sheets -- by SD card and by Raspberry Pi board. Agreed that Amazon Basics is the surprising front-runner for the cards, although the IOPing results are shockingly bad for most cards except SanDisk Ultra 16 + 32. Might need to use Analytical Hierarchy Process to determine how each performance characteristic should be weighted. For boards, the top 3 in your tests by a wide margin are: 1) Orange Pi i96, 2) BeagleBone Black (2GB eMMC), 3) Raspberry Pi 4 Model B (Revision 1.1 – 2GB) https://docs.google.com/spreadsheets/d/1ENHM6N38iHFd7dEYliaTzkvv178tMA-XGsCLWe-5FDc/edit?usp=sharing https://docs.google.com/spreadsheets/d/1ENHM6N38iHFd7dEYliaT...
- bsder 5y ago
  The BeagleBone Black hasn't been a 2GB eMMC for ... almost a decade? Where on Earth did he cough up a BBB that old and why is he using it for benchmarking?
  - bmlw 5y ago
    The same place I found the Raspberry Pi 1/Model B? In my cupboard :D I didn't go out and purchase everything brand new for this post. I used what I had available to me! I often see these 2GB models pop up on our local version of eBay and they're out there. Does the newer version have a different SD card reader or other hardware that will render the results pointless in a comparison with it? (Genuine question!)
    - bsder 5y ago
      For the eMMC, quite probably. The BBB Rev B BOM lists a Micron chip that tops out at 30MB/s read and 6.6MB/s write (pretty close to measured). The BBB Rev C BOM lists Micron chips in that range but also lists chips like the Kingston EMMC04G-M627 which is capable of 250MB/s read and 25 MB/s write. This link seems to indicate that the Rev C is around 2x faster on eMMC. https://www.tablix.org/~avian/blog/archives/2017/08/beaglecore_module_emmc_and_sd_card_benchmarks/ https://www.tablix.org/~avian/blog/archives/2017/08/beagleco... (the RPi Zero numbers seem to correlate with yours so it seems you are measuring the same things) So, the eMMC numbers are going to be highly specific to the board manufactured.
ezekiel68 5y ago
Who is this Amazon company and what kind of magic turbo pixie dust have they spread on their commodity "Basics" MicroSDs?
PennRobotics 5y ago
I have a weird wish that someone would test all these SD cards for their latency while an crossing allocation unit boundary in SPI mode. This is quite different than average latency. One work project of mine involved a bunch of sensors, an 8-bit microcontroller without much memory, and an SD card. It turns out that cost, brand name, and speed class have nothing to do with AU boundary latency. Trying to write 255 bytes every 2 to 10 milliseconds at a few MHz on a chip with 2.5 KB SRAM, the limiting factor is the SD controller needing up to a hundred milliseconds every 10 seconds or so (depending on AU size) during which real-time data cannot be buffered in the SD card, microcontroller, or sensor FIFO. There's simply too much data waiting to store! ----- A few notes for anyone at all who cares: Single-level cell (aka industrial) SD didn't result in a faster AU change speed. One SD card---out of 10 or 12 tried---would not pull down its MISO line unless an extra clock pulse was manually bit-banged, which is a really stupid behavior to debug after having a few working cards. Tracking down this bug took a lot of garbled data, scope measurements, and re-readings of the SD specification to see if my code was wrong or the card manufacturer was non-compliant. The specification has a maximum write latency of something like 200 ms, and that's the sort of detail you miss until you've spent time and money designing/ordering a PCB and realizing you should've just added a 64Mbit flash IC and dealt with the inconvenience of UART data transfer.
- bullen 5y ago
  I use 1TB SanDisk (Raspberry 2) and 4GB Panasonic SLC (Raspberry 4) in my distributed database cluster: http://github.com/tinspin/rupy http://github.com/tinspin/rupy In my experience industrial versions of mainstream cards are less reliable! I agree, you have to measure latency and longevity on these edge (maximum longevity or space) SD cards as bandwidth is limited by SPI (4 is not really that much faster than 2). USB3 is not stable enough for 99.9999 uptime. If you use the cards above and mostly write once on the 1TB and replicate all that data on 3 cards in 2 geographical locations in realtime (6x in total minimum = you have redundancy even if one node fails on local and global level) you should be fine. After having 8GB Odroid MMC fail on me twice after 5 years, I invested in Intel Atom servers with X25-E (65nm SLC 100.000 writes per bit) drives as backup plan if the SD card solution fails/has problems. Power goes from 2W to 25W (50W with 10x SATA SSDs) with that change so you cannot afford more than 1-2 Atoms per location as lead-acid backup will be too expensive/large otherwise. Remember, if you do not power flash memory for a long time the data is corrupted/lost. You need your disks to be powerable on solar/wind, in practice this means Raspberry 2 when the mains go. As electricity prices rise the supply quality will degrade. Power outages will increase exponentially until the powergrid fails without replacement parts/transport. Peak energy is not a speculation, it's the only sure thing.
hepinhei 5y ago
the majority of those SDCArds can perform better than that (mainly read). it seems to me the boards are the bottleneck.

168 points by bmlw 5y ago

dtx1 5y ago

I'm suprised by the strong results of the amazon basic cards, though what is missing from this dataset is the "how fast does it die" measurement

bmlw 5y ago
I started that testing around a week ago! Will hopefully have enough data for a post in the coming weeks but it's a bit easier to test for speed first and before you kill cards :D

von_lohengramm 5y ago

I really dislike how the only the biggest number from each category is bolded. I really doubt that 5 tests was enough to make 21.21MB/s significantly (statistically speaking) faster than 21.16MB/s. Other than that, I'm really surprised by just how much faster the IO on the RPi4 is. Quite the difference!

gompertz 5y ago
My thought too. Appreciate the authors effort; but given how tight all the numbers are along with the 5 tests, if I were the experimenter I'd feel like this was a big waste of my time and inconclusive.
- von_lohengramm 5y ago
  As a reader, I found this more interesting as an IO benchmark of various SBCs. While it may not have been the first benchmark of its kind, I don't really seek out that information, so this was interesting to me at least.
- nerdponx 5y ago
  Not at all IMO. Concluding that they all perform about the same is informative in and of itself.
- bmlw 5y ago
  No need to worry, I often feel like what I'm doing is a waste of time. I do agree with what someone else said though as I feel like whilst the sequential reads and writes on most boards are largely the same, the random reads/writes are a little more varied and this may be what people are looking for. In either case, if a £5 card performs largely the same as a £15 card, I'd say it was worth doing for the money savings. Whether that £5 card will last as long as the £15 one is a different matter but hopefully with some of the other tests I'm doing, we'll find out!
  - von_lohengramm 5y ago
    Haha I know what you mean! It was just a nitpicky thing, I found this interesting nevertheless.
bmlw 5y ago
Thanks for your comment. I'm not sure if just leaving them as is and letting the reader decide how they want to interpret/read the results would be better? I'm doing these mainly out of my own curiosity in my endless free time at the moment and I'm new to creating content like this so I'm open to feedback!
gruez 5y ago
Having data bars like in excel[1] solves this issue. The ability to easily visualize any difference is a nice bonus as well. For instance 2.16 MB/s vs 2.55 MB/s is statistically significant, but it's probably not something that you'll notice in regular use unless you have a stopwatch. However 2.65 MB/s vs 4.49 MB/s is easily noticeable. [1] https://support.microsoft.com/en-us/office/use-data-bars-color-scales-and-icon-sets-to-highlight-data-f118d0a6-5921-4e2e-905b-fe00f3378fb9 https://support.microsoft.com/en-us/office/use-data-bars-col...

cyounkins 5y ago

Here's a massive dataset: https://pibenchmarks.com/ https://pibenchmarks.com/

m463 5y ago
That's an odd database for SD cards. A lot of the images do not match the SD card description (in size).

walrus01 5y ago

I am far less concerned about read/write speeds than I am about write endurance and overall reliability/lifespan of microSD cards in raspberry pi 3/4. Would be interesting to see a write torture test until failure comparing the samsung "PRO ENDURANCE" microsd cards to regular u1/u3 class cards.

icameron 5y ago
This is a good point, but there is a larger problem anytime you are relying on SD cards for a R/W root partition. Especially when making a product out of one of these boards. It's so easy to follow a tutorial and start an idea from a Bone or Pi or whatever and have some cool functionality and think it's ready to sell. Unfortunately, many of these boards don't have on board flash and by default are booting from SD card with a Linux distribution. It will fail a lot more than a real hard drive does. A cheap SD card is not the same as an SSD when you get down to it, no way around it. Absolutely use the SD card to log data, but don't make booting dependent on cheap removable flash media. Choose an SBC that has high quality flash with good EEC. Source: Experience datalogging with SBCs for 3 companies for 10 years of my career.
- koz1000 5y ago
  I'm curious about the quality of the uSD card holders on all of these units. Do they all use quality non-corrosive fingers with enough force on the contacts? I once had an intern that told me all about his RPi project to do model rocket telemetry, then after some interrogation sheepishly admitted that the project never quite worked because the g forces and vibration at launch were enough to make the card holder lose contact and the kernel panicked. Bummer.
  - colechristensen 5y ago
    Rockets are difficult environments for connectors, the spring loaded compliant mechanism of an sd card holder just isn’t appropriate for that application. You could get it to work but it would involve a lot of vibration analysis and design to be gentle enough to the connector and you would have to keep track of every time a card was connected and removed.

kup0 5y ago

I know USB booting on pi is still not perfect, but it almost seems to make more sense at this point to boot off of a decent USB flash drive, I would think? If you get a decent brand with good benchmarks, I would think that reads/writes would blow all SD cards away. What I'm not sure about is latency or lifespan, though I would think _good_ USB flash drives are likely to be better quality flash than SD, without really being significantly costly in comparison? I say this as someone that uses SD cards currently, and I understand if others do also- it's the easiest/default option, but I've definitely given thought to switching

StillBored 5y ago
USB flash drives are mostly just as bad. Many advertise large read/write numbers but they overwhelmingly all hit a wall once the MB or two of cache is full. And they seem to be as unreliable as SD these days. The solution is USB->SATA SSD converters with SATA SSDs. Those will easily sustain a few hundred MB/sec read/write. The problem then becomes the USB inefficiencies, but for the most part wear leveling/etc make them pretty bullet proof. The PFTF UEFI firmware will reliably boot off just about everything you can plug in (USB DVD/etc included).
dikei 5y ago
I've had a USB flash drive failed after only 3 months in the Raspberry Pi before, so now I'm using a SATA-USB cable with a SSD drive, looking good so far.

dsr_ 5y ago

I have to wonder how lucky the Amazon card is: is the manufacturer consistent, or is it just a rebrand of whoever has made them a deal this quarter? Or several rebrands?

bmlw 5y ago
This is actually part of my worry too and I bought 2 units (of the 2x64GB) from the UK and Swedish sites and both are showing up as the same manufacturer etc. Would definitely like to hear from others that have them though to see how consistent things are over in North America.
m463 5y ago
I wonder the same thing. I had a Kingston SATA SSD that failed. It didn't just become read-only, it failed completely and catastrophically. One day everything was there, the next day the drive couldn't access anything. I delved deeper after that and now stick to major manufacturers.

deleted 5y ago

[deleted]

Seattle3503 5y ago

My understanding is that the SD card standard is often the bottleneck here. It would be nice if devices (like the Pi) could move to the faster UFS standard.

rsaxvc 5y ago
SD6 adds class A2 cards that allow queueing.
- Seattle3503 5y ago
  Presumably the cards they tested have that?
  - rsaxvc 5y ago
    Some, but I'm not sure Linux supports it yet. I had also seen some patches for turning on the DDR SD mode, again not sure if they're used today though.

masyukun 5y ago

This is great data! I sliced it 2 different ways in Google Sheets -- by SD card and by Raspberry Pi board. Agreed that Amazon Basics is the surprising front-runner for the cards, although the IOPing results are shockingly bad for most cards except SanDisk Ultra 16 + 32. Might need to use Analytical Hierarchy Process to determine how each performance characteristic should be weighted. For boards, the top 3 in your tests by a wide margin are: 1) Orange Pi i96, 2) BeagleBone Black (2GB eMMC), 3) Raspberry Pi 4 Model B (Revision 1.1 – 2GB) https://docs.google.com/spreadsheets/d/1ENHM6N38iHFd7dEYliaTzkvv178tMA-XGsCLWe-5FDc/edit?usp=sharing https://docs.google.com/spreadsheets/d/1ENHM6N38iHFd7dEYliaT...

bsder 5y ago
The BeagleBone Black hasn't been a 2GB eMMC for ... almost a decade? Where on Earth did he cough up a BBB that old and why is he using it for benchmarking?
- bmlw 5y ago
  The same place I found the Raspberry Pi 1/Model B? In my cupboard :D I didn't go out and purchase everything brand new for this post. I used what I had available to me! I often see these 2GB models pop up on our local version of eBay and they're out there. Does the newer version have a different SD card reader or other hardware that will render the results pointless in a comparison with it? (Genuine question!)
  - bsder 5y ago
    For the eMMC, quite probably. The BBB Rev B BOM lists a Micron chip that tops out at 30MB/s read and 6.6MB/s write (pretty close to measured). The BBB Rev C BOM lists Micron chips in that range but also lists chips like the Kingston EMMC04G-M627 which is capable of 250MB/s read and 25 MB/s write. This link seems to indicate that the Rev C is around 2x faster on eMMC. https://www.tablix.org/~avian/blog/archives/2017/08/beaglecore_module_emmc_and_sd_card_benchmarks/ https://www.tablix.org/~avian/blog/archives/2017/08/beagleco... (the RPi Zero numbers seem to correlate with yours so it seems you are measuring the same things) So, the eMMC numbers are going to be highly specific to the board manufactured.

ezekiel68 5y ago

Who is this Amazon company and what kind of magic turbo pixie dust have they spread on their commodity "Basics" MicroSDs?

PennRobotics 5y ago

I have a weird wish that someone would test all these SD cards for their latency while an crossing allocation unit boundary in SPI mode. This is quite different than average latency. One work project of mine involved a bunch of sensors, an 8-bit microcontroller without much memory, and an SD card. It turns out that cost, brand name, and speed class have nothing to do with AU boundary latency. Trying to write 255 bytes every 2 to 10 milliseconds at a few MHz on a chip with 2.5 KB SRAM, the limiting factor is the SD controller needing up to a hundred milliseconds every 10 seconds or so (depending on AU size) during which real-time data cannot be buffered in the SD card, microcontroller, or sensor FIFO. There's simply too much data waiting to store! ----- A few notes for anyone at all who cares: Single-level cell (aka industrial) SD didn't result in a faster AU change speed. One SD card---out of 10 or 12 tried---would not pull down its MISO line unless an extra clock pulse was manually bit-banged, which is a really stupid behavior to debug after having a few working cards. Tracking down this bug took a lot of garbled data, scope measurements, and re-readings of the SD specification to see if my code was wrong or the card manufacturer was non-compliant. The specification has a maximum write latency of something like 200 ms, and that's the sort of detail you miss until you've spent time and money designing/ordering a PCB and realizing you should've just added a 64Mbit flash IC and dealt with the inconvenience of UART data transfer.

bullen 5y ago
I use 1TB SanDisk (Raspberry 2) and 4GB Panasonic SLC (Raspberry 4) in my distributed database cluster: http://github.com/tinspin/rupy http://github.com/tinspin/rupy In my experience industrial versions of mainstream cards are less reliable! I agree, you have to measure latency and longevity on these edge (maximum longevity or space) SD cards as bandwidth is limited by SPI (4 is not really that much faster than 2). USB3 is not stable enough for 99.9999 uptime. If you use the cards above and mostly write once on the 1TB and replicate all that data on 3 cards in 2 geographical locations in realtime (6x in total minimum = you have redundancy even if one node fails on local and global level) you should be fine. After having 8GB Odroid MMC fail on me twice after 5 years, I invested in Intel Atom servers with X25-E (65nm SLC 100.000 writes per bit) drives as backup plan if the SD card solution fails/has problems. Power goes from 2W to 25W (50W with 10x SATA SSDs) with that change so you cannot afford more than 1-2 Atoms per location as lead-acid backup will be too expensive/large otherwise. Remember, if you do not power flash memory for a long time the data is corrupted/lost. You need your disks to be powerable on solar/wind, in practice this means Raspberry 2 when the mains go. As electricity prices rise the supply quality will degrade. Power outages will increase exponentially until the powergrid fails without replacement parts/transport. Peak energy is not a speculation, it's the only sure thing.

hepinhei 5y ago

the majority of those SDCArds can perform better than that (mainly read). it seems to me the boards are the bottleneck.