4 ms·

Please for the love of god, do not use Postgres on a SD card, you'll corrupt it quickly, even spinning rust over USB2.0 is faster and more reliable. That's my e

Please for the love of god, do not use Postgres on a SD card, you'll corrupt it quickly, even spinning rust over USB2.0 is faster and more reliable. That's my experience with it at least.

sneakernets 8y ago
How long have we had SD Cards for them to still be unreliable like this? This is unacceptable.
- dleslie 8y ago
  Now you have good odds of purchasing a counterfeit. Doesn't seem to matter where you buy from, too. The consumer electronics market is a shit show.
  - sneakernets 8y ago
    This is exactly my fear. There is no quality control anymore.
- AlotOfReading 8y ago
  You can have high capacity, low cost, or reliability. Pick 2. Consumer cards choose capacity and cost, because that's what sells. Industrial SD cards pick capacity and reliability, but cost at least 4x more than similar consumer cards due to inherent tradeoffs in the hardware/firmware design.
  - dpedu 8y ago
    Are they though? I looked on Amazon; the price difference between a 'regular' 32GB microsd and a 'high endurance' card of the same capacity is $3 (7.99 vs 10.99).
    - AlotOfReading 8y ago
      Consumer high endurance cards are still MLC or TLC. They're better than regular consumer SD cards, but still not what you'd consider ideal reliability. Industrial cards are better QC'd and use more reliable (but less dense) SLC, as well as various smaller changes to operate reliably in extreme conditions. I'm sure some of it is industrial product markup as well, but the technology is fundamentally more expensive.
- bityard 8y ago
  The problem is that SD cards were designed for portable storage of photos and music, which is a completely different use case than running an OS or database on top of them. They're being asked to do something they were not designed for, and then people get upset about it. They make high-endurance SD cards that have the same (or better) reliability as SSDs. These are what should be used in any Raspberry Pi that does some non-trivial amount of writing to the disk. Or boot from USB or Ethernet.
megous 8y ago
It depends. I have multigig PostgreSQL DB on Orange Pi 2 and a consumer sandisk uSD, that I scrape hundreds of news articles to daily (incl. images), for several years. No issue, yet. Fsync and PostgreSQL checksumming enabled, of course. The same goes for the other Orange Pi PC I have, where I write weather data in 5 minute intervals for several years. No issue either. I have a very good power distribution (custom made cables, high quality 40A/5V power supply), which makes a big difference in reliability, overall. My Raspberry Pi has uSD card issues every 3rd boot (probably gets into a read-only mode). I have to boot it multiple times to get the uSD card to work. It uses Rpi branded power adapter. Also, for years (~7) I used a single 16G SD card for all my camcoder recordings. It has to had handled >1TB and it still works. I really can't complain about any of my (u)SD cards. (I run a cluster of 11 boards and I had one uSD card failure over 3 years. And it was some cheap "Samsung" from aliexpress. Verified Sandisk A1 cards never failed me so far.)
- sgt 8y ago
  Yeah but most people running Pi's will use the adapter it comes with. It's a 2.5A output 5V.
  - megous 8y ago
    Many people can have it runing from a random USB charger, as you don't have to buy the adapter along with the board and the decision was to power it through a microUSB connector, which certainly doesn't help.
- megous 8y ago
  Forgot to say, I also use f2fs everywhere. That should be nicer filesystem to the SD cards.
bityard 8y ago
They make high-endurance SD cards these days. They have the same reliability (or better) as an SSD but in a small package and for relatively cheap. These should be used in virtually all Raspberry Pis that see any non-trivial amount of disk use. RPi3+ also support booting from USB and Ethernet so you really don't need an SD card in a lot of cases either.
- extra88 8y ago
  > the same reliability (or better) as an SSD Can you give examples of microSD cards like this? What googling for "high endurance" turns up are cards touting their use for video purposes (e.g. dash or security cameras that run 24/7). Continuous writes to video files, overwriting each memory cell every X period of time it takes to fill the card, is more punishing than other uses but it could still be far less than the thrashing caused by running an operating system for some tasks. I've also seen the A1 and A2 classifications for higher IOPS cards, also good for running an OS, but independent of endurance. https://www.sdcard.org/developers/overview/application/ https://www.sdcard.org/developers/overview/application/
  - SpikedCola 8y ago
    I would consider "SLC" microSD cards to be "high-reliability" compared to MLC/TLC. However you pay for it - the cheapest 1GB SLC card on Digikey is $43 CAN[0]. Here is a helpful-looking image for selecting between different memory types [1] [0] https://www.digikey.ca/product-detail/en/swissbit/SFSD1024N1BM1TO-I-DF-221-STD/1052-1203-ND/5762190 https://www.digikey.ca/product-detail/en/swissbit/SFSD1024N1... [1] https://www.cactus-tech.com/assets/images/a/Selector%20Guide-a7e0e41a.png https://www.cactus-tech.com/assets/images/a/Selector%20Guide...
sgt 8y ago
Same. I've warned about this many times. It's a bit strange to hear that other people have been running Pi's for years with zero issues on SD, but that's definitely not been my experience. Note that I've run a few dozen Pi's that all ran 24/7. The final solution was to let the Pi boot via USB from an external 2.5 HDD. SSD is best as it consumes less power, but indeed a rust spinner will be fine too.

Please for the love of god, do not use Postgres on a SD card, you'll corrupt it quickly, even spinning rust over USB2.0 is faster and more reliable. That's my experience with it at least.

sneakernets 8y ago

How long have we had SD Cards for them to still be unreliable like this? This is unacceptable.

dleslie 8y ago
Now you have good odds of purchasing a counterfeit. Doesn't seem to matter where you buy from, too. The consumer electronics market is a shit show.
- sneakernets 8y ago
  This is exactly my fear. There is no quality control anymore.
AlotOfReading 8y ago
You can have high capacity, low cost, or reliability. Pick 2. Consumer cards choose capacity and cost, because that's what sells. Industrial SD cards pick capacity and reliability, but cost at least 4x more than similar consumer cards due to inherent tradeoffs in the hardware/firmware design.
- dpedu 8y ago
  Are they though? I looked on Amazon; the price difference between a 'regular' 32GB microsd and a 'high endurance' card of the same capacity is $3 (7.99 vs 10.99).
  - AlotOfReading 8y ago
    Consumer high endurance cards are still MLC or TLC. They're better than regular consumer SD cards, but still not what you'd consider ideal reliability. Industrial cards are better QC'd and use more reliable (but less dense) SLC, as well as various smaller changes to operate reliably in extreme conditions. I'm sure some of it is industrial product markup as well, but the technology is fundamentally more expensive.
bityard 8y ago
The problem is that SD cards were designed for portable storage of photos and music, which is a completely different use case than running an OS or database on top of them. They're being asked to do something they were not designed for, and then people get upset about it. They make high-endurance SD cards that have the same (or better) reliability as SSDs. These are what should be used in any Raspberry Pi that does some non-trivial amount of writing to the disk. Or boot from USB or Ethernet.

megous 8y ago

It depends. I have multigig PostgreSQL DB on Orange Pi 2 and a consumer sandisk uSD, that I scrape hundreds of news articles to daily (incl. images), for several years. No issue, yet. Fsync and PostgreSQL checksumming enabled, of course. The same goes for the other Orange Pi PC I have, where I write weather data in 5 minute intervals for several years. No issue either. I have a very good power distribution (custom made cables, high quality 40A/5V power supply), which makes a big difference in reliability, overall. My Raspberry Pi has uSD card issues every 3rd boot (probably gets into a read-only mode). I have to boot it multiple times to get the uSD card to work. It uses Rpi branded power adapter. Also, for years (~7) I used a single 16G SD card for all my camcoder recordings. It has to had handled >1TB and it still works. I really can't complain about any of my (u)SD cards. (I run a cluster of 11 boards and I had one uSD card failure over 3 years. And it was some cheap "Samsung" from aliexpress. Verified Sandisk A1 cards never failed me so far.)

sgt 8y ago
Yeah but most people running Pi's will use the adapter it comes with. It's a 2.5A output 5V.
- megous 8y ago
  Many people can have it runing from a random USB charger, as you don't have to buy the adapter along with the board and the decision was to power it through a microUSB connector, which certainly doesn't help.
megous 8y ago
Forgot to say, I also use f2fs everywhere. That should be nicer filesystem to the SD cards.

bityard 8y ago

They make high-endurance SD cards these days. They have the same reliability (or better) as an SSD but in a small package and for relatively cheap. These should be used in virtually all Raspberry Pis that see any non-trivial amount of disk use. RPi3+ also support booting from USB and Ethernet so you really don't need an SD card in a lot of cases either.

extra88 8y ago
> the same reliability (or better) as an SSD Can you give examples of microSD cards like this? What googling for "high endurance" turns up are cards touting their use for video purposes (e.g. dash or security cameras that run 24/7). Continuous writes to video files, overwriting each memory cell every X period of time it takes to fill the card, is more punishing than other uses but it could still be far less than the thrashing caused by running an operating system for some tasks. I've also seen the A1 and A2 classifications for higher IOPS cards, also good for running an OS, but independent of endurance. https://www.sdcard.org/developers/overview/application/ https://www.sdcard.org/developers/overview/application/
- SpikedCola 8y ago
  I would consider "SLC" microSD cards to be "high-reliability" compared to MLC/TLC. However you pay for it - the cheapest 1GB SLC card on Digikey is $43 CAN[0]. Here is a helpful-looking image for selecting between different memory types [1] [0] https://www.digikey.ca/product-detail/en/swissbit/SFSD1024N1BM1TO-I-DF-221-STD/1052-1203-ND/5762190 https://www.digikey.ca/product-detail/en/swissbit/SFSD1024N1... [1] https://www.cactus-tech.com/assets/images/a/Selector%20Guide-a7e0e41a.png https://www.cactus-tech.com/assets/images/a/Selector%20Guide...

sgt 8y ago

Same. I've warned about this many times. It's a bit strange to hear that other people have been running Pi's for years with zero issues on SD, but that's definitely not been my experience. Note that I've run a few dozen Pi's that all ran 24/7. The final solution was to let the Pi boot via USB from an external 2.5 HDD. SSD is best as it consumes less power, but indeed a rust spinner will be fine too.