3 ms·

Wow, that's rough. I use a set of raspberry pi 4Bs as a k8s cluster (with an SSD over USB, otherwise etcd crashes cause the storage is too slow). Luckily I hav

by kelvie 3y ago

Wow, that's rough. I use a set of raspberry pi 4Bs as a k8s cluster (with an SSD over USB, otherwise etcd crashes cause the storage is too slow). Luckily I have 5 of them (and a k8s cluster is fault tolerant), because the USB disk randomly does not get enough power -- the voltage drops a lot even pulling 2-3A out of most chargers, I can't imagine doing 5 (the USB cable resistances start to become a factor at higher amperages).

eurekin 3y ago
Hi! How's that working for you? I have a similar cluster, but for CEPH specifically and I couldn't bear the hassle (supposedly hassless updates stuck, until I cleaned up some dirs as a root etc.)
- kelvie 3y ago
  It's been working fine. I have the pi's set up to reboot on disk failures, which fixes 99% of the problems (looking at the up time, ony node still reboots once ever 3 weeks or so), and the pods just get migrated, it's not a huge deal. I use longhorn though, not CEPH, and it's been relatively painless.
ajsnigrutin 3y ago
If you have a cluster, why not just buy a single decent power supply for everything? You can even find good quality ones even on aliexpress, like meanwell lrs-350 (5V, 60A output) for ~$40 on aliexpress and a bit more locally. (not afilliated, just use them a lot for 12v systems)
- HankB99 3y ago
  In some situations power delivery to the Pi is not the issue. It is power delivery from USB to the SSD. I ran into this myself and it was weird. * When the Pi 4B first came out, I connected a cheap Inland (Microcenter house brand) SSD, booting from that, and it ran fine for a while. (Months?) "Better" SSDs like Crucial MX100 had problems under heavy load and I suspected that the instantaneous power demands of the SSD were not met by the USB port on the Pi. (The Pi 4B supports UAS and older Pis did not and the older ones never had problems with USB connected SSDs.) * At some point it started to run into difficulties with the SSD disconnecting under heavy load, requiring a power cycle to restore operation. At that point I began using a powered USB hub to work around this. * Recently a user on some forum (which I have forgotten) suggested that one particular SSD model from Kingston worked fine w/out the need for a powered hub. I acquired one and tested [1] and found this to be true. Just for grins, I also tested some other SSDs that had previously only worked with a powered hub and they worked fine when directly connected. This leads me to believe that the RpiOS folk have been tweaking some settings in the "firmware" [2] that have modified the power delivery from the USB ports. [1] Testing involved running several disk benchmarks as well as 'stress-ng' to load the storage as much as possible. [2] I'm unclear what the Rpi folk mean by "firmware". As an embedded systems developer "firmware" is something written to non-volatile solid state storage like flash or eeprom but when I updated firmware on a Pi, it installed a new kernel too.

by kelvie 3y ago

eurekin 3y ago

Hi! How's that working for you? I have a similar cluster, but for CEPH specifically and I couldn't bear the hassle (supposedly hassless updates stuck, until I cleaned up some dirs as a root etc.)

kelvie 3y ago
It's been working fine. I have the pi's set up to reboot on disk failures, which fixes 99% of the problems (looking at the up time, ony node still reboots once ever 3 weeks or so), and the pods just get migrated, it's not a huge deal. I use longhorn though, not CEPH, and it's been relatively painless.

ajsnigrutin 3y ago

If you have a cluster, why not just buy a single decent power supply for everything? You can even find good quality ones even on aliexpress, like meanwell lrs-350 (5V, 60A output) for ~$40 on aliexpress and a bit more locally. (not afilliated, just use them a lot for 12v systems)

HankB99 3y ago
In some situations power delivery to the Pi is not the issue. It is power delivery from USB to the SSD. I ran into this myself and it was weird. * When the Pi 4B first came out, I connected a cheap Inland (Microcenter house brand) SSD, booting from that, and it ran fine for a while. (Months?) "Better" SSDs like Crucial MX100 had problems under heavy load and I suspected that the instantaneous power demands of the SSD were not met by the USB port on the Pi. (The Pi 4B supports UAS and older Pis did not and the older ones never had problems with USB connected SSDs.) * At some point it started to run into difficulties with the SSD disconnecting under heavy load, requiring a power cycle to restore operation. At that point I began using a powered USB hub to work around this. * Recently a user on some forum (which I have forgotten) suggested that one particular SSD model from Kingston worked fine w/out the need for a powered hub. I acquired one and tested [1] and found this to be true. Just for grins, I also tested some other SSDs that had previously only worked with a powered hub and they worked fine when directly connected. This leads me to believe that the RpiOS folk have been tweaking some settings in the "firmware" [2] that have modified the power delivery from the USB ports. [1] Testing involved running several disk benchmarks as well as 'stress-ng' to load the storage as much as possible. [2] I'm unclear what the Rpi folk mean by "firmware". As an embedded systems developer "firmware" is something written to non-volatile solid state storage like flash or eeprom but when I updated firmware on a Pi, it installed a new kernel too.