4 ms·

fwiw - as an addendum to that comment - I like using S3 to get my data off the zero w ASAP. ..then corrupt os etc doesn't cause too much down time and data loss

fwiw - as an addendum to that comment - I like using S3 to get my data off the zero w ASAP. ..then corrupt os etc doesn't cause too much down time and data loss.

jws 8y ago
If you store your data under /run (or /var/run) your district will probably put it on a RAM disk. You’ll lose it in a crash, but you won’t be writing to the SD card and provoking it to corrupt in that crash. If you then adjust your systemd journal to not write very often you have eliminated most of your SD writes. The remote sensor systems I run lose power from time to time during the year, sometimes once a day. I’d say each time there used to be something like a 0.1% to 1% chance of corruption. Since I’ve moved to /run and a slower journal I haven’t lost an SD card[1]. Thus far it seems I have at least an order of magnitude improvement. [1] Well, I had a wind turbine tower collapse in an inaccessible location. The raspberry pi is in a lake now, but it can’t have gone far so I don’t call it “lost”. I’ll dive and recover it in the spring. That camera/sensor has spent a couple months underwater before. In fact, I’ve had three systems end up underwater for months and all the SD cards have been fine when recovered.
- ageofwant 8y ago
  Mate, seems you have your data recovery issues licked. Perhaps now is the time to spend some time on the structural integrity of your windmills :-)
- mrsteveman1 8y ago
  > If you store your data under /run (or /var/run) your district will probably put it on a RAM disk. You’ll lose it in a crash, but you won’t be writing to the SD card and provoking it to corrupt in that crash. > If you then adjust your systemd journal to not write very often you have eliminated most of your SD writes. The trouble is that the OS isn't the only one writing to flash in the card, the MicroSD card controller is erasing and writing to it at random times as well even if the filesystem is read-only, with no coordination between it and the OS. As a result you can easily end up in situations where the card decided to run some housekeeping to manage the flash just milliseconds before a power cycle or a power loss, and seemingly all consumer MicroSD cards are not up to the task of keeping the data intact at the same time.

fwiw - as an addendum to that comment - I like using S3 to get my data off the zero w ASAP. ..then corrupt os etc doesn't cause too much down time and data loss.

jws 8y ago

If you store your data under /run (or /var/run) your district will probably put it on a RAM disk. You’ll lose it in a crash, but you won’t be writing to the SD card and provoking it to corrupt in that crash. If you then adjust your systemd journal to not write very often you have eliminated most of your SD writes. The remote sensor systems I run lose power from time to time during the year, sometimes once a day. I’d say each time there used to be something like a 0.1% to 1% chance of corruption. Since I’ve moved to /run and a slower journal I haven’t lost an SD card[1]. Thus far it seems I have at least an order of magnitude improvement. [1] Well, I had a wind turbine tower collapse in an inaccessible location. The raspberry pi is in a lake now, but it can’t have gone far so I don’t call it “lost”. I’ll dive and recover it in the spring. That camera/sensor has spent a couple months underwater before. In fact, I’ve had three systems end up underwater for months and all the SD cards have been fine when recovered.

ageofwant 8y ago
Mate, seems you have your data recovery issues licked. Perhaps now is the time to spend some time on the structural integrity of your windmills :-)
mrsteveman1 8y ago
> If you store your data under /run (or /var/run) your district will probably put it on a RAM disk. You’ll lose it in a crash, but you won’t be writing to the SD card and provoking it to corrupt in that crash. > If you then adjust your systemd journal to not write very often you have eliminated most of your SD writes. The trouble is that the OS isn't the only one writing to flash in the card, the MicroSD card controller is erasing and writing to it at random times as well even if the filesystem is read-only, with no coordination between it and the OS. As a result you can easily end up in situations where the card decided to run some housekeeping to manage the flash just milliseconds before a power cycle or a power loss, and seemingly all consumer MicroSD cards are not up to the task of keeping the data intact at the same time.