12 ms·

You seem very knowledgeable about this. I'm a hobbyist who runs debian with a 6 drive raidz2 array in the basement. Hardware aside, do you have any housekeepin

by zeagle 5y ago

You seem very knowledgeable about this. I'm a hobbyist who runs debian with a 6 drive raidz2 array in the basement. Hardware aside, do you have any housekeeping suggestions that will help me keep it running well? My approach is a login script that tells me the health of my zpool. My crontab has a "0 2 * * 0 /sbin/zpool scrub tank" and 6 variants of "0 2 2 * * /usr/sbin/smartctl --test=long /dev/sda &> /dev/null". I've learned to resilver a dead drive recently, it's on a UPS, and I have automated iterative backups elsewhere for critical data that I've practiced restoring from to verify my solution works. Never had much luck with email alerts unless I want to get all of crontab's emails sent to me.

chungy 5y ago
Sounds like you've already got the major stuff covered :) Could also look into using zed for getting email notifications about pool problems as soon as they happen.
- zeagle 5y ago
  Thanks, I hadn't heard of zed. More reliable solution than catching a fault with a login script. I was pretty happy when I found msmtp and imagine I can make them work together.
marceldegraaf 5y ago
Aside from the usual cronjobs to scrub my local and backup pools, I do a few extra things: I'm using https://habilis.net/cronic/ https://habilis.net/cronic/ to make sure I don't mess up the email notification part of the cronjob. It's a simple wrapper script that sends an email in a readable format if a cronjob fails. I use Sanoid to create snapshots on my home server, and use Syncoid to push those to a cloud VPS with a beefy network drive as an off-site backup. Both tools are available here: https://github.com/jimsalterjrs/sanoid https://github.com/jimsalterjrs/sanoid The free tier of https://cronitor.io/ https://cronitor.io/ makes sure I'm alerted if a cronjob fails, or fails to run on time. Especially that last bit is interesting: that way I'm sure cronjobs aren't silently failing for days/weeks on end. I have 4 monitors set up in Cronitor: snapshot creation, zpool status on the local and backup machine, and send/receive with Syncoid. This is how that looks on the Cronitor dashboard: https://img.marceldegraaf.net/v6IpNAyxrZ54vgLqIJpY https://img.marceldegraaf.net/v6IpNAyxrZ54vgLqIJpY Let me know if you want more info or examples, happy to share whatever I can to help :-) EDIT: feel free to reach out via email as well, my address is in my profile.
- zeagle 5y ago
  That's great, thanks! I'll take a look at the links.

by zeagle 5y ago

chungy 5y ago

Sounds like you've already got the major stuff covered :) Could also look into using zed for getting email notifications about pool problems as soon as they happen.

zeagle 5y ago
Thanks, I hadn't heard of zed. More reliable solution than catching a fault with a login script. I was pretty happy when I found msmtp and imagine I can make them work together.

marceldegraaf 5y ago

Aside from the usual cronjobs to scrub my local and backup pools, I do a few extra things: I'm using https://habilis.net/cronic/ https://habilis.net/cronic/ to make sure I don't mess up the email notification part of the cronjob. It's a simple wrapper script that sends an email in a readable format if a cronjob fails. I use Sanoid to create snapshots on my home server, and use Syncoid to push those to a cloud VPS with a beefy network drive as an off-site backup. Both tools are available here: https://github.com/jimsalterjrs/sanoid https://github.com/jimsalterjrs/sanoid The free tier of https://cronitor.io/ https://cronitor.io/ makes sure I'm alerted if a cronjob fails, or fails to run on time. Especially that last bit is interesting: that way I'm sure cronjobs aren't silently failing for days/weeks on end. I have 4 monitors set up in Cronitor: snapshot creation, zpool status on the local and backup machine, and send/receive with Syncoid. This is how that looks on the Cronitor dashboard: https://img.marceldegraaf.net/v6IpNAyxrZ54vgLqIJpY https://img.marceldegraaf.net/v6IpNAyxrZ54vgLqIJpY Let me know if you want more info or examples, happy to share whatever I can to help :-) EDIT: feel free to reach out via email as well, my address is in my profile.

zeagle 5y ago
That's great, thanks! I'll take a look at the links.