4 ms·

Can someone explain this? I find it a bit scary that your have to do monthly manual work on your NAS or you'll lose. Edit: wait, I think I get it... "Scrubbing

Can someone explain this? I find it a bit scary that your have to do monthly manual work on your NAS or you'll lose. Edit: wait, I think I get it... "Scrubbing simply needs to read every file from the disk so the RAID layer notices and repairs a URE". So it's to avoid bit rot? I have to say, as someone who has a few TB of personal data on a NAS, bit rot is a bit scary. Data backup is mainly why I bought the lifetime pCloud+encryption package during the last Black Friday. I wonder how (if?) they avoid bit rot?

dathinab 5y ago
> So it's to avoid bit rot? Yes, it's also trivially to automatize in more or less all situations. The gotcha is that you have to do it/know about it...
- francis-io 5y ago
  I know that TrueNAS/FreeNAS add a scrub by default. Possibly ZFS on Linux does now too. LinusTechTips recently did a video about how they installed ZFS on Linux and didn't have a scrub cron. They started to lose data before noticing. https://youtu.be/Npu7jkJk5nM https://youtu.be/Npu7jkJk5nM
  - Annatar 5y ago
  - ap-andersson 5y ago
    I believe I got default cron-files when I installed ZFS on ubuntu 21.04. It could be that they were created and I had to uncomment one line in a file. Then a scrub on the pools would run once a month. Then I setup email on the server and everytime a scrub is done with any errors, I get an email. Quite easy for me to setup, even though its my first NAS that I built myself and first time using ZFS. Very surprised that LTT effed that up to be honest.
  - dathinab 5y ago
    It's less Linux and more weather or not you have a database/NAS centric distribution I think. The problem with by default installing a cron job when installing ZFS is that for a general purpose OS there a good default for when and how often to run it. And running it on the wrong time might even be a major problem. Through then tbh. having a bad default is probably still better then no default in this case. > lose data before noticing Is a bit of an overstatement as they didn't look for quite a while, they also did not only fail to do scrubbing, they also failed to setup automated health checks and reporting. Turning a non NAS focused Linux distribution into a well working and tuned NAS isn't easy (compared to using a good NAS OS/distribution), but making it somewhat work is easy. Which makes this a pretty common mistake for non-specialized people (i.e. like in their case).
fomine3 5y ago
just add zpool scrub to your crontab.
- throw0101a 5y ago
  Make make sure the error output of your cron jobs is sent out to a mailbox that you read.
  - fomine3 5y ago
    I recommend to receive events from zed, because not only scrub detect errors. It also support Pushbullet.
ksec 5y ago
That is why I think Consumer NAS is an unsolved problem. I have no need for VPN, Cloud Photos, or Mail Server etc. I need a simple, reliable network storage that prevent drive failure and bit riot. Right now you need to spend at least $400 on a NAS with lots of config before getting it done.
throw0101a 5y ago
> Edit: wait, I think I get it... "Scrubbing simply needs to read every file from the disk so the RAID layer notices and repairs a URE". If you don't know about ZFS scrubbing, but are using ZFS, you may wish to spend some time researching ZFS some more. > I find it a bit scary that your have to do monthly manual work on your NAS or you'll lose. Most distros / ZFS packages set up a cron job and e-mail out any errors. You do receive error reports from your NAS, right?

dathinab 5y ago

> So it's to avoid bit rot? Yes, it's also trivially to automatize in more or less all situations. The gotcha is that you have to do it/know about it...

francis-io 5y ago
I know that TrueNAS/FreeNAS add a scrub by default. Possibly ZFS on Linux does now too. LinusTechTips recently did a video about how they installed ZFS on Linux and didn't have a scrub cron. They started to lose data before noticing. https://youtu.be/Npu7jkJk5nM https://youtu.be/Npu7jkJk5nM
- Annatar 5y ago
- ap-andersson 5y ago
  I believe I got default cron-files when I installed ZFS on ubuntu 21.04. It could be that they were created and I had to uncomment one line in a file. Then a scrub on the pools would run once a month. Then I setup email on the server and everytime a scrub is done with any errors, I get an email. Quite easy for me to setup, even though its my first NAS that I built myself and first time using ZFS. Very surprised that LTT effed that up to be honest.
- dathinab 5y ago
  It's less Linux and more weather or not you have a database/NAS centric distribution I think. The problem with by default installing a cron job when installing ZFS is that for a general purpose OS there a good default for when and how often to run it. And running it on the wrong time might even be a major problem. Through then tbh. having a bad default is probably still better then no default in this case. > lose data before noticing Is a bit of an overstatement as they didn't look for quite a while, they also did not only fail to do scrubbing, they also failed to setup automated health checks and reporting. Turning a non NAS focused Linux distribution into a well working and tuned NAS isn't easy (compared to using a good NAS OS/distribution), but making it somewhat work is easy. Which makes this a pretty common mistake for non-specialized people (i.e. like in their case).

fomine3 5y ago

just add zpool scrub to your crontab.

throw0101a 5y ago
Make make sure the error output of your cron jobs is sent out to a mailbox that you read.
- fomine3 5y ago
  I recommend to receive events from zed, because not only scrub detect errors. It also support Pushbullet.

ksec 5y ago

That is why I think Consumer NAS is an unsolved problem. I have no need for VPN, Cloud Photos, or Mail Server etc. I need a simple, reliable network storage that prevent drive failure and bit riot. Right now you need to spend at least $400 on a NAS with lots of config before getting it done.

throw0101a 5y ago

> Edit: wait, I think I get it... "Scrubbing simply needs to read every file from the disk so the RAID layer notices and repairs a URE". If you don't know about ZFS scrubbing, but are using ZFS, you may wish to spend some time researching ZFS some more. > I find it a bit scary that your have to do monthly manual work on your NAS or you'll lose. Most distros / ZFS packages set up a cron job and e-mail out any errors. You do receive error reports from your NAS, right?