9 ms·

It's not a best practice, but the last 10 years I've run my home server with a smaller faster drive for the OS and a single larger disk for bulk storage that I

It's not a best practice, but the last 10 years I've run my home server with a smaller faster drive for the OS and a single larger disk for bulk storage that I choose using Backblaze Drive Stats. None of have failed yet (fingers-crossed). I really trust their methodology and it's an extremely valuable resource for me as a consumer. My most recent drive is a WDC WUH722222ALE6L4 22TiB, and looking at the stats (albeit only a few months of data), and overall trend of WDC, in this report gives me peace of mind that it should be fine for the next few years until it's time for the cycle to repeat.

kridsdale1 2y ago
No RAID 0 for the bulk storage? What’s your disaster plan?
- ecliptik 2y ago
  restic + rclone to cloud storage for data I care about, the majority of the data can easily be replaced if needed.
  - manosyja 2y ago
    That’s exactly how I do it.
- SteveNuts 2y ago
  Surely you mean RAID 1? Or 5, 6, 10 perhaps?
- BSDobelix 2y ago
  Disaster-plan is always backup (away from location) or out-of-house replication, raid is NOT a backup but a part of a system to keep uptime high and hands-on low (like redundant power and supply) Disaster = Your DC or Cellar is flooded or burned down ;)
qskousen 2y ago
I'm sure you're aware but consider putting another drive in for some flavor of RAID, it's a lot easier to rebuild a RAID than to rebuild data usually! Edit: By "some flavor" I mean hardware or software.
- walrus01 2y ago
  RAID doesn't cover all of the scenarios as offsite backup, such as massive electrical power surge, fire, flood, theft or other things causing total destruction of the RAID array. Ideally you'd want a setup that has local storage redundancy in some form of RAID and offsite backup.
  - bombcar 2y ago
    In fact for home users backup is WAY more important than RAID, because your NAS down for a (restore time) is not that important, but data loss is forever.
    - didntcheck 2y ago
      For essential personal data you're right, but a very common use case for a home NAS is a media server. The library is usually non-essential data - annoying to lose, but not critical. Combined with its large size, it's usually hard to justify a full offsite backup. RAID offers a cost-effective way to give it some protection, when the alternative is nothing
      - walrus01 2y ago
        For a number of people I know, they don't do any offsite backup of their home media server. It would not result in any possibly-catastrophic personal financial hassles/struggles/real data loss if a bunch of movies and music disappeared overnight. The amount of personally generated sensitive data that doesn't fit on a laptop's onboard storage (which should all be backed up offsite as well) will usually fit on like a 12TB RAID-1 pair, which is easier to back up than 40TB+ of movies.
gruez 2y ago
>It's not a best practice, but the last 10 years I've run my home server with a smaller faster drive for the OS and a single larger disk for bulk storage that I choose using Backblaze Drive Stats. None of have failed yet (fingers-crossed). I really trust their methodology and it's an extremely valuable resource for me as a consumer. I also have multiple drives in operation in the past decade and didn't experience any failures. However unlike you, I didn't use backblaze's drive stats to inform my purchase. I just bought whatever was cheapest, knowing that any TCO reduction from higher reliability (at best, around 10%) would eaten up by the lack of discounts the "best" drive. That's the problem with n=1 anecdotes. You don't know whether nothing bad happened because you followed "the right advice", or you just got lucky.
eatbitseveryday 2y ago
> WDC WUH722222ALE6L4 22TiB Careful... that is 22 TB, not 22 TiB. Disk marketing still uses base 10. TiB is base 2. 22 TB = 20 TiB
CTDOCodebases 2y ago
Take these stats with a grain of salt. I am becoming more and more convinced that hard drive reliability is linked to the batch more than to the individual drive models themselves. Often you will read online of people experiencing multiple failures from drives purchased from the same batch. I cannot prove this because I have no idea about Blackblazes procurement patterns but I bought one of the better drives in this list (ST16000NM001G) and it failed within a year. When it comes to hard drives or storage more generally a better approach is protect yourself against down time with software raid and backups and pray that if a drive does fail it does so within the warranty period.
- bragr 2y ago
  >Often you will read online of people experiencing multiple failures from drives purchased from the same batch I'll toss in on that anecdata. This has happened to me a several times. In all these cases we were dealing with drives with more or less sequential serial numbers. In two instances they were just cache drives for our CDN nodes. Not a big deal, but I sure kept the remote hands busy those weeks trying to keep enough nodes online. In a prior job, it was our primary storage array. You'd think that RAID6+hot spare would be pretty robust, but 3 near simultaneous drive failures made a mockery of that. That was a bad day. The hot spare starting doing its thing with the first failure, and if it had finished rebuilding before the subsequent failures, we'd have been ok, but alas.
  - tharkun__ 2y ago
    This has been the "conventional wisdom" for a very long time. Is this one of those things that get "lost with time" and every generation has to rediscover it? Like, 25+ years ago I would've bought hard drives for just my personal usage in a software raid making sure I don't get consecutive serial numbers, but ones that are very different. I'd go to my local hardware shop and ask them specifically for that. They'd show me the drives / serial numbers before I ever even bought them for real. I even used different manufacturers at some point when they didn't have non consecutive serials. I lost some storage because the drives weren't exactly the same size even though the advertized size matched, but better than having the RAID and extra cost be for nothing. I can't fathom how anyone that is running drives in actual production wouldn't have been doing that.
whalesalad 2y ago
hopefully you have 2x of these drives in some kind of raid mirror such that if one fails, you can simply replace it and re-mirror. not having something like this is risky.
- hypothesis 2y ago
  Wasn’t the issue with large drives that remaining drive has a high chance of failure during re-silvering?
  - fc417fc802 2y ago
    If you're doing statistics to plan the configuration of a large cluster with high availability, then yes. For home use where failures are extremely rare, no. Home use is also much more likely to suffer from unexpected adverse conditions that impact all the drives in the array simultaneously.
  - dapperdrake 2y ago
    Just triple mirror with cheap drives from different manufacturers.
  - HankB99 2y ago
    That may be true for pools that never get scrubbed. Or for management that doesn't watch SMART stats in order to catch a situation before it degrades to the point where one drive fails and another is on its last legs. With ZFS on Debian the default is to scrub monthly (second Sunday) and resilvering is not more stressful than that. The entire drive contents (not allocated space) has to be read to re-silver. Also define "high chance." Is 10% high? 60%? I've replaced failed drives or just ones I wanted to swap to a larger size at least a dozen times and never had a concurrent failure.
loeg 2y ago
I switched to TLC flash last time around and no regrets. With QLC the situations where HDDs are cheaper, including the cost of power, are growing narrower and narrower.
- MrDrMcCoy 2y ago
  It really depends on your usage patterns. Write-heavy workloads are still better cases for spinning rust due to how much harder they are on flash, especially at greater layer depths.
  - Aachen 2y ago
    Plus that SSDs apparently have a very dirty manufacturing process, worse than the battery or screen in your laptop. I recently learned this because the EU is starting to require reporting CO2e for products (mentioned on a Dutch podcast: https://tweakers.net/geek/230852/tweakers-podcast-356-switch-2-verwachtingen-tiktok-spanningen-en-ces-videos.html https://tweakers.net/geek/230852/tweakers-podcast-356-switch...). I don't know how a hard drive stacks up but if the SSD is the worst of all of a laptop's components, odds are that it's better and so one could make the decision to use one or the other based on whether an SSD is needed rather than just tossing it in because it's cheap Probably it also matters if you get a bulky 3.5" HDD when all you need is a small flash chip with a few GB of persistent storage — the devil is in the details but I simply didn't realise this could be a part of the decision process
    - mercutio2 2y ago
      If this is really a significant concern for you, are you accounting for the CO2e of the (very significant) difference in energy consumption over the lifetime of the device? It seems unlikely to me that in a full lifecycle accounting the spinning rust would come out ahead.
      - Aachen 2y ago
        The figure already includes the lifetime energy consumption and it's comparatively insignificant. The calculation even includes expected disposal and recycling! It sounded really comprehensive besides having to make assumptions about standard usage patterns, but then the usage is like 10% of the lifetime emissions so it makes a comparatively small difference if I'm a heavy gamer or leave it to sit and collect dust: 90% remains the same > If this is really a significant concern for you It literally affects everyone I'm afraid and simply not knowing about it (until now) doesn't stop warming either. Yes, this concerns everyone, although not everyone has the means to do something about it (like to buy the cleaner product)
Dylan16807 2y ago
Nobody should ever have peace of mind about a single drive. You probably have odds around 5% that the storage drive fails each cycle, and another 5% for the OS drive. That's significant. And in your particular situation, 3 refurbished WUH721414ALE6L4 are the same total price. If you put those in RAIDZ1 then that's 28TB with about as much reliability as you can hope to have in a single device. (With backups still being important but that's a separate topic.)
- meindnoch 2y ago
  >You probably have odds around 5% that the storage drive fails each cycle What do you mean by cycle?
  - Dylan16807 2y ago
    "My most recent drive [...] it should be fine for the next few years until it's time for the cycle to repeat." The amount of time they stay on a single drive.
    - deelowe 2y ago
      Drive manufacturers often publish the AFR. From there you can do the math to figure out what sort of redundancy you need. Rule of thumb is that the AFR should be in the 1-2% range. I haven't looked at BB's data, but I'm sure it supports this. Note, disk failure rates and raid or similar solutions should be used when establishing an availability target, not for protecting against data loss. If data loss is a concern, the approach should be to use back ups.
      - Dylan16807 2y ago
        You picked a weird place to reply, because that comment is just saying what "cycle" means. But yes, I've done the math. I'm just going with the BB numbers here, and after a few years it adds up. The way I understand "peace of mind", you can't have it with a single drive. Nice and simple.
_huayra_ 2y ago
I end up doing this too, but ensure that the "single data disk" is regularly backed up offsite too (several times a day, zfs send makes it easy). One needs an offsite backup anyway, and as long as your home server data workload isn't too high and you know how to restore (which should be practiced every so often), this can definitely work.

kridsdale1 2y ago

No RAID 0 for the bulk storage? What’s your disaster plan?

ecliptik 2y ago
restic + rclone to cloud storage for data I care about, the majority of the data can easily be replaced if needed.
- manosyja 2y ago
  That’s exactly how I do it.
SteveNuts 2y ago
Surely you mean RAID 1? Or 5, 6, 10 perhaps?
BSDobelix 2y ago
Disaster-plan is always backup (away from location) or out-of-house replication, raid is NOT a backup but a part of a system to keep uptime high and hands-on low (like redundant power and supply) Disaster = Your DC or Cellar is flooded or burned down ;)

qskousen 2y ago

I'm sure you're aware but consider putting another drive in for some flavor of RAID, it's a lot easier to rebuild a RAID than to rebuild data usually! Edit: By "some flavor" I mean hardware or software.

walrus01 2y ago
RAID doesn't cover all of the scenarios as offsite backup, such as massive electrical power surge, fire, flood, theft or other things causing total destruction of the RAID array. Ideally you'd want a setup that has local storage redundancy in some form of RAID and offsite backup.
- bombcar 2y ago
  In fact for home users backup is WAY more important than RAID, because your NAS down for a (restore time) is not that important, but data loss is forever.
  - didntcheck 2y ago
    For essential personal data you're right, but a very common use case for a home NAS is a media server. The library is usually non-essential data - annoying to lose, but not critical. Combined with its large size, it's usually hard to justify a full offsite backup. RAID offers a cost-effective way to give it some protection, when the alternative is nothing
    - walrus01 2y ago
      For a number of people I know, they don't do any offsite backup of their home media server. It would not result in any possibly-catastrophic personal financial hassles/struggles/real data loss if a bunch of movies and music disappeared overnight. The amount of personally generated sensitive data that doesn't fit on a laptop's onboard storage (which should all be backed up offsite as well) will usually fit on like a 12TB RAID-1 pair, which is easier to back up than 40TB+ of movies.

gruez 2y ago

>It's not a best practice, but the last 10 years I've run my home server with a smaller faster drive for the OS and a single larger disk for bulk storage that I choose using Backblaze Drive Stats. None of have failed yet (fingers-crossed). I really trust their methodology and it's an extremely valuable resource for me as a consumer. I also have multiple drives in operation in the past decade and didn't experience any failures. However unlike you, I didn't use backblaze's drive stats to inform my purchase. I just bought whatever was cheapest, knowing that any TCO reduction from higher reliability (at best, around 10%) would eaten up by the lack of discounts the "best" drive. That's the problem with n=1 anecdotes. You don't know whether nothing bad happened because you followed "the right advice", or you just got lucky.

eatbitseveryday 2y ago

> WDC WUH722222ALE6L4 22TiB Careful... that is 22 TB, not 22 TiB. Disk marketing still uses base 10. TiB is base 2. 22 TB = 20 TiB

CTDOCodebases 2y ago

Take these stats with a grain of salt. I am becoming more and more convinced that hard drive reliability is linked to the batch more than to the individual drive models themselves. Often you will read online of people experiencing multiple failures from drives purchased from the same batch. I cannot prove this because I have no idea about Blackblazes procurement patterns but I bought one of the better drives in this list (ST16000NM001G) and it failed within a year. When it comes to hard drives or storage more generally a better approach is protect yourself against down time with software raid and backups and pray that if a drive does fail it does so within the warranty period.

bragr 2y ago
>Often you will read online of people experiencing multiple failures from drives purchased from the same batch I'll toss in on that anecdata. This has happened to me a several times. In all these cases we were dealing with drives with more or less sequential serial numbers. In two instances they were just cache drives for our CDN nodes. Not a big deal, but I sure kept the remote hands busy those weeks trying to keep enough nodes online. In a prior job, it was our primary storage array. You'd think that RAID6+hot spare would be pretty robust, but 3 near simultaneous drive failures made a mockery of that. That was a bad day. The hot spare starting doing its thing with the first failure, and if it had finished rebuilding before the subsequent failures, we'd have been ok, but alas.
- tharkun__ 2y ago
  This has been the "conventional wisdom" for a very long time. Is this one of those things that get "lost with time" and every generation has to rediscover it? Like, 25+ years ago I would've bought hard drives for just my personal usage in a software raid making sure I don't get consecutive serial numbers, but ones that are very different. I'd go to my local hardware shop and ask them specifically for that. They'd show me the drives / serial numbers before I ever even bought them for real. I even used different manufacturers at some point when they didn't have non consecutive serials. I lost some storage because the drives weren't exactly the same size even though the advertized size matched, but better than having the RAID and extra cost be for nothing. I can't fathom how anyone that is running drives in actual production wouldn't have been doing that.

whalesalad 2y ago

hopefully you have 2x of these drives in some kind of raid mirror such that if one fails, you can simply replace it and re-mirror. not having something like this is risky.

hypothesis 2y ago
Wasn’t the issue with large drives that remaining drive has a high chance of failure during re-silvering?
- fc417fc802 2y ago
  If you're doing statistics to plan the configuration of a large cluster with high availability, then yes. For home use where failures are extremely rare, no. Home use is also much more likely to suffer from unexpected adverse conditions that impact all the drives in the array simultaneously.
- dapperdrake 2y ago
  Just triple mirror with cheap drives from different manufacturers.
- HankB99 2y ago
  That may be true for pools that never get scrubbed. Or for management that doesn't watch SMART stats in order to catch a situation before it degrades to the point where one drive fails and another is on its last legs. With ZFS on Debian the default is to scrub monthly (second Sunday) and resilvering is not more stressful than that. The entire drive contents (not allocated space) has to be read to re-silver. Also define "high chance." Is 10% high? 60%? I've replaced failed drives or just ones I wanted to swap to a larger size at least a dozen times and never had a concurrent failure.

loeg 2y ago

I switched to TLC flash last time around and no regrets. With QLC the situations where HDDs are cheaper, including the cost of power, are growing narrower and narrower.

MrDrMcCoy 2y ago
It really depends on your usage patterns. Write-heavy workloads are still better cases for spinning rust due to how much harder they are on flash, especially at greater layer depths.
- Aachen 2y ago
  Plus that SSDs apparently have a very dirty manufacturing process, worse than the battery or screen in your laptop. I recently learned this because the EU is starting to require reporting CO2e for products (mentioned on a Dutch podcast: https://tweakers.net/geek/230852/tweakers-podcast-356-switch-2-verwachtingen-tiktok-spanningen-en-ces-videos.html https://tweakers.net/geek/230852/tweakers-podcast-356-switch...). I don't know how a hard drive stacks up but if the SSD is the worst of all of a laptop's components, odds are that it's better and so one could make the decision to use one or the other based on whether an SSD is needed rather than just tossing it in because it's cheap Probably it also matters if you get a bulky 3.5" HDD when all you need is a small flash chip with a few GB of persistent storage — the devil is in the details but I simply didn't realise this could be a part of the decision process
  - mercutio2 2y ago
    If this is really a significant concern for you, are you accounting for the CO2e of the (very significant) difference in energy consumption over the lifetime of the device? It seems unlikely to me that in a full lifecycle accounting the spinning rust would come out ahead.
    - Aachen 2y ago
      The figure already includes the lifetime energy consumption and it's comparatively insignificant. The calculation even includes expected disposal and recycling! It sounded really comprehensive besides having to make assumptions about standard usage patterns, but then the usage is like 10% of the lifetime emissions so it makes a comparatively small difference if I'm a heavy gamer or leave it to sit and collect dust: 90% remains the same > If this is really a significant concern for you It literally affects everyone I'm afraid and simply not knowing about it (until now) doesn't stop warming either. Yes, this concerns everyone, although not everyone has the means to do something about it (like to buy the cleaner product)

Dylan16807 2y ago

Nobody should ever have peace of mind about a single drive. You probably have odds around 5% that the storage drive fails each cycle, and another 5% for the OS drive. That's significant. And in your particular situation, 3 refurbished WUH721414ALE6L4 are the same total price. If you put those in RAIDZ1 then that's 28TB with about as much reliability as you can hope to have in a single device. (With backups still being important but that's a separate topic.)

meindnoch 2y ago
>You probably have odds around 5% that the storage drive fails each cycle What do you mean by cycle?
- Dylan16807 2y ago
  "My most recent drive [...] it should be fine for the next few years until it's time for the cycle to repeat." The amount of time they stay on a single drive.
  - deelowe 2y ago
    Drive manufacturers often publish the AFR. From there you can do the math to figure out what sort of redundancy you need. Rule of thumb is that the AFR should be in the 1-2% range. I haven't looked at BB's data, but I'm sure it supports this. Note, disk failure rates and raid or similar solutions should be used when establishing an availability target, not for protecting against data loss. If data loss is a concern, the approach should be to use back ups.
    - Dylan16807 2y ago
      You picked a weird place to reply, because that comment is just saying what "cycle" means. But yes, I've done the math. I'm just going with the BB numbers here, and after a few years it adds up. The way I understand "peace of mind", you can't have it with a single drive. Nice and simple.

_huayra_ 2y ago

I end up doing this too, but ensure that the "single data disk" is regularly backed up offsite too (several times a day, zfs send makes it easy). One needs an offsite backup anyway, and as long as your home server data workload isn't too high and you know how to restore (which should be practiced every so often), this can definitely work.