3 ms·

don't build it yourself Sorry, but what a bunch of FUD. Any seasoned sysadmin can build an array with such low requirements. It's a different story when you n

by moe 16y ago

don't build it yourself Sorry, but what a bunch of FUD. Any seasoned sysadmin can build an array with such low requirements. It's a different story when you need high IOPS or extreme availability - but none of that was asked for here. Wazoox listed the off the shelf parts that you need, from there it's a matter of being able to operate a screwdriver and configuring your favorite RAID controller or soft-raid. Of course you can double-up and buy the feel-good solution from a storage-vendor (DELL MD1000 would be the standard in this range). But in this case it doesn't seem worth to pay a markup for essentially the same hardware.

phamilton 16y ago
Yes, but next day support makes all the difference. When I worked as an HPC sysadmin, we saw about 2-3 DIMM failures a week in our 10,000 core cluster. I'm sure we could have gotten the hardware cheaper, even with replacement DIMMs for the 3 years period of service, but Dell made it quite frictionless. A simple web login and form and the new DIMM would show up the next day. We ship the old one back (using their shipping label) and move on. Our time was worth more than we would have saved building it ourselves. 500 TB in 1 or 2 TB drives is a lot of hardware. <complete back of the hand calculation> If we have 250 drives with a 1 000 000 hour mtbf, then (according to an exponential distribution), you've got around ~5% chance of a drive failure each week. Not huge, but worth noting. </ complete back of the hand calculation>
- moe 16y ago
  Well, a 10k core cluster is a far cry from a 500TB array. The latter fits comfortably into 2-3 racks (if your DC allows the power density). The former sounds like 15+ racks - entirely different ballgame. I'm absolutely not denying that you should have a dedicated admin to look after such a deployment. But by today's standards it's just not a lot of hardware anymore. And definitely not the size where you must spend a fortune on "platinum" support contracts that can easily cost as much (per year) as a fully fleshed out spare shelf... FWIW, I'm personally running a few moderately sized storage clusters (the largest is ~200T, 3x3 MD1000) and we're seeing disk failures at a rate of 2-3 per year.

by moe 16y ago

phamilton 16y ago

Yes, but next day support makes all the difference. When I worked as an HPC sysadmin, we saw about 2-3 DIMM failures a week in our 10,000 core cluster. I'm sure we could have gotten the hardware cheaper, even with replacement DIMMs for the 3 years period of service, but Dell made it quite frictionless. A simple web login and form and the new DIMM would show up the next day. We ship the old one back (using their shipping label) and move on. Our time was worth more than we would have saved building it ourselves. 500 TB in 1 or 2 TB drives is a lot of hardware. <complete back of the hand calculation> If we have 250 drives with a 1 000 000 hour mtbf, then (according to an exponential distribution), you've got around ~5% chance of a drive failure each week. Not huge, but worth noting. </ complete back of the hand calculation>

moe 16y ago
Well, a 10k core cluster is a far cry from a 500TB array. The latter fits comfortably into 2-3 racks (if your DC allows the power density). The former sounds like 15+ racks - entirely different ballgame. I'm absolutely not denying that you should have a dedicated admin to look after such a deployment. But by today's standards it's just not a lot of hardware anymore. And definitely not the size where you must spend a fortune on "platinum" support contracts that can easily cost as much (per year) as a fully fleshed out spare shelf... FWIW, I'm personally running a few moderately sized storage clusters (the largest is ~200T, 3x3 MD1000) and we're seeing disk failures at a rate of 2-3 per year.