6 ms·

Okay, so maybe I'm missing the point here, but, uh... No shit? Is there anybody who seriously is tossing 135Tb at a single JBOD? I'm not a storage guy, not e

by blhack 15y ago

Okay, so maybe I'm missing the point here, but, uh... No shit? Is there anybody who seriously is tossing 135Tb at a single JBOD? I'm not a storage guy, not even close, and even I know that that is a frighteningly ridiculous, get-you-fired idea. These are meant as individual parts in a larger, highly redundant setup. That's the entire point. Is the next post going to be: "Why you won't be able to download music at 1Gbps even with a GIGABIT SWITCH! Cisco lies!"?

tommi 15y ago
Read the post. In particular "Why the folks at Backblaze don’t care about the “risks”" section.
- blhack 15y ago
  I know, my point is that anybody with enough knowledge about this topic to be interested in this article would already know.
smackfu 15y ago
It seems like an interesting one-off kind of scenario. They have a lot of downloaded data, none of which is irreplaceable, but which would take a significant amount of time to redownload. This pod is a backup but a not very rigorous one, since the goal is to never have to download all the data, not to never have to download any of the data.
serverascode 15y ago
I'm looking at enclosures that can hold 60 disks. So 3TB SAS disks would be 180TB. Most enclosures of that kind have dual everything. Some SAS drives even have dual connections I believe (should look that up again). At some point it's really a function of the size of the drives that makes the volume large. I think 60 drives is going to pretty much be the physical limit in 4 rack units. You can buy the NL-108 from Isilon which has 108TB (36 3TB disks) if you want "enterprise." ;) Eventually they'll use 4TB drives in those.
soult 15y ago
I take it you only read the first part of the article. They used RAID6 over 15 drives each, combining those 15 drives via LVM, allowing the loss of 2 drives from the same group without data loss. This is the same configuration Backblaze uses as well, because even though their fancy custom software mirrors all data to different storage pods, it is easier to just replace a drive and rebuild the raid than rebuilding the whole JBOD from the data on other pods.
- nobody314159 15y ago
  The problem is that a large number of the failures that would take out 2 of 15 drives in a raid array will also take out all 15! Think bad power supply, mains spike, cooling failure, fire, flood ..... If you need raid, you also need a separate copy in a separate location.
  - nknight 15y ago
    Your point isn't entirely invalid, but I lose HDDs a hell of a lot more often than I encounter any of the other problems you mention. I don't think I've seen a power supply pop since 2006, I've never lost UPS/surge-protected gear to a mains spike, every server room or datacenter I've cared about has been environmentally monitored, heptafluoropropane exists for a reason, and why are you building a datacenter on a floodplain? In contrast, I lose 5-10% of the spinning HDDs I care about every year.
    - trin_ 15y ago
      just curious but do you have any data for SSD in use?
      - nknight 15y ago
        No, I've not dealt with enough of them for long enough to have anything meaningful to say on the subject. So far I've not even had one clearly die (though there have been some very odd transient incidents.) Jeff Atwood made an interesting post on that very subject a while back, though: http://www.codinghorror.com/blog/2011/05/the-hot-crazy-solid-state-drive-scale.html http://www.codinghorror.com/blog/2011/05/the-hot-crazy-solid...

by blhack 15y ago

tommi 15y ago

Read the post. In particular "Why the folks at Backblaze don’t care about the “risks”" section.

blhack 15y ago
I know, my point is that anybody with enough knowledge about this topic to be interested in this article would already know.

smackfu 15y ago

It seems like an interesting one-off kind of scenario. They have a lot of downloaded data, none of which is irreplaceable, but which would take a significant amount of time to redownload. This pod is a backup but a not very rigorous one, since the goal is to never have to download all the data, not to never have to download any of the data.

serverascode 15y ago

I'm looking at enclosures that can hold 60 disks. So 3TB SAS disks would be 180TB. Most enclosures of that kind have dual everything. Some SAS drives even have dual connections I believe (should look that up again). At some point it's really a function of the size of the drives that makes the volume large. I think 60 drives is going to pretty much be the physical limit in 4 rack units. You can buy the NL-108 from Isilon which has 108TB (36 3TB disks) if you want "enterprise." ;) Eventually they'll use 4TB drives in those.

soult 15y ago

I take it you only read the first part of the article. They used RAID6 over 15 drives each, combining those 15 drives via LVM, allowing the loss of 2 drives from the same group without data loss. This is the same configuration Backblaze uses as well, because even though their fancy custom software mirrors all data to different storage pods, it is easier to just replace a drive and rebuild the raid than rebuilding the whole JBOD from the data on other pods.

nobody314159 15y ago
The problem is that a large number of the failures that would take out 2 of 15 drives in a raid array will also take out all 15! Think bad power supply, mains spike, cooling failure, fire, flood ..... If you need raid, you also need a separate copy in a separate location.
- nknight 15y ago
  Your point isn't entirely invalid, but I lose HDDs a hell of a lot more often than I encounter any of the other problems you mention. I don't think I've seen a power supply pop since 2006, I've never lost UPS/surge-protected gear to a mains spike, every server room or datacenter I've cared about has been environmentally monitored, heptafluoropropane exists for a reason, and why are you building a datacenter on a floodplain? In contrast, I lose 5-10% of the spinning HDDs I care about every year.
  - trin_ 15y ago
    just curious but do you have any data for SSD in use?
    - nknight 15y ago
      No, I've not dealt with enough of them for long enough to have anything meaningful to say on the subject. So far I've not even had one clearly die (though there have been some very odd transient incidents.) Jeff Atwood made an interesting post on that very subject a while back, though: http://www.codinghorror.com/blog/2011/05/the-hot-crazy-solid-state-drive-scale.html http://www.codinghorror.com/blog/2011/05/the-hot-crazy-solid...