6 ms·
10 petabytes - visualized
- geekfactor 16y agoI love #3, especially the Stay Puft Marshmallow Man!
- rphlx 16y agoYeah, I think Stay Puft should be mandatory in all big-stack-of-stuff infographics.
- xtacy 16y agoIs there any reason why 10 PB was split in that particular combination of 2, 1.5 and 1TB drives? EDIT: And, just a comparison: Google processes over 20PB of data per day!
- jerf 16y agoIt looks like that is their actual inventory of drives. At that scale, when 2TB drives become the most economical to buy that does not mean one immediately dives in and replaces all the 1.5TB drives. You have to do a bit more math than that. It's also hard to tell whether they're actively decommissioning their 1TB drives or if they've just on a growth curve and they've done significantly more business while 1.5TB and 2TB drives were the more economical choice, but sooner or later the cost/benefit curves do cross and you do want to start decommissioning the smaller drives, it's only a matter of time.
- quinndupont 16y agoThere's a big difference between "storage" and "processing". But it's true, Google stores plenty of data too.
- jodrellblank 16y agoand why it takes more 2TB drives than 1TB drives...
- Retric 16y agoThat's the drives they actually have. 2280 * 2 + 3166 * 1.5 + 1 * 749 = 10,058 TB.
- kayoone 16y agoI am not american, nor do we use inches here in germany but when looking at #3 it should be noted that 5.75 inches is actually the drive length, not height as they state. Well depends on how you look at it but it confused me in the beginning.
- jacquesm 16y agoI'll give you a much more compact way to envision 10 Pb: 10 Petabytes is 10,000,000,000,000,000 bytes or 80,000,000,000,000,000 bits, divided by 8,000,000,000 bits per full human genome (2 bits per base-pair) that's about 10,000,000 cells (not red blood cells because they don't contain DNA), or about 5 milliliters! (10 um diameter on average so about 500 cubic um, so 2 million or so per ml), and that includes all the stuff besides the DNA in the cells. Hard disk storage certainly is impressively compact but it still has a long way to go before we beat mother nature. -1 already eh? Downmodders please correct my math or say what you think is wrong with the comparison, I note the article ends on: "How would you visualize ten petabytes of storage?".
- deleted 16y ago[deleted]
- melpo 16y agoFirst off, I didn't downvote you. However, the comparison is wrong imho because disk storage is not simply about storing information. Disks also need fast read/write access, which DNA certainly doesn't have for us (for now?).
- jacquesm 16y agoWe are envisioning storage, a bunch of disconnected drives piled on top of each other is also not accessible for read/write access.
- ramchip 16y agoI see many debatable points: - Blood is far from being that dense, but I'll assume you mean to calculate a theoretical limit. - If you use the whole of a cell's DNA to encode information, the cell will die. It can't be used to freely encode information like a hard disk. - There is no way to retrieve information in this system. This is a bit like packing a large amount of extremely high-density magnetic platters in a box and calling this a storage system, which is very different from using actual hard drives with all the complex reading/writing system (heads, space between the platters, magnets...). Even if they're not connected, it still takes room. - Reliability of hard disks is far, far better than this. Encoding information as a single copy in a single cell is wildly unsafe. Even using multiple copies, it's likely to degrade with mutations. Edit: I was under the impression that a base pair could only encode one bit of information, since only T-C and A-G are valid combinations. Wikipedia appears to disagree with me but I see no source; does anyone know why Wiki says a base pair could encode 2 bits?
- quinndupont 16y agoMy 350GB of music is backed up on Backblaze (a wonderful service), but seeing those visualizations reminded me of the environmental impact of my digital packratting. That's a sizable data center keeping my data happy and backed up.
- shadowpwner 16y agoOff topic, but what do you listen to that takes up 350GB? Is everything in FLAC?
- wipt 16y agoA fair collection of (DJ) mixes (live or not) and sets can easily account for a few hundred gigabytes. Take Tiesto's "Club Life" and Armin Van Buurin's "A State of Trance" for example. With each set being around two hours or more of high quality mp3, with the former having over 300 episodes, and the latter having around a hundred. These things add up quickly.
- quinndupont 16y agoI listen to a lot of music (from all genres). I only purchase/download full albums (even if most of the songs are not very good), and only listen to at least 196k MP3 (most are at least 256k or higher). Some are Apple Lossless (converted from FLAC). And throw in a few discographies in there and all of a sudden you have 350GB. I've discovered that I add about 50GB/6 months.
- rphlx 16y agoI wonder how many thousands of people are paying them to back up stuff they could easily redownload. Great business model if they de-dupe internally.
- kayoone 16y agothey encrypt all data and i dont think it would be wise of them analyzing their clients data for dupes...if somebody would get to know that their business would be gone.
- skeltoac 16y agoIf you took all of those drives apart and put the spindles in a pile you'd have a big pile of motors. The Petabox consumes 6kW so let's be generous and call it 60kW of servo motors. 60kW is roughly 80hp. (I know this is a specious comparison in several ways but bear with me.) 80hp is enough power a small but highway-capable motorcycle. A single 60kW brushless DC motor weighs over 100kg. That's heavier than a good many of us HN readers. Imagine a motor like that bolted to one long spindle of large platters--say the 99cm platters from a 1961 hard drive. If these platters were remade with the data density of modern 2TB drives, how tall would the spindle be if this giant hard drive had a 10PB capacity? How over- or under-sized would our motor be?
- husein10 16y agoit'd be rad if you could visualize the ratios to better illustrate the scale. 10 petabytes : 10000 terabytes :: 10000 gigabytes : 10 terabytes
- Detrus 16y agoThe visualization in storage pods is most helpful. Just put the drives, in their current size in a physical space you can compare to your office. The skyscraper comparison is fun but misleading. The stack of drives would be very very thin. It would work better if drives were put in one box and that box was compared to a room, house, etc.. There can be other visualizations like electrical power consumption compared to a normal computer. How much time it would take to read/write that data. How many processors are needed to read/write that data in reasonable time, compare their surface area to the our unfolded brain, which is a large cloth napkin. I think these comparisons would be more useful to people in the field than how many books would be needed to store that much data, the standard pop show example.
- cperciva 16y agoAnyone want to guess how much data they actually have stored? Backblaze's pods are a data-loss nightmare -- lots of single points of failure which will wipe out many TB of data at a time -- and backblaze has stated that they replicate data across multiple pods. Given that the 10 PB seems to be the amount of raw storage backblaze has, I'm guessing that the amount of actual data stored is much less -- depending on what sort of erasure correction scheme they're using, of course. (They're still much bigger than Tarsnap, of course!)
- rarrrrrr 16y agoAt SpiderOak we get a 3x replication equivalent for about 35% overhead, using Reed-Solomon at the cluster level (on top of RAID6 at the machine level.) Not nearly as expensive as outright replication. Agree those SATA port multiplies are worrisome. In the beginning, our prototype machines used them to squeeze as many drives into a single machine as possible. They have unusually low tolerance for electrical interference and make it possible for one badly malfunctioning drive to take an entire array offline until manually serviced. We've seen occasions where just touching a cable attached to a port multiplier caused the Linux kernel to emit "dazed and confused" NMI events. I am not brave enough to try them again, even in a redundant setup.
- cperciva 16y ago3x replication equivalent for about 35% overhead How did you compute this "replication equivalent"?
- Retric 16y agoPicked a number from thin air? Raid six requires 2 drives for back and is normally used in set's of 8 or 16 drives but looks like they are using 45 drives. So 45/43 = 4.65% overhead from using RAID. Not if they lose 35% on top of that they are around 41% overhead. But, they are taking a huge it on write speeds, network traffic and reliability for doing so. Edit: Looks like they have 10,058 TB before partitioning the drives so my guess is ~3-6TB of actual user data.
- 16y ago
- bane 16y agoAmazing, a discussion of data without comparing the internationally accepted unit of "library of congress". I'm more interested in how many LoCs 10 Pb is? If we built a city of LoCs, how many acres would that city be? These are the questions that keep me up late at night.
- joanou 16y agoThat's a sizable operation and difficult to visualize. I'm guessing their failure points reduce potential damage to minimal levels. At AltDrive, we use ZFS RaidZ2 with a number of hot spares. Six drives would have to fail consecutively on a given machine before any data loss... we replace them as they occur. And we have inter-H/W duplication. ZFS is self healing and makes management easy. Additionally we periodically compare the integrity of the user's AES-256 CTR encrypted data against the database records and make corrections if necessary. From our customer's and white label provider's view, it just works. http://altdrive.com http://altdrive.com
- ghkluifge 16y ago●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● Gold autumn, personality Mes clothing + Shoes, Travel bag that grabs an eye coat + Chao packet Free transport ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------● ●------ http://u.ly/73I http://u.ly/73I ------●
- drhodes 16y agoOr about 9 (American) football fields covered with 4GB DVDs: https://sites.google.com/site/10petabytes/viz https://sites.google.com/site/10petabytes/viz
- alexqgb 16y agoI had no idea Backblaze had gotten this popular, but I'm glad to see their business growing. If you happen to have the profile they've optimized for, it really is the best thing going.