6 ms·

You asked, "What market is there for big hard drives outside of FAANG [etc.]?" I answered. The home prosumer market. And if anyone ever bothered to develop a

You asked, "What market is there for big hard drives outside of FAANG [etc.]?" I answered. The home prosumer market. And if anyone ever bothered to develop a multi-terabyte solution for "normal people", whereby they could easily have their DVD / Blu-ray / Compact Disc collection converted from disc to a media server, probably a lot more people. 100 TB hard drives will end up in consumer PCs by 2035, I'm certain of it, even if we won't really need it. 8K is just plain stupid for home use, I honestly don't know why its being pushed, since you'd need a 120" screen or more, but I'm sure there'll be new media types beyond 4K... 4K60 FPS for instance, or high-resolution VR movies. I could easily see a world where 4K60 FPS movies take up 200 GB space each, and high-res VR movies are that large or even larger.

notacoward 6y ago
> You asked, "What market is there for big hard drives outside of FAANG [etc.]?" No, I didn't. PaulHoule did. > 100 TB hard drives will end up in consumer PCs by 2035, I'm certain of it, even if we won't really need it. I think you're confusing the need for more capacity with the need for more capacity within a single drive. This is the same distinction you'll find in the original RAID papers. With a big single drive ("SLED" in the papers) your performance per megabyte stored and your reliability per megabyte stored keep going down. At some point this will make your system unusable. I contend that we're already on the edge and these newer technologies that only increase capacity will push a lot of people over. Increasing video resolutions only make the problem worse, not better. How many 4K60 streams are you going to get over a single interface? Now consider an alternative: same amount of storage, across multiple drives. In the same physical space, because that higher areal density can be used to make drives physically smaller just as well as it helps make them logically bigger. In a similar power/heat envelope, because smaller also means lighter. But with better performance across more heads and more external interfaces. And with better reliability because with multiple drives you can add some redundancy (and even if you don't at least a single failure will only cost you some of your data). Why would you want a single big disk instead? Complexity? It's not actually that big a deal. Non-specialists build such systems every day. The only thing the drive vendors need to do is use those improvements in areal density to make drives physically smaller instead of logically larger. Stop making the capacity/bandwidth gap bigger. Let people build balanced systems that actually work, instead of on systems with a striking resemblance to those used for "virtual tape" cold storage in the bad old days.
- Dylan16807 6y ago
  > Increasing video resolutions only make the problem worse, not better. How many 4K60 streams are you going to get over a single interface? How many does a prosumer need? One, maybe two. Maybe 5 in a NAS. A single drive today can already handle that in 8K. When it comes to media files, we hit storage limits all the time but we're nowhere near the bandwidth limits of a hard drive. We're not on the edge when it comes to performance per megabyte. Program files and game files are miles away on one side, and videos and photos are miles away on the other side. There's some point where increasing the density of hard drive platters is too slow for a prosumer media library, but I'm confident it's far out there, past a petabyte.
  - notacoward 6y ago
    So what percentage of the market would you say that is? 1% would be generous. Should that drive the technology?
    - Dylan16807 6y ago
      I would say that the vast majority of the consumer hard drive market is in the bracket where size matters much more than performance. I have no idea what fraction of the server market. But there's also an important thing to note about product families. You talk about using density improvements to make drives smaller, and then install more of them to keep performance up. I think that's reasonable, but I also think that one of the best ways to do that is to reduce the platter count and drive height. In that world, where the main product is thin drives, it takes only a small amount of engineering effort to keep making an XL model that has lower performance but is significantly cheaper per TB.
      - notacoward 6y ago
        I said I'm not an expert on the HDD market, but I have seen some stuff so here's a bit of perspective. The hyperscalers are more than 50% of the market. They're each big enough that price distortion from their own buys is a real concern that they plan for. (Note BTW that these are often left out of analysts' charts, because leaving them in makes it harder to see patterns among the rest.) More than half of what's left is sold to businesses, with most of that going to companies big enough to build and run their own data centers (including supercomputer facilities). And then, finally, all consumer drive sales account for something less than 20% of the total. Yes, size does matter a lot for those markets. If you want performance use flash. However, again, size per drive is a red herring. What matters is the capacity you can fit into a system, whether it's a laptop or a server. Having that much capacity present as a single volume through a single interface is simply not ideal either for performance (which might not be the same goal but still has a lower limit) or for reliability. That's all I've been saying. You're better off combining multiple lower-capacity drives, even if you can get by (at least for a while) with a single larger drive. Serious video folks and even gamers have known the advantages of dual drive RAID-0 or RAID-1 for years. > where the main product is thin drives, it takes only a small amount of engineering effort What you're now suggesting is no more than what I suggested nearly a day and several posts ago (look for "drive manufacturers could help"), which you and "others" took issue with. Yes, drive manufacturers can and should make those thin drives, and then sell multiples packed into a single enclosure like we already have today. The fact that it's multiple physical drives could be more or less transparent. The transparent version would be cheaper and offer the system designer more flexibility. The non-transparent version, akin to existing HW RAID or even multiple platters today, would be a bit easier to conceptualize for people not used to thinking of enclosures and spindles and platters and heads as separate things, but it would be a bit more expensive (controller plus memory as part of the package) and not necessarily better. In short, using "drive" to mean both the package with connectors on the side and the piece(s) of oxide-coated metal inside it is sloppy, and leads to wrong conclusions. Once you realize that higher density creates more options than "every limit the same except for higher capacity" then it quickly becomes clear that 120TB on a single spindle isn't the best use of that technology.

notacoward 6y ago

> You asked, "What market is there for big hard drives outside of FAANG [etc.]?" No, I didn't. PaulHoule did. > 100 TB hard drives will end up in consumer PCs by 2035, I'm certain of it, even if we won't really need it. I think you're confusing the need for more capacity with the need for more capacity within a single drive. This is the same distinction you'll find in the original RAID papers. With a big single drive ("SLED" in the papers) your performance per megabyte stored and your reliability per megabyte stored keep going down. At some point this will make your system unusable. I contend that we're already on the edge and these newer technologies that only increase capacity will push a lot of people over. Increasing video resolutions only make the problem worse, not better. How many 4K60 streams are you going to get over a single interface? Now consider an alternative: same amount of storage, across multiple drives. In the same physical space, because that higher areal density can be used to make drives physically smaller just as well as it helps make them logically bigger. In a similar power/heat envelope, because smaller also means lighter. But with better performance across more heads and more external interfaces. And with better reliability because with multiple drives you can add some redundancy (and even if you don't at least a single failure will only cost you some of your data). Why would you want a single big disk instead? Complexity? It's not actually that big a deal. Non-specialists build such systems every day. The only thing the drive vendors need to do is use those improvements in areal density to make drives physically smaller instead of logically larger. Stop making the capacity/bandwidth gap bigger. Let people build balanced systems that actually work, instead of on systems with a striking resemblance to those used for "virtual tape" cold storage in the bad old days.

Dylan16807 6y ago
> Increasing video resolutions only make the problem worse, not better. How many 4K60 streams are you going to get over a single interface? How many does a prosumer need? One, maybe two. Maybe 5 in a NAS. A single drive today can already handle that in 8K. When it comes to media files, we hit storage limits all the time but we're nowhere near the bandwidth limits of a hard drive. We're not on the edge when it comes to performance per megabyte. Program files and game files are miles away on one side, and videos and photos are miles away on the other side. There's some point where increasing the density of hard drive platters is too slow for a prosumer media library, but I'm confident it's far out there, past a petabyte.
- notacoward 6y ago
  So what percentage of the market would you say that is? 1% would be generous. Should that drive the technology?
  - Dylan16807 6y ago
    I would say that the vast majority of the consumer hard drive market is in the bracket where size matters much more than performance. I have no idea what fraction of the server market. But there's also an important thing to note about product families. You talk about using density improvements to make drives smaller, and then install more of them to keep performance up. I think that's reasonable, but I also think that one of the best ways to do that is to reduce the platter count and drive height. In that world, where the main product is thin drives, it takes only a small amount of engineering effort to keep making an XL model that has lower performance but is significantly cheaper per TB.
    - notacoward 6y ago
      I said I'm not an expert on the HDD market, but I have seen some stuff so here's a bit of perspective. The hyperscalers are more than 50% of the market. They're each big enough that price distortion from their own buys is a real concern that they plan for. (Note BTW that these are often left out of analysts' charts, because leaving them in makes it harder to see patterns among the rest.) More than half of what's left is sold to businesses, with most of that going to companies big enough to build and run their own data centers (including supercomputer facilities). And then, finally, all consumer drive sales account for something less than 20% of the total. Yes, size does matter a lot for those markets. If you want performance use flash. However, again, size per drive is a red herring. What matters is the capacity you can fit into a system, whether it's a laptop or a server. Having that much capacity present as a single volume through a single interface is simply not ideal either for performance (which might not be the same goal but still has a lower limit) or for reliability. That's all I've been saying. You're better off combining multiple lower-capacity drives, even if you can get by (at least for a while) with a single larger drive. Serious video folks and even gamers have known the advantages of dual drive RAID-0 or RAID-1 for years. > where the main product is thin drives, it takes only a small amount of engineering effort What you're now suggesting is no more than what I suggested nearly a day and several posts ago (look for "drive manufacturers could help"), which you and "others" took issue with. Yes, drive manufacturers can and should make those thin drives, and then sell multiples packed into a single enclosure like we already have today. The fact that it's multiple physical drives could be more or less transparent. The transparent version would be cheaper and offer the system designer more flexibility. The non-transparent version, akin to existing HW RAID or even multiple platters today, would be a bit easier to conceptualize for people not used to thinking of enclosures and spindles and platters and heads as separate things, but it would be a bit more expensive (controller plus memory as part of the package) and not necessarily better. In short, using "drive" to mean both the package with connectors on the side and the piece(s) of oxide-coated metal inside it is sloppy, and leads to wrong conclusions. Once you realize that higher density creates more options than "every limit the same except for higher capacity" then it quickly becomes clear that 120TB on a single spindle isn't the best use of that technology.