8 ms·

Note that a few dozen terabytes is also nothing. 12 TiB is about $800 on an SSD and about $250 on a HDD. Plus some overhead for the enclosure and redundancy, of

Note that a few dozen terabytes is also nothing. 12 TiB is about $800 on an SSD and about $250 on a HDD. Plus some overhead for the enclosure and redundancy, of course. It costs on the order of a day's pay for an engineer, at most. I don't excuse wanton waste of storage because it's easy for sloppy practices to balloon to massive confusion and inefficiency. But that discipline should be enforced by good engineering practices, not by limiting resources.

wil421 3y ago
Are you talking consumer prices, enterprise redundant SSDs in a Data Center, or enterprise cloud storage? In my experience the latter two could add another digit to your price.
- b112 3y ago
  And all of that needs to be backed up, secured, stored, tested for validity regularly, and so on. More disk space isn't one drive, but redundancy too And backups. And even network bandwidth. This thread sort of highlights the problems. DEVs don't get it. Nor does management.
greedo 3y ago
This is such a uninformative comment. Business/Enterprise storage is far, far more expensive than a drive you pick up at Best Buy or Fry's.
- willcipriano 3y ago
  I'd bet the salary of the guy who is making this a pain could buy enough storage that they don't have to worry about it anymore.
  - greedo 3y ago
    Depends on the org. Some orgs are well managed, others don't want to pay for stuff behind the scenes until it bites them in the ass. They don't know what they don't know.
- doublepg23 3y ago
  I’ve been getting organic and cage-free hard drives nowadays.
  - justsomehnguy 3y ago
    > and cage-free hard drives nowadays. SOmewhat recently I toyed with an idea of HDDs with an 'interposer' which would be just a dumb ATA2iSCSI interface, with Ethernet or WiFi connectivity. That would allow you to place those drives literally anywhere and with a minimal footprint. Sadly it would be too costly for the home usage (I assume about $30/unit at best) and for the enterprise usage... 1Gbit is too slow, 10Gbit is too hot and bulky and then you can't sell a $3 plastic case for $100 'vendor approved with light-path(r)(tm) diagnostic indication' for each drive.
- marcosdumay 3y ago
  Well, that's a choice your ops people make. There is absolutely no technical reason to provide all of your storage needs with a single quality of disks. You may do that to get some economies of scale, but if the impact is that high, you should rethink it. You can just as well have two different solutions, one with a huge amount of disk space, little redundancy and low performance, and one with a limited amount of space, plenty of redundancy and high performance. And yes, I know that goes against every common ops procedure, because disks are so cheap. But the next thing you hear is always that disks are not cheap at all, and you really can't have it both ways.
  - datadrivenangel 3y ago
    When you start requiring redundancy, you start needing 2-6x the number of disks, minimum. Still cheap, but not that cheap.
  - acdha 3y ago
    The problem is that staff time costs an order of magnitude more and most of these things can go wrong. For example, you don’t just want to buy random lots of whatever disks are cheapest because then you have to track recalls and firmware updates for everything. If you have two tiers available, someone will use the less reliable one because that’s how they fit their budget but then because it’s “in production” they’ll expect the same level of service. All of these are manageable but what you’re really hearing is that the technical issues are really the tip of the social iceberg most organizations have. One of the reasons people pick AWS isn’t just that it’s usually cheaper than the full cost of rolling your own but that lots of these things don’t affect you: you never fail to provision an EBS volume because the VPs of finance and IT are still arguing about procuring a new rack of disks, people can’t request endless customizations because the options are “take it or leave it”, etc.
  - tremon 3y ago
    Single disks are cheap. Multi-disk storage systems are not. Multi-tiered storage systems are even more expensive. I don't see an inherent conflict there.
- thomastjeffery 3y ago
  So don't waste your money on it then! Why in the world would you buy one "Enterprise" disk instead of three "consumer" disks?
  - justinclift 3y ago
    Well, "enterprise storage" can actually be multiple, redundant copies of the data behind the scenes. So, internally the storage team might quote say $1000/TB (simple numbers for an example) for a given storage quantity. And behind the scenes they'll likely have at least redundant storage arrays, plus backups and 24/7 monitoring for all of the data, etc.
    - thomastjeffery 3y ago
      Yes, but that still refutes the point all the same.
  - greedo 3y ago
    Because if you're providing services to a large business, you're typically not managing servers with physical disks attached. You're using a SAN with fiber connections or NFS mounts. Most SANs require specific drives sold by the vendor with firmware they've tested, mounted to custom sleds etc. You can't just connect a WD Mybook drive. Mom and Pop businesses using single servers can do all they want in regards to drives, I don't care. I would argue they shouldn't have servers, but if you do have servers, you should at minimum use RAID etc, not a drive plugged into your USB port.
    - thomastjeffery 3y ago
      If you are in that situation, then obviously you have to do what you have to do. Absolutely none of that has to apply to the sort of situation we are talking about here. The only real reason to spend more per disk is when you know all your disks are going to fail, and extending the lifespan per average disk will definitely save you more than the enterprise markup costs. So you better have dozens or hundreds of disks in the first place. For any truly valuable data, you should at minimum have a backup in one different physical location. That backup should be include at least one redundant disk. None of that is worth spending a dime on more expensive hardware.
phpisthebest 3y ago
Entry level Enterprise grade HDD storage will run you about $1,000 per TB. All Flash arrays will get even more $$$$ Price out a Nimble or PureStorage array can see if they will sell you 12TB for $800... I will wait.

wil421 3y ago

Are you talking consumer prices, enterprise redundant SSDs in a Data Center, or enterprise cloud storage? In my experience the latter two could add another digit to your price.

b112 3y ago
And all of that needs to be backed up, secured, stored, tested for validity regularly, and so on. More disk space isn't one drive, but redundancy too And backups. And even network bandwidth. This thread sort of highlights the problems. DEVs don't get it. Nor does management.

greedo 3y ago

This is such a uninformative comment. Business/Enterprise storage is far, far more expensive than a drive you pick up at Best Buy or Fry's.

willcipriano 3y ago
I'd bet the salary of the guy who is making this a pain could buy enough storage that they don't have to worry about it anymore.
- greedo 3y ago
  Depends on the org. Some orgs are well managed, others don't want to pay for stuff behind the scenes until it bites them in the ass. They don't know what they don't know.
doublepg23 3y ago
I’ve been getting organic and cage-free hard drives nowadays.
- justsomehnguy 3y ago
  > and cage-free hard drives nowadays. SOmewhat recently I toyed with an idea of HDDs with an 'interposer' which would be just a dumb ATA2iSCSI interface, with Ethernet or WiFi connectivity. That would allow you to place those drives literally anywhere and with a minimal footprint. Sadly it would be too costly for the home usage (I assume about $30/unit at best) and for the enterprise usage... 1Gbit is too slow, 10Gbit is too hot and bulky and then you can't sell a $3 plastic case for $100 'vendor approved with light-path(r)(tm) diagnostic indication' for each drive.
marcosdumay 3y ago
Well, that's a choice your ops people make. There is absolutely no technical reason to provide all of your storage needs with a single quality of disks. You may do that to get some economies of scale, but if the impact is that high, you should rethink it. You can just as well have two different solutions, one with a huge amount of disk space, little redundancy and low performance, and one with a limited amount of space, plenty of redundancy and high performance. And yes, I know that goes against every common ops procedure, because disks are so cheap. But the next thing you hear is always that disks are not cheap at all, and you really can't have it both ways.
- datadrivenangel 3y ago
  When you start requiring redundancy, you start needing 2-6x the number of disks, minimum. Still cheap, but not that cheap.
- acdha 3y ago
  The problem is that staff time costs an order of magnitude more and most of these things can go wrong. For example, you don’t just want to buy random lots of whatever disks are cheapest because then you have to track recalls and firmware updates for everything. If you have two tiers available, someone will use the less reliable one because that’s how they fit their budget but then because it’s “in production” they’ll expect the same level of service. All of these are manageable but what you’re really hearing is that the technical issues are really the tip of the social iceberg most organizations have. One of the reasons people pick AWS isn’t just that it’s usually cheaper than the full cost of rolling your own but that lots of these things don’t affect you: you never fail to provision an EBS volume because the VPs of finance and IT are still arguing about procuring a new rack of disks, people can’t request endless customizations because the options are “take it or leave it”, etc.
- tremon 3y ago
  Single disks are cheap. Multi-disk storage systems are not. Multi-tiered storage systems are even more expensive. I don't see an inherent conflict there.
thomastjeffery 3y ago
So don't waste your money on it then! Why in the world would you buy one "Enterprise" disk instead of three "consumer" disks?
- justinclift 3y ago
  Well, "enterprise storage" can actually be multiple, redundant copies of the data behind the scenes. So, internally the storage team might quote say $1000/TB (simple numbers for an example) for a given storage quantity. And behind the scenes they'll likely have at least redundant storage arrays, plus backups and 24/7 monitoring for all of the data, etc.
  - thomastjeffery 3y ago
    Yes, but that still refutes the point all the same.
- greedo 3y ago
  Because if you're providing services to a large business, you're typically not managing servers with physical disks attached. You're using a SAN with fiber connections or NFS mounts. Most SANs require specific drives sold by the vendor with firmware they've tested, mounted to custom sleds etc. You can't just connect a WD Mybook drive. Mom and Pop businesses using single servers can do all they want in regards to drives, I don't care. I would argue they shouldn't have servers, but if you do have servers, you should at minimum use RAID etc, not a drive plugged into your USB port.
  - thomastjeffery 3y ago
    If you are in that situation, then obviously you have to do what you have to do. Absolutely none of that has to apply to the sort of situation we are talking about here. The only real reason to spend more per disk is when you know all your disks are going to fail, and extending the lifespan per average disk will definitely save you more than the enterprise markup costs. So you better have dozens or hundreds of disks in the first place. For any truly valuable data, you should at minimum have a backup in one different physical location. That backup should be include at least one redundant disk. None of that is worth spending a dime on more expensive hardware.

phpisthebest 3y ago

Entry level Enterprise grade HDD storage will run you about $1,000 per TB. All Flash arrays will get even more $$$$ Price out a Nimble or PureStorage array can see if they will sell you 12TB for $800... I will wait.