4 ms·
If you're actually hyperscale, say 100 EB of tape, then you'll have dedicated teams just for tape archives. The heterogeneity doesn't seem like much of a proble
by pradn 5y ago
If you're actually hyperscale, say 100 EB of tape, then you'll have dedicated teams just for tape archives. The heterogeneity doesn't seem like much of a problem when you have 200 engineers x 300k USD per engineer to spend on managing the system.
If anything, we're seeing hyperscalers becoming more heterogeneous with CPUs. Google has training and inference TPUs, custom silicon for encoding media, and many more custom CPUs. It makes sense for storage if the benefit is there.
- notacoward 5y agoThose are completely different levels of heterogeneity. A GPU/TPU might draw a lot of power, but does it have to be separate power? Do you need to put that stuff on a separate slab to isolate it from vibration? Does it need its own air filtration system? Do you need a new runbook, because your usual redundancy approaches can't be used for million-dollar systems? Do your DC techs need new training, because the usual "yank and replace" isn't applicable either? I've watched colleagues in a team adjacent to my own work through these issues. Have you? Slapping a commodity card into a box and loading some commodity software on it seems like a cakewalk by comparison. Obviously people thought it was worth it anyway, but I think you're seriously misunderstanding where the difficulties lie and what kind of resources are needed to overcome them. 200+ engineers would be overkill (unless FB engineers are ~4x more productive than Google engineers) but you'll need other kinds of specialists as well and the installation/operational costs will still be high. It's unlikely to be worth it for an organization much smaller or different.
- pradn 5y agoYes, these factors would probably limit massive tape archives to only the biggest companies.