4 ms·

1 Terabyte of hard disk cost ~50USD. 247 Petabyte ~ 247000 Terabyte > 50000 USD. Network cards, bandwidth, electricity cost > I can't guess. Couple of good e

by ph2082 7y ago

1 Terabyte of hard disk cost ~50USD. 247 Petabyte ~ 247000 Terabyte > 50000 USD. Network cards, bandwidth, electricity cost > I can't guess. Couple of good engineers (hardware and software ones), which they definitely have. May be they could have built their own cloud in < ~10-15 million USD. And that won't be recurring cost. May be they missed article about Bank of America saving ~2 Billion USD, by building their own cloud.

SEJeff 7y ago
You realize that the entire openstack project came from the opensourcing of NASAs opennebula project, right? They've got one of the biggest infiniband networks in the world underpinning it.
- ph2082 7y ago
  I didn't knew that. Thank you for telling. Now I am more curious why go along with AWS instead of using Openstack. Need to find some case study of openstack vs rest of cloud provider.
  - duskwuff 7y ago
    Because OpenStack is a piece of software, not a provider. And it's instructive to consider why none of the major cloud providers use it...
rmrfstar 7y ago
In addition to saving money, they will also make the US more resilient by helping avoid a concentration of expertise and an infrastructure mono-culture. I suspect that ideas like this will become more popular as the US asks itself "what happened to our resilience?" [1] https://en.wikipedia.org/wiki/Self-Reliance https://en.wikipedia.org/wiki/Self-Reliance
supdatecron 7y ago
Your numbers are way off, as you didn't account for redundancy of the drives (any failure or bit flips of 1 of those 2,470 drives will cause corruption of likely the entire data set). > Network cards, bandwidth, electricity cost > I can't guess. This is where a huge amount of cost is. > And that won't be recurring cost. Maintenance, humans, cooling, drive replacements, property, building, land tax, payroll tax are all recurring costs.
- ph2082 7y ago
  > Your numbers are way off, as you didn't account for redundancy of the drives (any failure or bit flips of 1 of those 2,470 drives will cause corruption of likely the entire data set). Let take another setup of same count as backup. Then another setup as back up of back up. ~150K > This is where a huge amount of cost is. Maintenance, humans, cooling, drive replacements cost > can't be greater than first time set up cost. > property, building, land tax, payroll tax Nasa runs on Government budget, I am sure they can claim some tax break there. The point I am trying to make is, it may be cheaper to do in-house with the level of engineering talent they have.
  - sitkack 7y ago
    The government should be running its own object store. And by government, I mean coordinated by Internet2/NSF with federation across all member orgs. https://en.wikipedia.org/wiki/Internet2 https://en.wikipedia.org/wiki/Internet2 Use backblaze pods, demand off peak bandwidth of gilded age megacorps that own said fiber for sync/replication. https://www.backblaze.com/b2/storage-pod.html https://www.backblaze.com/b2/storage-pod.html 480TB/4U Have 3x sites around the US the build the pods, each new pod gets preloaded with a smattering of rarely requested and low replication count objects (as a redundant backup). Then shipped to the site where it will be used. Local writes go directly to pods which are then kept in sync with the rest of the cluster. edit, from the TFA ``` And to put a cherry on top, the report found the project's organizers didn't consult widely enough, didn't follow NIST data integrity standards, and didn't look for savings properly during internal reviews, in part because half of the review team worked on the project itself. ```
- 7y ago

by ph2082 7y ago

SEJeff 7y ago

You realize that the entire openstack project came from the opensourcing of NASAs opennebula project, right? They've got one of the biggest infiniband networks in the world underpinning it.

ph2082 7y ago
I didn't knew that. Thank you for telling. Now I am more curious why go along with AWS instead of using Openstack. Need to find some case study of openstack vs rest of cloud provider.
- duskwuff 7y ago
  Because OpenStack is a piece of software, not a provider. And it's instructive to consider why none of the major cloud providers use it...

rmrfstar 7y ago

In addition to saving money, they will also make the US more resilient by helping avoid a concentration of expertise and an infrastructure mono-culture. I suspect that ideas like this will become more popular as the US asks itself "what happened to our resilience?" [1] https://en.wikipedia.org/wiki/Self-Reliance https://en.wikipedia.org/wiki/Self-Reliance

supdatecron 7y ago

Your numbers are way off, as you didn't account for redundancy of the drives (any failure or bit flips of 1 of those 2,470 drives will cause corruption of likely the entire data set). > Network cards, bandwidth, electricity cost > I can't guess. This is where a huge amount of cost is. > And that won't be recurring cost. Maintenance, humans, cooling, drive replacements, property, building, land tax, payroll tax are all recurring costs.

ph2082 7y ago
> Your numbers are way off, as you didn't account for redundancy of the drives (any failure or bit flips of 1 of those 2,470 drives will cause corruption of likely the entire data set). Let take another setup of same count as backup. Then another setup as back up of back up. ~150K > This is where a huge amount of cost is. Maintenance, humans, cooling, drive replacements cost > can't be greater than first time set up cost. > property, building, land tax, payroll tax Nasa runs on Government budget, I am sure they can claim some tax break there. The point I am trying to make is, it may be cheaper to do in-house with the level of engineering talent they have.
- sitkack 7y ago
  The government should be running its own object store. And by government, I mean coordinated by Internet2/NSF with federation across all member orgs. https://en.wikipedia.org/wiki/Internet2 https://en.wikipedia.org/wiki/Internet2 Use backblaze pods, demand off peak bandwidth of gilded age megacorps that own said fiber for sync/replication. https://www.backblaze.com/b2/storage-pod.html https://www.backblaze.com/b2/storage-pod.html 480TB/4U Have 3x sites around the US the build the pods, each new pod gets preloaded with a smattering of rarely requested and low replication count objects (as a redundant backup). Then shipped to the site where it will be used. Local writes go directly to pods which are then kept in sync with the rest of the cluster. edit, from the TFA ``` And to put a cherry on top, the report found the project's organizers didn't consult widely enough, didn't follow NIST data integrity standards, and didn't look for savings properly during internal reviews, in part because half of the review team worked on the project itself. ```
7y ago