3 ms·
It seems like the OP really wants to use a database instead of a file system.
by Derbasti 9y ago
It seems like the OP really wants to use a database instead of a file system.
- davidmr 9y agoI don't think so; HPC is a different beast There are very few if any organizations on earth that have a database that can write a sustained 1TByte/second+ from as few as 300-400 node to 100s of PB of stateful media for years on end. Ken Batcher (maybe of OSU?) wrote a quote that the rest of us have been using for years: "Supercomputers are a tool for converting a CPU-bound problem into a HPC-bound problem." The filesystems start to look more like databases over time, but it's not like they can throw down a nice Cassandra cluster and have it pick up the slack. I'm not saying it will never happen, but I don't think it's am option at the moment.
- davidmr 9y agoRather "into an I/O-bound problem". Apologies for ruining Ken's great joke.
- emmelaich 9y agoExactly. Layer a system on top of the usual I/O and you get the ratio of performance / statefulness you want.
- davidmr 9y agoThey do, more or less. It's called MPI-IO, but it's still tough.