6 ms·

The Approaching Death of the Parallel File System

22 points by Katydid 11y ago

codemac 11y ago
... submarine for EMC + LANL. Missing all kinds of things about the storage market, and definitely "enterprise-y" focus with a significant number of markets missing, from SDS to Cloud to AFA.. please do not read this article if you want an overview of what's wrong with POSIX and where the industry is moving.
- notacoward 11y ago
  Here's more info on MarFS, which they mention: https://github.com/mar-file-system/marfs https://github.com/mar-file-system/marfs Basically a federated model, using GPFS for metadata plus Scality/ViPR for data. Seems kind of hacky to me, but then a lot of things do when they start out. P.S. Also, it's really nice to see my tax dollars paying for an EMC employee's office space.
  - noahdesu 11y ago
    It may be the case that your tax dollars aren't paying for that office since it is actually in NMC, across the street from LANL (http://newmexicoconsortium.org/ http://newmexicoconsortium.org/). But I don't really know.
- deleted 11y ago
  [deleted]
hackuser 11y ago
I'm interested, but don't have enough understanding of very large systems to follow this article (without spending a lot of time on it). I'd love to learn: Could someone provide a little background on the current state of the art in very large system storage? Also, what about the planned future systems would be incompatible with POSIX?
__david__ 11y ago
I feel like people have been predicting this for years. Yet POSIX is still here… I suspect it will be for a long time.
- cbsmith 11y ago
  POSIX will always be there. Whether you are using it for your typical distributed filesystem use case...
bsder 11y ago
I stand amazed that we still haven't gotten back to what AFS had back in 1992 or what VAX had back in 1986.
- pjc50 11y ago
  .. which was what, exactly? Could you elaborate please?
  - notacoward 11y ago
    I'm not the GP, but I think I can answer for AFS. To this day, nothing has come along that handles the global-scale filesystem use case as well as AFS did way back when. What we have instead are these: * Parallel filesystems that fall apart at more than a few ms latency. * Geo-distributed filesystems that are too slow/insecure/unstable to be useful for anything that really matters. * Ways to sync separate remote filesystems, but with no pretense of consistency. * Things that don't satisfy even basic filesystem semantics (e.g. object stores). * Things that are proprietary, and thus useless/irrelevant outside of their originating organizations (e.g. Google). That's why there are still some very large installations of AFS or its lineal descendants; I know of one that's 50K+ machines. I once had a dream of creating such a successor, but have largely abandoned it. It's not that the technical challenges aren't both interesting and solvable. It's more that the financial, logistical, and personnel aspects of getting such a difficult project completed and used by anyone would drown out all of the fun. I know a few others who have reached the same conclusion, sometimes up front and sometimes after losing years to the effort. The only way such a thing will ever get done is if some deep-pockets org needs it enough to fund and support its development, while also being fully committed to open source. Unfortunately, every org with such capability seems satisfied with one of the alternatives mentioned above.
    - cbsmith 11y ago
      > It's more that the financial, logistical, and personnel aspects of getting such a difficult project completed and used by anyone would drown out all of the fun. Nailed it. I would argue that MapR is basically a company that did exactly this and figured out a way to monetize it by packaging it as part of a map/reduce system.

22 points by Katydid 11y ago

... submarine for EMC + LANL. Missing all kinds of things about the storage market, and definitely "enterprise-y" focus with a significant number of markets missing, from SDS to Cloud to AFA.. please do not read this article if you want an overview of what's wrong with POSIX and where the industry is moving.

notacoward 11y ago
Here's more info on MarFS, which they mention: https://github.com/mar-file-system/marfs https://github.com/mar-file-system/marfs Basically a federated model, using GPFS for metadata plus Scality/ViPR for data. Seems kind of hacky to me, but then a lot of things do when they start out. P.S. Also, it's really nice to see my tax dollars paying for an EMC employee's office space.
- noahdesu 11y ago
  It may be the case that your tax dollars aren't paying for that office since it is actually in NMC, across the street from LANL (http://newmexicoconsortium.org/ http://newmexicoconsortium.org/). But I don't really know.
deleted 11y ago
[deleted]

hackuser 11y ago

I'm interested, but don't have enough understanding of very large systems to follow this article (without spending a lot of time on it). I'd love to learn: Could someone provide a little background on the current state of the art in very large system storage? Also, what about the planned future systems would be incompatible with POSIX?

__david__ 11y ago

I feel like people have been predicting this for years. Yet POSIX is still here… I suspect it will be for a long time.

cbsmith 11y ago
POSIX will always be there. Whether you are using it for your typical distributed filesystem use case...

bsder 11y ago

I stand amazed that we still haven't gotten back to what AFS had back in 1992 or what VAX had back in 1986.

pjc50 11y ago
.. which was what, exactly? Could you elaborate please?
- notacoward 11y ago
  I'm not the GP, but I think I can answer for AFS. To this day, nothing has come along that handles the global-scale filesystem use case as well as AFS did way back when. What we have instead are these: * Parallel filesystems that fall apart at more than a few ms latency. * Geo-distributed filesystems that are too slow/insecure/unstable to be useful for anything that really matters. * Ways to sync separate remote filesystems, but with no pretense of consistency. * Things that don't satisfy even basic filesystem semantics (e.g. object stores). * Things that are proprietary, and thus useless/irrelevant outside of their originating organizations (e.g. Google). That's why there are still some very large installations of AFS or its lineal descendants; I know of one that's 50K+ machines. I once had a dream of creating such a successor, but have largely abandoned it. It's not that the technical challenges aren't both interesting and solvable. It's more that the financial, logistical, and personnel aspects of getting such a difficult project completed and used by anyone would drown out all of the fun. I know a few others who have reached the same conclusion, sometimes up front and sometimes after losing years to the effort. The only way such a thing will ever get done is if some deep-pockets org needs it enough to fund and support its development, while also being fully committed to open source. Unfortunately, every org with such capability seems satisfied with one of the alternatives mentioned above.
  - cbsmith 11y ago
    > It's more that the financial, logistical, and personnel aspects of getting such a difficult project completed and used by anyone would drown out all of the fun. Nailed it. I would argue that MapR is basically a company that did exactly this and figured out a way to monetize it by packaging it as part of a map/reduce system.