4 ms·
On one of the bigger experiments we use quite a few things: - AFS as a federated posix file system for user home directories. My impression is that a distribut
by dguest 7y ago
On one of the bigger experiments we use quite a few things:
- AFS as a federated posix file system for user home directories. My impression is that a distributed posix filesystem is... well... hard, for basically the reasons listed in the link. We're actually trying to phase it out, starting by reducing the size of the federation by cutting off access outside the CERN network.
- A few in-house developments like xrootd [1] (basically CERN's version of an object store) and EOS (a posix file system built on top), to store data. These projects have their roots in a time when CERN was at the forefront of "big data" and it made sense to develop an in-house project. These days there are a number of alternatives and my impression is that the reasons for continuing the projects are mostly historical.
- For read-only data we have cvmfs [2], a FUSE module which is synced to some other file system a few times a day. Making it read-only simplifies the metadata handling considerably: it's actually quite nice for a CERN project.
- Some people have started using Ceph for more experimental things, but in general these "industry" projects are only starting to replace the home-grown ones.
[1]: https://xrootd.slac.stanford.edu/index.html https://xrootd.slac.stanford.edu/index.html
[2]: https://cernvm.cern.ch/portal/filesystem https://cernvm.cern.ch/portal/filesystem