4 ms·

Was it Ceph they were using at CERN (ATLAS Project, at least?) they were using some kind of file system federation.

by kzrdude 7y ago

tyingq 7y ago
Googling isn't much help. You can find references to AFS, DFS, VM-FS, and EOS...all being used at CERN.
- kzrdude 7y ago
  Oh I see, I was thinking of AFS, I'm sure, I thought it was built on ceph or vice versa.
shdb 7y ago
they do use it. https://www.youtube.com/watch?v=OopRMUYiY5E https://www.youtube.com/watch?v=OopRMUYiY5E https://www.youtube.com/watch?v=0i7ew3XXb7Q https://www.youtube.com/watch?v=0i7ew3XXb7Q
dguest 7y ago
On one of the bigger experiments we use quite a few things: - AFS as a federated posix file system for user home directories. My impression is that a distributed posix filesystem is... well... hard, for basically the reasons listed in the link. We're actually trying to phase it out, starting by reducing the size of the federation by cutting off access outside the CERN network. - A few in-house developments like xrootd [1] (basically CERN's version of an object store) and EOS (a posix file system built on top), to store data. These projects have their roots in a time when CERN was at the forefront of "big data" and it made sense to develop an in-house project. These days there are a number of alternatives and my impression is that the reasons for continuing the projects are mostly historical. - For read-only data we have cvmfs [2], a FUSE module which is synced to some other file system a few times a day. Making it read-only simplifies the metadata handling considerably: it's actually quite nice for a CERN project. - Some people have started using Ceph for more experimental things, but in general these "industry" projects are only starting to replace the home-grown ones. [1]: https://xrootd.slac.stanford.edu/index.html https://xrootd.slac.stanford.edu/index.html [2]: https://cernvm.cern.ch/portal/filesystem https://cernvm.cern.ch/portal/filesystem

Was it Ceph they were using at CERN (ATLAS Project, at least?) they were using some kind of file system federation.

tyingq 7y ago

Googling isn't much help. You can find references to AFS, DFS, VM-FS, and EOS...all being used at CERN.

kzrdude 7y ago
Oh I see, I was thinking of AFS, I'm sure, I thought it was built on ceph or vice versa.

shdb 7y ago

they do use it. https://www.youtube.com/watch?v=OopRMUYiY5E https://www.youtube.com/watch?v=OopRMUYiY5E https://www.youtube.com/watch?v=0i7ew3XXb7Q https://www.youtube.com/watch?v=0i7ew3XXb7Q

dguest 7y ago

On one of the bigger experiments we use quite a few things: - AFS as a federated posix file system for user home directories. My impression is that a distributed posix filesystem is... well... hard, for basically the reasons listed in the link. We're actually trying to phase it out, starting by reducing the size of the federation by cutting off access outside the CERN network. - A few in-house developments like xrootd [1] (basically CERN's version of an object store) and EOS (a posix file system built on top), to store data. These projects have their roots in a time when CERN was at the forefront of "big data" and it made sense to develop an in-house project. These days there are a number of alternatives and my impression is that the reasons for continuing the projects are mostly historical. - For read-only data we have cvmfs [2], a FUSE module which is synced to some other file system a few times a day. Making it read-only simplifies the metadata handling considerably: it's actually quite nice for a CERN project. - Some people have started using Ceph for more experimental things, but in general these "industry" projects are only starting to replace the home-grown ones. [1]: https://xrootd.slac.stanford.edu/index.html https://xrootd.slac.stanford.edu/index.html [2]: https://cernvm.cern.ch/portal/filesystem https://cernvm.cern.ch/portal/filesystem