4 ms·

This is fascinating. How are they managing or even taking backup for this gigantic storage?

ephimetheus 2y ago
For experiment data, there is a layer on top of all of this that distributes datasets across the computing grid. That system has a way to handle replicate at the dataset level.
rob_c 2y ago
Tape and off-site replicas at globally distributed data centres for science. Of the 1EB a huge amount of that is probably in automated recall and replication with "users" running staged processing of the data at different sites ultimately with data being reduced to "manageable" GB-TB level for scientists to do science
- fnands 2y ago
  Yup, lots of tape for stuff in cold storage, and then some subset of that on disk spread out over several sites. It's kinda interesting to watch anything by Alberto Pace, the head of storage at CERN to get an understanding of the challenges and constraints: https://www.youtube.com/watch?v=ym2am-FumXQ https://www.youtube.com/watch?v=ym2am-FumXQ I was basically on the helpdesk for the system for a few years so had to spend a fair amount of time helping people replicate data from one place to another, or from tape onto disk.
JBorrow 2y ago
They use a distributed data management tool called RUCIO: https://rucio.cern.ch https://rucio.cern.ch to distribute data on the grid.
tantalor 2y ago
What is Rucio? Rucio enables centralized management of large volumes of data backed by many heterogeneous storage backends. Data is physically distributed over a large number of storage servers, potentially each relying on different storage technologies (SSD/Disk/Tape/Object storage) and, frequently, managed by different teams of system administrators. Rucio builds on top of this heterogeneous infrastructure and provides an interface which allows users to interact with the storage backends in a unified way. The smallest operational unit in Rucio is a file. Rucio enables users to upload, download, and declaratively manage groups of such files. https://rucio.cern.ch/documentation/started/what_is_rucio https://rucio.cern.ch/documentation/started/what_is_rucio
lnauta 2y ago
At some tier 1 sites in the grid there is no backup for the tapes as its costly and you only lose some statistics when a tape is lost.

For experiment data, there is a layer on top of all of this that distributes datasets across the computing grid. That system has a way to handle replicate at the dataset level.

rob_c 2y ago

Tape and off-site replicas at globally distributed data centres for science. Of the 1EB a huge amount of that is probably in automated recall and replication with "users" running staged processing of the data at different sites ultimately with data being reduced to "manageable" GB-TB level for scientists to do science

fnands 2y ago
Yup, lots of tape for stuff in cold storage, and then some subset of that on disk spread out over several sites. It's kinda interesting to watch anything by Alberto Pace, the head of storage at CERN to get an understanding of the challenges and constraints: https://www.youtube.com/watch?v=ym2am-FumXQ https://www.youtube.com/watch?v=ym2am-FumXQ I was basically on the helpdesk for the system for a few years so had to spend a fair amount of time helping people replicate data from one place to another, or from tape onto disk.

JBorrow 2y ago

They use a distributed data management tool called RUCIO: https://rucio.cern.ch https://rucio.cern.ch to distribute data on the grid.

tantalor 2y ago

What is Rucio? Rucio enables centralized management of large volumes of data backed by many heterogeneous storage backends. Data is physically distributed over a large number of storage servers, potentially each relying on different storage technologies (SSD/Disk/Tape/Object storage) and, frequently, managed by different teams of system administrators. Rucio builds on top of this heterogeneous infrastructure and provides an interface which allows users to interact with the storage backends in a unified way. The smallest operational unit in Rucio is a file. Rucio enables users to upload, download, and declaratively manage groups of such files. https://rucio.cern.ch/documentation/started/what_is_rucio https://rucio.cern.ch/documentation/started/what_is_rucio

lnauta 2y ago

At some tier 1 sites in the grid there is no backup for the tapes as its costly and you only lose some statistics when a tape is lost.