5 ms·

Castor: CERN Advanced STORage Manager

64 points by naves 4mo ago

john_strinlai 4mo ago
looks like the image on the right is broken, but it is supposed to be: https://cta.web.cern.ch/cta/assets/images/namespace_statistics.png https://cta.web.cern.ch/cta/assets/images/namespace_statisti... (looks like this submission uses https://castor.web.cern.ch/content/home.html https://castor.web.cern.ch/content/home.html instead of https://castor.web.cern.ch/castor/ https://castor.web.cern.ch/castor/ the second link does not have the broken image)
boznz 4mo ago
The various CERN web pages such as this were a treasure trove of information when I was working on my last novel. I actually included a few paragraphs on Castor thinking of using it as a side-plot, but my editor cut the plot out along with a few other technical niceties. Sigh!
perlgeek 4mo ago
"Castor" was the name of a storage system used for transporting nuclear waste in Germany. There were quite a few protests against shipping nuclear waste through the country. Wouldn't have been my choice for a software project :-)
- tempay 4mo ago
  It’s also French for Beaver which is more likely the origin of the name.
  - rzzzt 4mo ago
    It's also Latin and Greek for beaver which is more likely the origin of the name.
    - tempay 4mo ago
      Latin and Greek aren’t one of the working languages at CERN (French and English are)
      - randiantech 4mo ago
        also spanish
        elashri 4mo ago
        I would say "Italian" :)
        pjmlp 4mo ago
        And Portuguese. :) And to keep this thread, I think our three languages should count as one, because at least 20 years ago, it was quite common to have Portuguese, Italian and Spanish mingle in several activities. Source, ATLAS TDAQ/HLT Alumni.
        elashri 4mo ago
        I was just commenting based on the cliche that Italians are everywhere at CERN. So you will always hear Italian language.
        pjmlp 4mo ago
        They are, that is how learnt Italian without much effort. :)
  - antonvs 4mo ago
    Castor oil makes you poop, maybe there’s a data management metaphor in there somewhere.
    - linksnapzz 4mo ago
      I imagine that an ancient roman would think "Oleum Castorum? It's either oil you get from rendering beavers, or oil you use to lube your beaver..."
Melatonic 4mo ago
This is actually super useful for real world stuff. Thanks for this. Tape is boring but when an intern / AI / tectonic plate accidently destroys your database setup it is a huge lifesaver Anybody know what these fancy Oracle tapes are? Is it just their implementation of a regular standard?
- tempay 4mo ago
  See this conference talk from last week: https://indico.cern.ch/event/1471803/contributions/6967379/ https://indico.cern.ch/event/1471803/contributions/6967379/ There isn’t a recording but slides at linked from that page.
- linksnapzz 4mo ago
  If it's Oracle Tape, it's proprietary T10000-series 1/2in linear tape and associated drives, that they got when they absorbed Sun (and Sun got when they bought StorageTek). Multiple vendors made tape media for these, but they were not compatible w/ LTO tape nor the IBM 3590-series enterprise tape format.
tempay 4mo ago
I’m a little confused by this submission. CASTOR is the old system that has since been replaced by the CERN Tape Array since ~2020: https://cta.web.cern.ch/cta/ https://cta.web.cern.ch/cta/ This is mentioned on the page but it’s easy to miss. For the current status of tape storage at CERN see: https://indico.cern.ch/event/1471803/contributions/6967379/attachments/3281280/5863828/2026-05-26_CHEP_Archiving60PBperMonthToTape_jleduc.pdf https://indico.cern.ch/event/1471803/contributions/6967379/a... For reference, most disk storage for physics data uses an in-house solution called EOS: https://eos-web.web.cern.ch/eos-web/ https://eos-web.web.cern.ch/eos-web/
- bfkwlfkjf 4mo ago
  Does tape array replace castor? Just from the names it sounds like tape array is the actual storage, and castor is an abstraction that automatically decides what's kept on disk and what's kept on tape
  - tempay 4mo ago
    The abstraction isn’t really a thing any more. It was a nice idea but in practice it’s an operational nightmare not knowing if data is available and for how long it will be. For reference staging can take days during intense activity and you don’t want to loose performance randomly seeking around and switching between tapes.
  - sam_bristow 4mo ago
    The linked page seems to think it does. "As of June 29th 2020, CTA, the CERN Tape Archive, started to be operated as the successor of CASTOR and gradually replaced it."
bitbytebane 4mo ago
[dead]
mrlonglong 4mo ago
They now have over an exabyte worth of data on tapes.
Lapsa 4mo ago
[dead]
dokyun 4mo ago
Wonder how this compares to Venti[1]. It looks a lot more complicated (not really a good thing). [1]: https://doc.cat-v.org/plan_9/4th_edition/papers/venti/ https://doc.cat-v.org/plan_9/4th_edition/papers/venti/
- linksnapzz 4mo ago
  You could use tape as a backing for Venti arenas; don't know if anyone ever did so. The original Bell Labs fileserver used an MO jukebox for WORM archives, which today LTFS tape is a pretty close approximation of.
pezezin 4mo ago
Fun fact: CERN sells old data tapes as souvenirs, I got myself one of the old LHC tapes :)
- zerr 4mo ago
  With the data included (not wiped)?
  - tempay 4mo ago
    As far as I know nobody has ever tried to read one ;)
    - tempay 4mo ago
      Though 5PB of data is freely available via https://opendata.cern.ch/ https://opendata.cern.ch/ With another 4PB available which can be processed on request to extract samples of interest: https://opendata.cern.ch/docs/lhcb-releases-service-to-access-run2-data https://opendata.cern.ch/docs/lhcb-releases-service-to-acces...
  - pezezin 4mo ago
    According to the label, the data is included. But it is just experimental data from the LHC, nothing that has to stay secret.
adev_ 4mo ago
A few historical additions for anybody interest: - CASTOR at CERN had also its disk centric derivative named DPM (Disk pool manager) that helped to power the LHC computing grid for multiple decades (WLCG) before getting deprecated. - Interestingly DPM had an architecture quite aligned with the original Google File system even if developed completely separately: (One metadata node, multiple disk node. Design to do Write-once-read-many with very partial POSIX semantics). - The LHC computing Grid is an association of research centers with their own infrastructure. As such, they had (historically) many diffent storage systems with diffent protocols and interface. - To unify this madness, an attempt to do a "standard" protocol was made in the 2000s: the SRM protocol (storage resources manager). In a pure XKCD fashion, it went as bad as you can imagine. It tried to rely on the tech of the time (XML, SOAP, WSDL) and is a school case of terrible protocol design (bloated, slow, weak consistency, massive server overhead, stupidly complex to implement and quite insecure). The spec are worth a read if you want a good laugh [1]. - After 20y of struggle, SRM was eventually dropped for a more pragmatic and ad hoc solution based on HTTP + xrootd [2]. EOS itself uses xrootd quite extensively. (if this did not change) - The history of computing at CERN is globally interesting because it is a pretty good image of the evolution of computing and of the "tech fashions" associated with it. [1]: https://sdm.lbl.gov/srm-wg/doc/SRM.spec.v2.1.1.html https://sdm.lbl.gov/srm-wg/doc/SRM.spec.v2.1.1.html [2]: https://xrootd.org/ https://xrootd.org/
Davidbrcz 4mo ago
I was an intern at CERN in mid 2010s and worked on this !
_pferreir_ 4mo ago
As others have said, CASTOR has been discontinued, and replaced with CTA: https://gitlab.cern.ch/cta/CTA https://gitlab.cern.ch/cta/CTA Its memory is still alive in CTA, however: https://gitlab.cern.ch/cta/CTA/-/blob/main/catalogue/TapeSearchCriteria.hpp#L79 https://gitlab.cern.ch/cta/CTA/-/blob/main/catalogue/TapeSea...

64 points by naves 4mo ago

looks like the image on the right is broken, but it is supposed to be: https://cta.web.cern.ch/cta/assets/images/namespace_statistics.png https://cta.web.cern.ch/cta/assets/images/namespace_statisti... (looks like this submission uses https://castor.web.cern.ch/content/home.html https://castor.web.cern.ch/content/home.html instead of https://castor.web.cern.ch/castor/ https://castor.web.cern.ch/castor/ the second link does not have the broken image)

boznz 4mo ago

The various CERN web pages such as this were a treasure trove of information when I was working on my last novel. I actually included a few paragraphs on Castor thinking of using it as a side-plot, but my editor cut the plot out along with a few other technical niceties. Sigh!

perlgeek 4mo ago

"Castor" was the name of a storage system used for transporting nuclear waste in Germany. There were quite a few protests against shipping nuclear waste through the country. Wouldn't have been my choice for a software project :-)

tempay 4mo ago
It’s also French for Beaver which is more likely the origin of the name.
- rzzzt 4mo ago
  It's also Latin and Greek for beaver which is more likely the origin of the name.
  - tempay 4mo ago
    Latin and Greek aren’t one of the working languages at CERN (French and English are)
    - randiantech 4mo ago
      also spanish
      - elashri 4mo ago
        I would say "Italian" :)
        pjmlp 4mo ago
        And Portuguese. :) And to keep this thread, I think our three languages should count as one, because at least 20 years ago, it was quite common to have Portuguese, Italian and Spanish mingle in several activities. Source, ATLAS TDAQ/HLT Alumni.
        elashri 4mo ago
        I was just commenting based on the cliche that Italians are everywhere at CERN. So you will always hear Italian language.
        pjmlp 4mo ago
        They are, that is how learnt Italian without much effort. :)
- antonvs 4mo ago
  Castor oil makes you poop, maybe there’s a data management metaphor in there somewhere.
  - linksnapzz 4mo ago
    I imagine that an ancient roman would think "Oleum Castorum? It's either oil you get from rendering beavers, or oil you use to lube your beaver..."

Melatonic 4mo ago

This is actually super useful for real world stuff. Thanks for this. Tape is boring but when an intern / AI / tectonic plate accidently destroys your database setup it is a huge lifesaver Anybody know what these fancy Oracle tapes are? Is it just their implementation of a regular standard?

tempay 4mo ago
See this conference talk from last week: https://indico.cern.ch/event/1471803/contributions/6967379/ https://indico.cern.ch/event/1471803/contributions/6967379/ There isn’t a recording but slides at linked from that page.
linksnapzz 4mo ago
If it's Oracle Tape, it's proprietary T10000-series 1/2in linear tape and associated drives, that they got when they absorbed Sun (and Sun got when they bought StorageTek). Multiple vendors made tape media for these, but they were not compatible w/ LTO tape nor the IBM 3590-series enterprise tape format.

tempay 4mo ago

I’m a little confused by this submission. CASTOR is the old system that has since been replaced by the CERN Tape Array since ~2020: https://cta.web.cern.ch/cta/ https://cta.web.cern.ch/cta/ This is mentioned on the page but it’s easy to miss. For the current status of tape storage at CERN see: https://indico.cern.ch/event/1471803/contributions/6967379/attachments/3281280/5863828/2026-05-26_CHEP_Archiving60PBperMonthToTape_jleduc.pdf https://indico.cern.ch/event/1471803/contributions/6967379/a... For reference, most disk storage for physics data uses an in-house solution called EOS: https://eos-web.web.cern.ch/eos-web/ https://eos-web.web.cern.ch/eos-web/

bfkwlfkjf 4mo ago
Does tape array replace castor? Just from the names it sounds like tape array is the actual storage, and castor is an abstraction that automatically decides what's kept on disk and what's kept on tape
- tempay 4mo ago
  The abstraction isn’t really a thing any more. It was a nice idea but in practice it’s an operational nightmare not knowing if data is available and for how long it will be. For reference staging can take days during intense activity and you don’t want to loose performance randomly seeking around and switching between tapes.
- sam_bristow 4mo ago
  The linked page seems to think it does. "As of June 29th 2020, CTA, the CERN Tape Archive, started to be operated as the successor of CASTOR and gradually replaced it."

bitbytebane 4mo ago

[dead]

mrlonglong 4mo ago

They now have over an exabyte worth of data on tapes.

Lapsa 4mo ago

[dead]

dokyun 4mo ago

Wonder how this compares to Venti[1]. It looks a lot more complicated (not really a good thing). [1]: https://doc.cat-v.org/plan_9/4th_edition/papers/venti/ https://doc.cat-v.org/plan_9/4th_edition/papers/venti/

linksnapzz 4mo ago
You could use tape as a backing for Venti arenas; don't know if anyone ever did so. The original Bell Labs fileserver used an MO jukebox for WORM archives, which today LTFS tape is a pretty close approximation of.

pezezin 4mo ago

Fun fact: CERN sells old data tapes as souvenirs, I got myself one of the old LHC tapes :)

zerr 4mo ago
With the data included (not wiped)?
- tempay 4mo ago
  As far as I know nobody has ever tried to read one ;)
  - tempay 4mo ago
    Though 5PB of data is freely available via https://opendata.cern.ch/ https://opendata.cern.ch/ With another 4PB available which can be processed on request to extract samples of interest: https://opendata.cern.ch/docs/lhcb-releases-service-to-access-run2-data https://opendata.cern.ch/docs/lhcb-releases-service-to-acces...
- pezezin 4mo ago
  According to the label, the data is included. But it is just experimental data from the LHC, nothing that has to stay secret.

adev_ 4mo ago

A few historical additions for anybody interest: - CASTOR at CERN had also its disk centric derivative named DPM (Disk pool manager) that helped to power the LHC computing grid for multiple decades (WLCG) before getting deprecated. - Interestingly DPM had an architecture quite aligned with the original Google File system even if developed completely separately: (One metadata node, multiple disk node. Design to do Write-once-read-many with very partial POSIX semantics). - The LHC computing Grid is an association of research centers with their own infrastructure. As such, they had (historically) many diffent storage systems with diffent protocols and interface. - To unify this madness, an attempt to do a "standard" protocol was made in the 2000s: the SRM protocol (storage resources manager). In a pure XKCD fashion, it went as bad as you can imagine. It tried to rely on the tech of the time (XML, SOAP, WSDL) and is a school case of terrible protocol design (bloated, slow, weak consistency, massive server overhead, stupidly complex to implement and quite insecure). The spec are worth a read if you want a good laugh [1]. - After 20y of struggle, SRM was eventually dropped for a more pragmatic and ad hoc solution based on HTTP + xrootd [2]. EOS itself uses xrootd quite extensively. (if this did not change) - The history of computing at CERN is globally interesting because it is a pretty good image of the evolution of computing and of the "tech fashions" associated with it. [1]: https://sdm.lbl.gov/srm-wg/doc/SRM.spec.v2.1.1.html https://sdm.lbl.gov/srm-wg/doc/SRM.spec.v2.1.1.html [2]: https://xrootd.org/ https://xrootd.org/

Davidbrcz 4mo ago

I was an intern at CERN in mid 2010s and worked on this !

_pferreir_ 4mo ago

As others have said, CASTOR has been discontinued, and replaced with CTA: https://gitlab.cern.ch/cta/CTA https://gitlab.cern.ch/cta/CTA Its memory is still alive in CTA, however: https://gitlab.cern.ch/cta/CTA/-/blob/main/catalogue/TapeSearchCriteria.hpp#L79 https://gitlab.cern.ch/cta/CTA/-/blob/main/catalogue/TapeSea...