3 ms·

I'm wondering what kind of computation engine and database they use to process and store the torrents of data. I bet its a no-no-no-SQL

• carterschonwald 17y ago
  There've been a few articles on that topic over time, though I don't have any links handy. Databases don't even enter the basic processing infrastructure, as they're dealing with on the order of terabytes of new data per second! I thing they even had to connect their cluster together with fiberoptics just to be able to handle all the data traffic!
  • deleted 17y ago
    [deleted]
  • scdlbx 17y ago
    Typically a cluster is connected with fiberoptics or similar media anyways. That they are doing the same is not surprising.
• lokijuhyghj 17y ago
  Back in the ATLAS days it was a drinking from the fire hose system. It couldn't handle all the data so it sampled what you hoped was a random sample of it. So if you looked at 1% of the output and found N particles you could assume that there were 100N produced. There was some cool hardware being built back then, transputer arrays and a lot of hopefully experiments with FPGAs. One difficulty was that you needed to get the computers close enough to the detector to keep the interconnects short, but not so close that the chips got fried.