3 ms·
I only have second hand insight into how they do it, but my impression is that their spiders log the raw data right to disk and then, periodically (a few times
by jlgreco 14y ago
I only have second hand insight into how they do it, but my impression is that their spiders log the raw data right to disk and then, periodically (a few times a year?) they process all the accumulated data in one mind-bogglingly massive job. I don't know what the data goes in to as a result of this processing though.