10 ms·

So I work at one of the participating institutes, and you're very much dead-on about transfer speeds/logistics limiting options here. I'm not an astronomer, so

So I work at one of the participating institutes, and you're very much dead-on about transfer speeds/logistics limiting options here. I'm not an astronomer, so grain of salt, but the data is generated at a bit of an unwieldy pace: there are four collector nodes at each site, each of which generates ~16Gbps of raw data (though read speeds from disk after observation is more like 8Gbps). This, as you say, forces most locations to ship physical drives, and also makes centrally planning these observations rather tricky, as there's very little visibility into each station's observations until some time afterwards. But I'm optimistic that some institutions (hopefully including mine!) will be able to transfer these data over the network after the next run of EHT observations. Exciting stuff for sure.

kloch 4y ago
How many seconds of observation data are required for one of these images?
- KAMSPioneer 4y ago
  For the entire image? I'm not sure, but a lot. Usually an EHT observing session is days in length, and the responsible astronomer(s) will reside at the telescope for a week or so...but I don't believe that 100% of that time is spent feeding data into the collectors. I can ask my coworker tomorrow for clarification on the actual observing time and post an update.
whoopdedo 4y ago
Where can I read more about how that volume of data is generated? I'm looking at the description of ALMA right now. So is each "node" a cluster of antennas? And each antenna is collecting a high-resolution snapshot across a wide frequency band. And each snapshot has to be timestamped. What's the sampling rate? It says the frequency range is 31GHz to 950GHz but how wide can a single snapshot be? Then to move it around are you using InfiniBand or something even faster?
- adgjlsfhk1 4y ago
  much faster than infiniband. A suitcase full of hard drives.
  - whoopdedo 4y ago
    That's for moving between sites. I'm talking about the link between the antennas to each observatory's main computer.
- KAMSPioneer 4y ago
  So ALMA, being an array, is a bit of a different beast than single-dish telescopes like the one where I'm employed; they do indeed have an array of antennae and correlate all the collected data at a central correlator (at least, for normal observations). My institute has a single, much larger primary reflector (30 meter diameter) and does not require such a process during normal observation. However, during VLBI observations, which EHT is, the receiver is dumping data to four collector computers, which is what I was referring to as "nodes" generating ~16Gbps of data apiece. I wish I could shed some more light on the ins-and-outs of exactly how these observations work, but I just run the computers, man. :) What I can tell you is that in order to move data off of the collector machines, they typically use m5copy, which is a part of the JIVE project (they have a Github repo: https://github.com/jive-vlbi/jive5ab https://github.com/jive-vlbi/jive5ab). All communication between the control computer and the collectors happens within a private, physically-distinct network, but it's just standard commodity networking equipment between the control computer and the collectors. The folks in charge of the node's design are in the process of removing some of the bottlenecks to make electronic transfers more viable (the current spec doesn't even include a 10Gbps uplink!).

kloch 4y ago

How many seconds of observation data are required for one of these images?

KAMSPioneer 4y ago
For the entire image? I'm not sure, but a lot. Usually an EHT observing session is days in length, and the responsible astronomer(s) will reside at the telescope for a week or so...but I don't believe that 100% of that time is spent feeding data into the collectors. I can ask my coworker tomorrow for clarification on the actual observing time and post an update.

whoopdedo 4y ago

Where can I read more about how that volume of data is generated? I'm looking at the description of ALMA right now. So is each "node" a cluster of antennas? And each antenna is collecting a high-resolution snapshot across a wide frequency band. And each snapshot has to be timestamped. What's the sampling rate? It says the frequency range is 31GHz to 950GHz but how wide can a single snapshot be? Then to move it around are you using InfiniBand or something even faster?

adgjlsfhk1 4y ago
much faster than infiniband. A suitcase full of hard drives.
- whoopdedo 4y ago
  That's for moving between sites. I'm talking about the link between the antennas to each observatory's main computer.
KAMSPioneer 4y ago
So ALMA, being an array, is a bit of a different beast than single-dish telescopes like the one where I'm employed; they do indeed have an array of antennae and correlate all the collected data at a central correlator (at least, for normal observations). My institute has a single, much larger primary reflector (30 meter diameter) and does not require such a process during normal observation. However, during VLBI observations, which EHT is, the receiver is dumping data to four collector computers, which is what I was referring to as "nodes" generating ~16Gbps of data apiece. I wish I could shed some more light on the ins-and-outs of exactly how these observations work, but I just run the computers, man. :) What I can tell you is that in order to move data off of the collector machines, they typically use m5copy, which is a part of the JIVE project (they have a Github repo: https://github.com/jive-vlbi/jive5ab https://github.com/jive-vlbi/jive5ab). All communication between the control computer and the collectors happens within a private, physically-distinct network, but it's just standard commodity networking equipment between the control computer and the collectors. The folks in charge of the node's design are in the process of removing some of the bottlenecks to make electronic transfers more viable (the current spec doesn't even include a 10Gbps uplink!).