10 ms·
12 Billion-year-old signal from the end of the universe's 'dark age'
- hanselot 6y agoThat's pretty awesome. I wonder how this is the orange guy's fault?
- bhickey 6y agoAn old friend of mine is one of the authors on this. Their instruments generate a lot of data. I don't know if this is the case for their array in Australia, but when he was working in South Africa they had to carry hard drives out of the country because there wasn't enough fiber capacity out of the continent.
- lattalayta 6y agoI recently found out that that is referred to as the sneakernet, and is actually still a preferred way to transfer a lot of data. https://en.wikipedia.org/wiki/Sneakernet https://en.wikipedia.org/wiki/Sneakernet
- EarthLaunch 6y agoAWS has a little-known service for trucks transporting 100PB of data in shipping containers: https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/
- amelius 6y ago"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." -- Andrew S. Tanenbaum (circa 1989)
- wolco 6y agoWhat do their prices look like for 100 pb?
- carlosf 6y agoDamn. I once spoke with a friend who helped a museum in Rio de Janeiro to migrate its backups using AWS Snowball, but thats a whole new level. https://aws.amazon.com/snowball/ https://aws.amazon.com/snowball/
- hendiatris 6y agoWhich one? IMS?
- asib 6y agoI always wondered who would actually use this. Apparently a company called DigitalGlobe[0], which has a large collection of satellite images. [0]: https://www.networkworld.com/article/3154898/this-company-is-using-amazon-snowmobile-to-transfer-petabytes-of-data-to-the-cloud.html https://www.networkworld.com/article/3154898/this-company-is...
- mr_toad 6y agoFor when your forklift migration needs a literal forklift.
- hyperpallium 6y agoIn A Fire Upon the Deep, the Skroderiders' cargo was a one-time pad.
- yetihehe 6y agoIf you send someone several TB of random data on hard disk, you can now very securely send them several TB worth of encrypted data. It's many years of audio of reasonable quality, 1tb is about 72 days of raw 44.khz 16 bit stereo audio. With some speech codecs, that could be years of voice data. One time pad can be bigger than a notepad.
- mattkrause 6y agoMy lab still does this (or, will do it when we’re finally allowed back in). We do a little bit of compression right on the instrument’s computer. After that, a briskly-walking postdoc can move a few terabytes a minute. The exact rate depends on whether they stop for coffee, but it always blows our internal 100 Mbps network out of the water.
- codezero 6y agoI was just leaving radio astronomy when the MWA was in later stages of design/deployment, I remember the discussions around designing the data storage for it, and it was centered around the idea of having to fly the data set back and forth (At the time, around 2008-2011ish?) - so it was so large that it wasn't really feasible to use the Internet for each data collection run. On the order of many terabytes if I recall correctly.
- torrance 6y agoThe MWA has a direct fibre connection from site to a super computing facility in Perth. Although some processing (correlation and averaging) is usually done on site to reduce the data a little, there are some observation modes that send the raw voltages.
- mhh__ 6y ago"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway" - A. Tanenbaum of MINIX (amongst other things) fame
- koheripbal 6y agoIt's actually incredibly easy to beat the bandwidth of the internet with your car, with just a couple HDs. I routinely drive HDs full of backups a couple hours when going between DC sites.
- deleted 6y ago[deleted]
- alsobrsp 6y agoThe latency will kill you.
- nine_k 6y agoNot necessarily: the car may be there much sooner than the last byte sent via the fiber arrives. Not everything is streaming video; some datasets make most sense only as a whole. Backups are an example.
- doublerebel 6y agoI think the subtext is, it is easily 1000x more dangerous personally to drive the data along public roads, than to send it over a network. The value in avoiding such danger is perhaps incalculable.
- throwaway_pdp09 6y agoData can be encrypted to make it safe. Data will be duplicated onto disks before being driven away. It's not like there can only ever be one copy of the data.
- codingdave 6y agoThat isn't as uncommon as you may think. Especially when talking about DR (disaster recovery) plans, sneakernet is certainly a valid way to get backup data from off-site because you never know what the network status will be when you need to enact your plan.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- NikolaeVarius 6y agoEvent horizon telescope used HDD's being flown around to aggregate their data
- duxup 6y agoI worked for a company CERN had bought equipment from a while back. I was on the technical support team. I had a direct path to engineering when CERN called. The amount of data in an ultra short time was seemingly endless. So many "need more interfaces, more bandwidth...buffer tuning... buffer tuning". When a card burnt up mysteriously I made a sci fi joke about black holes and CERN in my ticket to engineering. It was the one and only time I got them to make a joke back ;)
- 8ytecoder 6y agoNow, it’s unfair to leave your audience hanging like that.
- duxup 6y agoI wish I could remember the joke, but IIRC it was pretty terrible as I was just desperate to get something back from that engineering team that seemed like ... fun. Fortunately they either didn't care for, missed or just never acknowledged most of my puns and really vague references. Outside that one interaction the only thing I knew from them was that despite me being not the best technically compared to my peers, they asked I work specific tickets at times... but I think that was mostly because they knew I didn't make things up and would be honest and say "I don't know". Also I'd give them a heads up when I saw something hitting the fan to give them time to prepare ;)
- BrandoElFollito 6y agoI was doing my PhD when the LHC was being designed. It was interesting to listen to the discussions about signal and data management for the detections closest to the vertex. The rate was so incredible that the triggers were incredibly optimized to get rid of the noise and keep the fist wave of useful data (it was then processed several times down the road). The other interesting thing was to see how people were fighting for real estate in the detector, to be able to place their elements somewhere interesting. I crawled once under one of the detectors (ATLAS) and imagining the several thousand tons above me was frightening.
- danans 6y agoThis is how Google Street view cars work too, because the sheer size of the high resolution photograpic data is so huge that it would make no sense to transmit it over a mobile network, vs just downloading it once the car finishes it's route.
- joering2 6y agoThey could be using peoples wifis on their way to transfer the data as they dash thru the neighbour street. ;)
- savrajsingh 6y agoIt’s amazing that we think this is even possible to achieve — pick up a particular signal that’s been floating around since the beginning of the universe and get interesting info from it... would be so cool if they manage it!
- oh_sigh 6y agoFloating around doesn't do it justice....it's been flying at the speed of light away from its origin for the past 12 billion years until it ran into a little sensor.
- danans 6y ago> This radio telescope consists of 4,096 dipole antennas Does anyone know the significance, if any, of there being 2^12 antennas?
- jandrese 6y agoProbably a 64x64 grid.
- danans 6y agoBut why 64x64 then? Is that just to match some characteristic (i.e . number of inputs) of the data aggregation/logging device (more likely), or is it something about the physics of the type of signal they are trying to detect (less likely)? There are, after all, plenty of possible square grid combinations bit only some are powers of 2.
- praveen9920 6y agoIt must be for separating the visible area into smaller grids for analysis. I also assume some level of error correction is needed and pair of antennas data need to be somehow correlated
- torrance 6y agoThere are 128 grids of 4x4 antennas. There’s nothing special about the 128 figure, and very soon it will be doubled - but there were also talks of a smaller upgrade too. The 4x4 tile size is a compromise between sensitivity and field of view. 5x5 or 6x6 configurations would be much more sensitive to signal but at the expense of a much smaller field of view, since the field of view is formed using phased arrays. For imaging purposes actually we’d really like the smaller field of view, because it would make it easier to avoid nasty bright radio sources from washing out our observations, but for detecting this epoch if reinionization signal, the bigger field of view is better since it’s a global full-sky signal.
- consp 6y agoLOFAR uses 24 (or 48) per station for the "high" frequency tiles and 96 for the lower frequency ones if I'm not mistaken, so a power of two is not by definition required.
- divbzero 6y agoI wonder if the kangaroos hopping by create any interference that they need to correct for.
- flycaliguy 6y agoNot likely. I worked at a research station right down the road from a zoo and we didn’t see any interference from any of the animals. Now, the security fence on the other hand? Static City we used to call it.
- badosu 6y agoA question pops in my head whenever I see news of signal detection from the early age of the universe. As the universe expands we can see further in the past since light has been traveling a longer distance. However, since time has passed to reach the stage where this distance is significant there's a limit to how far we can see, and this limit is continually pushed further as the universe expands (and accelerates expansion). Now, my question is, how is it possible that we can see so far in the past? Has the early expansion rate of the universe been much faster than what I assume that it's still possible to capture these signals, are we capturing signals spread and reflected over time, or is there some other explanation/wrong assumption?
- wahern 6y ago> Now, my question is, how is it possible that we can see so far in the past? I think you're overthinking it. Forget about expansion for a moment, or other weird effects. We can see 12 billion years into the past because the universe is far larger than 12 billion (or 24 billion) light years across--possibly infinite, possibly not, but nonetheless at least that large, and at least that large at the moment the signal was emitted. What we're seeing isn't a reflection, it's light that has been traveling 12 billion years in a straight line from point of origin. Because of expansion, the earth[1] and the emission source may have been closer than 12 billion light years, but they were still very far apart. The big bang doesn't mean the universe exploded from a point source, at least not in any intuitive sense of the phrase. The exact topology is still unknown, but assuming a flat, infinite universe, the universe was already flat and infinite when the signal was emitted. See https://en.wikipedia.org/wiki/Shape_of_the_universe https://en.wikipedia.org/wiki/Shape_of_the_universe [1] Disregarding that the earth and our solar system didn't yet exist.
- waheoo 6y agoThe universe is bigger than the observable universe.
- badosu 6y agoIs that verified? That would be a reasonable explanation
- ComodoHacker 6y agoCan someone ELI5 how sparse neutral hydrogen could physically emit signal so strong it could travel this far and be detected? Like, there were no stars, no nuclear reactions, no interactions with energy exchange(?)... Who lost all that energy?
- torrance 6y agoThe emission is the so-called ‘forbidden’ hyperfine transition of the spin state of Hydrogen’s one electron. It’s flips from spin up to spin down, and the energy difference is a photon at about 1420 MHz. It’s called forbidden because it’s extremely unlikely to occur - so unlikely that I don’t believe it’s been observed in the lab. But in aggregate, with vast vast clouds of hydrogen it becomes likely enough and strong enough to detect. We observe this emission from hydrogen in our own galaxy and through the local universe. And it’s quite strong to easily detect. In this experiment, they’re looking for that same 1420 MHz signal, redshifted down to something like 180 MHz (where exactly it is isn’t known for sure).
- ComodoHacker 6y agoThank you!
- MichalSternik 6y agoHonest to god question: do you think there's a market for moving encrypted SDD drives in big european cities? For stuff that could not and should not be transfered over network. I had this idea for a bicycle messenger & hardware encryption startup, but i don't have any idea what the demand really is like.
- rpadovani 6y agoI would have used that 3/4 times during the last year in a big EU city: we do consulting, and one of our customer is adamant their data cannot go on the public cloud. So they have their own data center, and some file sharing utility. Problem is, you need to go through their VPN, that is super slow, and their sharing utility closes connections after a while. Result: transferring 30/40 GiB of data is painful, and we often go to their office to retrieve the data instead of using the Internet. In an ideal world, I wouldn't need a service like the one you suggest, but since a lot of our customers seems to be in the Stone Age, I would probably use it.
- filleokus 6y agoI have friends who do regular SSD file transfers. Mostly between different countries but sometimes in the same city. They just use regular messaging services and simple of the shelf encryption software with out of bound key transfer (I.e email...). Hardware encryption (and platform for key sharing and booking deliveries) could certainly be useful, but I’m not sure about the transport port. I feel like outsourcing it to normal transporters is more reasonable