7 ms·
Algorithm detected over a hundred asteroids after studying old telescope data
- dontbenebby 4y agoWhat's a good way to learn how to do this work, if you have scripting knowledge but not physics? I am working on a set of images to parse with OpenCV (non astronomical) but could apply the same techniques as those in the OP. Often I find lately the barrier to a project is sociological, not technical.
- anovikov 4y agoIf you have 4 dots on a photo or different photos and you want to find out if they belong to the same object (shot at different times) and find orbit of that object, it's trivial to do it. You find centroids of them using opencv, then recognise the plate using the like of www.astrometry.net to build WCS and convert (x,y) coordinates into (Ra, Dec) coordinates, and then you can run orbit determination code and check if resulting orbit is sane (that is, is within certain boundaries of "possible" asteroid orbits). If these are just random dots that don't belong to the same object, or some of them are noise/false detections (like hot pixel, meteor, or similar), the orbit will be invalid/impossible. Trick is now, how to put together these possible groups of 4 for detection. Because they say they have 68 billion observations, certainly combining each to each 4 times is not possible. There must be some heuristic and/or trick to it.
- hkgjjgjfjfjfjf 4y ago
- dontbenebby 4y ago>Because they say they have 68 billion observations, certainly combining each to each 4 times is not possible. There must be some heuristic and/or trick to it. Maybe define heuristics for what ISN'T useful first to pare down the data set (looking for negatives), then focus on the sort where you're actually looking for positives? When I'm sorting photos for art purposes, I shoot a lot, years at a time, then spend a month or two sorting into albums, I actualy lost hours of work because I had to restore my phone from a backup recently. But the general idea, when doing a completely manual process, is you grab say, 1kish of 20k images, then from there further split them into 3-4 albums. Now if they were astronomy photos then you could go into that set of about 500 and manually look, think what's a heuristic to get me from where I started to here more quickly. (Sorry if articulating myself poorly, this site spikes my anxiety -- if I give useful information people just siphon it up, but if you give an opinion they don't like, folks get real ornery.)
- anovikov 4y agoIn fact, it might be that their method is a lot lot simpler: - Detect any protracted images on the photos which shouldn't be protracted - Interpret ends of them as 2 observations and dots at 1/3 and 2/3 as 2 others (exploiting the fact that speed of angular motion at such a short interval is essentially constant). - Run a (very crude) orbit detection using these - Look for any suspicious dots on other images within the margin of error provided by this orbit (that is, where the object with that orbit should've been if it was real, at a time each other shot was made). - If found, use these for orbit refinement, rinse and repeat until on all shots where the object should've been apparent, it is present, or reject it as a false detection if that doesn't happen.
- ad404b8a372f2b9 4y agoThe hard part of research is the domain/scientific knowledge, not the scripting. If you look at the original paper, the method is informed by their knowledge of astrophysics, and their knowledge of the current state of the field and existing methods. Many (maybe most) researchers are terrible programmers, it's a non-issue. If you want to learn how to do this kind of work you have to learn the science.
- dontbenebby 4y agoI guess my question is what you mean by "know the science". For example I know different types of stars and how their compositions differ, but not detailed mathematical models. If by "the science" you mean the latter, great, I can drop the idea, if the former I might be able to add value. (For example, I've read a lot about black holes, Hawking radiation etc, but not in a detailed mathematical sense, just more general ideas like the concept of an event horizon.)
- pvg 4y agoYou don't have to 'know the science' to the degree of detail the people driving the research part of the project are, I think the bigger problem is many of the opportunities for grunt work go to undergrads and graduate students where the hope (in theory at least) is that it also helps advance their educational goals.
- dontbenebby 4y agoNo, the goal is cheap labor, they hire more PhD students than there are tenure track positions, this was a common gripe when so many undergrads commited suicide at IU, they threatened to shut down the newspaper if they reported it and told grad students who were basically extorted into finishing under the threat their advisers would forever gatekeep roles that would take them out of precarity. (That's why I dropped out, worked briefly in policy, learned to hack, then sold stock out of my retirement fund to get an open source intelligence certification from Bellingcat -- because THAT MANY sets of people refused to follow basic social norms.)
- 4y ago
- gammarator 4y agoThere are open positions for software engineers working with these teams: https://b612foundation.org/open-positions/ https://b612foundation.org/open-positions/ https://www.lsstcorporation.org/lincc/job_opportunities https://www.lsstcorporation.org/lincc/job_opportunities
- dontbenebby 4y agoSorry, this is a bit much for my skillset -- I'm more suited to the title of "analyst" but struggle to connect with those roles, but I'll point any folks I know who are stronger software engineers at it. (It's a common issue for us security folks: we're good at writing out pretty code that automates things on the command line with bash and a few esoteric CLI tools, but a full stack developer, I am not.)
- ALittleLight 4y agoKnowing how many asteroids are known is important context to determine the significance of this result. Googling, I get 1,113,527 current known asteroids. I wonder how many additional asteroids they expect to find by applying this technique to other existing datasets and what portion of all available data they used to discover these asteroids.
- ourmandave 4y agoHow old is the data? You can only forecast an asteroid's future path so far, given the snapshot of data you have. Veritasium video with Dave Jewitt of UCLA explains a lot about what were doing about asteroids. https://www.youtube.com/watch?v=4Wrc4fHSCpw https://www.youtube.com/watch?v=4Wrc4fHSCpw
- mturmon 4y agoMostly better source article appeared recently in: https://news.ycombinator.com/item?id=31616340 https://news.ycombinator.com/item?id=31616340 NASA has had a dedicated search for many years: https://cneos.jpl.nasa.gov/ https://cneos.jpl.nasa.gov/ Just detecting these objects in past images isn’t helpful if the orbit can’t be constrained enough to continue tracking it!
- gammarator 4y agoIn many cases the new asteroid discoveries can then be used to identify other apparitions of the same object that were previously not recognized as being asteroids. That enables further orbit refinement.
- mturmon 4y agoSome colleagues have been involved with the NEO detection effort, so I felt called to mention that problem, which does limit the usefulness of these detections, and illustrates one of the system level problems. From the Wired article in the post I linked: > The THOR team has shown the potential to map out the trajectories of numerous asteroids in our neighborhood, but there are few key caveats. Since their images date from a few years ago, asteroids that haven’t been reobserved lately have effectively been lost at this point, although they could be picked up again in newer images down the road, says Paul Chodas, director of the Center for Near-Earth Object Studies at NASA’s Jet Propulsion Laboratory in Pasadena, California.
- xt00 4y agoScott Manley talks about it here: https://youtu.be/W4sOABdM9Q0?t=665 https://youtu.be/W4sOABdM9Q0?t=665
- lostdog 4y agoI think this is the paper: https://arxiv.org/abs/2105.01056 https://arxiv.org/abs/2105.01056 . It's all open sourced, and Python. From the paper, previously you needed a sequence of images from a single night to generate a "tracklet," which lets you estimate a guess of the orbit of the asteroid. The guess can be compared with images from other nights, to confirm the existence and orbit of the asteroids. Without the tracklet, the asteroid could be heading in any direction, meaning that the set of possible orbits is huge, and there would be many many possible matches to other images. The key insight of THOR (their algorithm) is that even a mediocre guess of the orbit makes the true orbit stand out. From an observed object, they predict a test orbit. Then they take all other images that could be along that orbit, project all objects in each image to the original image according the orbit, and then stack the projected images together. If their guess of the orbit was perfect, there should be a clear dot in the stacked projections, where that single asteroid was seen in all the images. But even an imperfect guess is still useful. Any object that's somewhat close to the guessed orbit will show up as a line in the stacked image. So find the lines, use some math to refine the estimate of that objects orbit, and now you've got the object. (I'm not in this field, so take all explanations with a grain of salt please).
- kadoban 4y agoReally interesting. They're using a Hough transform, which is used to pick out shapes from an image (lines and circles most often), especially popular before ML techniques for object recognition existed. Sounds like a great way to search for asteroids, never would have thought of it. It's essentially just a coordinate transform that lets you add up evidence for a particular object, and the objects stand out as little bright points. I'm sure they're doing lots of clever things on top of it of course though.
- dylan604 4y agoLots(most?) comet discovery is now done off of sequences of images and looking for the dots that move in relation to the background stars once the stars are aligned. At that point, you could do differences with subtraction between the plates so that aligned stars disappear and only interesting dots remain. No machine learning required.
- agomez314 4y agoWait, no machine learning?
- geoduck14 4y agoNope. Just Machine Doing
- zh3 4y agoPrevious discussion: https://news.ycombinator.com/item?id=31568837 https://news.ycombinator.com/item?id=31568837
- hulitu 4y agoI wonder how many needles it detected.