4 ms·
My experience with big data analysis immediately leads me to suspect that in a dataset of tens of millions of elements, 100 elements having anomalous behavior c
by shadowgovt 7y ago
My experience with big data analysis immediately leads me to suspect that in a dataset of tens of millions of elements, 100 elements having anomalous behavior can actually be operator error, hardware error, noise, anomalous near-Earth object miscatalogued as a star, &c. The article touches on this possibility: "Another explanation – although very unlikely – is they're all just scratches after all, and never existed to begin with." It's unlikely, but unlikely phenomena become phenomena one expects to see if one "rolls the dice" 10 million times.
But it is still an interesting story and a topic worth exploring, even if subsequent research concludes the team has found 100 smudges on glass.
- acqq 7y ago> one expects to see if one "rolls the dice" 10 million times. This study is based on the catalogue of 600 million objects, 60 times more than 10 million times: "In this latest study they compared 600 million objects in the USNO catalogue with a collection put together by the University of Hawaii's Pan-STARR system." "In an earlier study Villarroel and her team compared the positions of some 10 million objects" Emphasis mine.
- shadowgovt 7y agoAh, good catch. So it's even likelier than I anticipate that some errors of improbable cause were observed.
- perl4ever 7y agoI'm surprised that given the amount of data people often deal with at work these days, that more people don't have an intuition for how unlikely coincidences happen when you're dealing with vast numbers of things.
- bookofjoe 7y ago>In probability theory, the birthday problem or birthday paradox concerns the probability that, in a set of n randomly chosen people, some pair of them will have the same birthday. By the pigeonhole principle, the probability reaches 100% when the number of people reaches 367 (since there are only 366 possible birthdays, including February 29). However, 99.9% probability is reached with just 70 people, and 50% probability with 23 people. These conclusions are based on the assumption that each day of the year (excluding February 29) is equally probable for a birthday. Actual birth records show that different numbers of people are born on different days. In this case, it can be shown that the number of people required to reach the 50% threshold is 23 or fewer.[1] For example, if half the people were born on one day and the other half on another day, then any two people would have a 50% chance of sharing a birthday. It may well seem surprising that a group of just 23 individuals is required to reach a probability of 50% that at least two individuals in the group have the same birthday: this result is perhaps made more plausible by considering that the comparisons of birthday will actually be made between every possible pair of individuals = 23 × 22/2 = 253 comparisons, which is well over half the number of days in a year (183 at most), as opposed to fixing on one individual and comparing his or her birthday to everyone else's. From:https://en.wikipedia.org/wiki/Birthday_problem https://en.wikipedia.org/wiki/Birthday_problem
- deleted 7y ago[deleted]
- learnstats2 7y agoI agree: I'd believe in this only if a "star" was observed on two or three clearly independent occasions (enough that the relative presence of anomalies could be confidently ruled out), and then not observed further.
- michelpp 7y agoThese are comparisons of deep, long exposure time survey images. They aren't imaging one star at an instant time, but entire fields of them over many minutes to hours, so: > operator error would effect the entire image. If it were damage to the film/sensor, scratches etc, it wouldn't look like a star point source with a star like point spread function. > hardware error, noise Digital sensors do add noise to an image, but it doesn't occur as bright point sources concentrated in a small area. noise can be removed through statistical analysis, dark frame subtraction, lucky imaging, etc. Dust on the lenses or mirrors aren't in the focal plane and don't resolve to points. Dust on film or sensors would block light, not produce it. Many exposures all over the sky quickly characterize the on sensor defects and other instrument specific information that can be removed from the data. > anomalous near-earth objects these are long exposures, if they are anomalous objects, they would have to have very slow apparent motion against the stellar background. satellites and planes would leave trails. balloons would drift with the wind. what would stay basically static through the entire long-exposure against the celestial sphere? I appreciate your skepticism, I'm just taking the other side of the argument. Thoughts?
- anewguy9000 7y agoim with you. this is astrophysics: there are other lines of evidence to consider: like the missing stars sharing certain properties (color, velocity), suggesting an astrophysical explanation (remember it's not strange for stars to go away. whats new here is how)