5 ms·

As someone who works decently close to, but not in, this area, I am surprised to see this on the front page of HN. The paper authors do not use correct statisti

As someone who works decently close to, but not in, this area, I am surprised to see this on the front page of HN. The paper authors do not use correct statistical practices (e.g. H_0 cannot be fixed “as a nuisance parameter” to remove a degeneracy with another parameter - nuisance parameters must be marginalized over!) and the authors fail to account for several effects in their model (e.g. stretch/color factors for each supernova must be varied) which are known to be necessary for robust inference of cosmological parameters from supernovae data. This is an honest question since I have seen this phenomenon occur a few times now with cosmology/astrophysics papers on HN: How did the original poster find this? And why has it gotten such interest/points? I sincerely hope it is simply a well-intentioned interest in our universe (which it greatly heartens me to see!) combined with naïveté (not meant pejoratively, just to refer to lacking context) wrt the technical nature of this work, but I am interested to hear your thoughts.

throwaway81523 2y ago
It has been circulating on the intertubes. I saw it on a more general interest site earlier today, before seeing it on HN just now.
light_hue_1 2y ago
Funny. This is exactly how I feel about most of the AI/ML papers posted here. Very strange selection of pretty sketchy papers.
- codethief 2y ago
  https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_a...
andrewflnr 2y ago
> well-intentioned interest in our universe I mean, probably. Though HN does have a taste for fringe theories, which might color your interpretation of "well-intentioned". And most of us aren't really qualified to assess the statistical rigor of astrophysics papers, myself certainly included.
necubi 2y ago
I saw it floating around Twitter with some expansive commentary "This could be an incredible revolution in Cosmology. The Dark Energy model of the universe, which won a Nobel Prize in 2011, may be completely wrong.", etc. For whatever reason, engineers really hate dark energy and will glom onto any fringe theory that appears to disprove it. Not to psychoanalyze too much, but it seems to be a topic where non-experts get to feel like they're smarter than those PhD cosmologists because they watched a Sabine Hossenfelder video. See literally any thread about dark energy (or dark matter, which elicits similar reactions) on HN.
- deleted 2y ago
  [deleted]
marcosdumay 2y ago
That is probably the same phenomenon that makes the most sketchy papers on nutrition science to appear on popular news. Sketchy things have the most interesting results. People that want entertaining news select for interesting results, and the sketchy ones get over-represented.
mmooss 2y ago
> why has it gotten such interest/points? For context: The top comment on most HN stories, especially research, tries to completely discredit the OP, often by finding flaws, and especially in statistical methods. Everything has flaws. I think people are interested in what is valuable and possible. Shakespeare's work has many flaws, but that's not what people focus on. Also, while you aren't responsible for all those other top comments, why should I believe yours? Usually I just ignore these comments (but I appreciate your curiosity).
- JoeAltmaier 2y ago
  Failures in statistical methods are not covered by 'everything has flaws'. Those are fatal, existential flaws. Something that is not likely true (though it is erroneously presented as likely of being true) are likely false.
  - mmooss 2y ago
    Working backwards ... > Something that is not likely true (though it is erroneously presented as likely of being true) are likely false. That doesn't make sense to me. Because something isn't proven here, that doesn't make it more likely to be false; it's just uncertain. Poor evidence is not evidence either way. To say it's false, it would need to be proven false, with good evidence. If I assert, 'the universe is expanding because Pluto is further away today than yesterday', my argument wouldn't support the claim but that doesn't logically imply that the universe is not expanding. > Failures in statistical methods are not covered by 'everything has flaws'. Those are fatal, existential flaws. Why are errors in statistical methods -- if they exist here: we have a hot take by a random, anonymous Internet commenter (using a new account) against scientists who spent a long time on this work, and put their names and reputations on it -- somehow more fatal than other errors? For example, some statistical errors lead to weaker results, but results nonetheless. Some lead to results with a somewhat different meaning. (Some lead to stronger results.) We need to deal with imperfect information all the time and find value in it, or we would have almost no information. I spent last week solving a problem with several routers interacting; I had some clear data, some unreliable information, and some black-hole uncertainty; I had to work with what I had and solve the problem. The idea that science is exempt from that is a fantasy of non-scientists, of the religion of science.
SpaceManNabs 2y ago
I have mentioned this in two comments but HN gets really kooky when it comes to cosmology. A few years ago i saw a barrage of highly upvoted papers on MOND and de Broglie stuff. It really made me wonder what else gets posted here that is patently absurd but i dont have the prerequisite knowledge to filter it.
codethief 2y ago
I can't speak to the technical nature of the work, as I don't work in the field, but the last-named author, likely the advisor, seems to be a respectable researcher. > I have served on the committee of the International Society on General Relativity and Gravitation 2017-2022, and am a past President of the Australasian Society for General Relativity and Gravitation and a past President of the New Zealand Institute of Physics. I served 8 years on the editorial board of Classical and Quantum Gravity, 2012-2019. http://www2.phys.canterbury.ac.nz/~dlw24/ http://www2.phys.canterbury.ac.nz/~dlw24/ So while your criticism may well be justified, you make it sound like this is a fringe paper, which I don't think is the case. So instead of opening a meta discussion about what physics papers get posted & upvoted on HN and why (which is hardly novel), I'd be much more interested in how big the issues you're mentioning are and whether the results of the paper could be salvageable. I at least do think that Wiltshire's research is interesting and he has made good points[0] about the challenges of coarse-graining spacetime structures and where & why the underlying assumptions of LCDM might fail to hold. [0]: See e.g. the lecture notes on https://arxiv.org/abs/1311.3787 https://arxiv.org/abs/1311.3787
- astro-cosmo-q 2y ago
  I am certainly not suggesting anything negative about the character or reputation of the authors of the work. I think that work on alternatives to the accepted concordance cosmology model (LCDM) should definitely be explored, and David Wiltshire et al. should pursue this if they deem it promising. As an aside: You may not find this compelling, which is understandable, but I will note that the vast majority of cosmologists (very conservatively 95%) do not question the FLRW aspect of the cosmological concordance model (which is what the paper here does away with in the alternative timescape cosmology), even if they question other parts of it (i.e. by considering dynamical dark energy, neutrino interactions, etc.). I agree that timescape is an interesting idea, but it seems like only a few people have been working on it for over a decade now - unless you have a very (and I think unfairly) dim view of professional cosmologists, if there was a strong case to be made for the timescape model based in data, the greater community would have adopted it by now. Finally, and independent of the sociological points above, an answer to your question about specifics aside from the types of statistical/modeling issues of the type I mentioned above. Again this is not exactly my area and I have not done the analysis myself, so I will refrain from strong opinions. However, my immediate reaction to this is that it is easy to fit one particular dataset (in this case part of the Pantheon+ supernovae sample) with a more complicated model than LCDM, but often these types of models fail when other cosmological datasets are included. Considering such joint constraints with multiple datasets is not an “extra ask” of alternatives to LCDM - to be taken seriously any alternative model should hold up in a joint analysis of precision cosmological datasets (see, for example, the tests of a more seriously considered alternative model in the community, early dark energy [0]). These days, this is often the combination of Cosmic Microwave Background (CMB) Anisotropies, galaxy Baryon Acoustic Oscillations (BAO), and one of several supernovae samples - see e.g. [1] for a somewhat pedagogical overview. The failure to fit several datasets is a common issue e.g. with many MOND papers. In this case, the authors do not try to fit data to anything but a single (modified) supernovae dataset. I would expect that if they included a fit to CMB/BAO data, there would be trouble with the timescape model, as this is exactly what was found in an analysis of the timescape model in another state-of-the-art supernova dataset [2] - there the timescape model could not accommodate the supernova data when combined with CMB/BAO data [2]. [0] https://arxiv.org/abs/2302.09032 https://arxiv.org/abs/2302.09032 [1] https://arxiv.org/pdf/2007.08991 https://arxiv.org/pdf/2007.08991 [2] https://arxiv.org/pdf/2406.05048 https://arxiv.org/pdf/2406.05048
shepardrtc 2y ago
If this is the case then the paper is easily dismissed, correct? How is it being published if the flaws are immediate, obvious, and completely destructive to the idea being presented? I'm legitimately asking - I've published papers and reviewed them, and if I thought something was so thoroughly wrong, I certainly wouldn't give it the ok.
astro-cosmo-q 2y ago
A parenthetical remark just to clarify since I was reading quickly. Wrt the second point in parentheses about stretch factors - upon a second look it is not clear to me exactly what is happening since they say the factors are both fixed for each supernova but then also list the (global?) stretch and color factors in Table 1 (implying that they are varied).
mnau 2y ago
> And why has it gotten such interest/points? * I have seen incredible incompetency in my field. I have no illusions that it is better in other fields (plus personal stories from others about their fields). * Replication crisis. Generally speaking, it threw a massive shade on a lot of fields. * Generally speaking, the "trust the science" message during COVID did a massive damage. * Science news influencers. Dark energy is one of common themes of Sabine Hossenfelder and generally rather negative. * Dark energy is supposed make up ~70% of everything in universe. Yet basically nothing is know about it. That's a hard sell. And yes, every point has a massive asterisk/that's a massive simplification/incorrect. That doesn't help overall perception. Perception is everything on social media. Basically, if there was a theme, it would be "WTF are they doing?" Or if you want a more poetic version "Modern science has been a voyage into the unknown, with a lesson in humility waiting at every stop." There is something seductive about an idea that instead of using existing stuff, scientists just made something new and shiny and were wrong. Happens in our field every day. Your comment only reinforces this. This is a peer-reviewed paper in a reputable journal (Monthly Notices of the Royal Astronomical Society). Yet you are asserting a significant failures to use correct statistical practices. If you are correct, that is just another notch to the "WTF are they doing?" category.
SiempreViernes 2y ago
Maybe it's some deep cultural hangup the CS people got at university, there's certainly too much "I could have done physics" in the top personas of tech for it to be pure coincidence. https://www.youtube.com/watch?v=GmJI6qIqURA&t=1 https://www.youtube.com/watch?v=GmJI6qIqURA&t=1

throwaway81523 2y ago

It has been circulating on the intertubes. I saw it on a more general interest site earlier today, before seeing it on HN just now.

light_hue_1 2y ago

Funny. This is exactly how I feel about most of the AI/ML papers posted here. Very strange selection of pretty sketchy papers.

codethief 2y ago
https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_a...

andrewflnr 2y ago

> well-intentioned interest in our universe I mean, probably. Though HN does have a taste for fringe theories, which might color your interpretation of "well-intentioned". And most of us aren't really qualified to assess the statistical rigor of astrophysics papers, myself certainly included.

necubi 2y ago

I saw it floating around Twitter with some expansive commentary "This could be an incredible revolution in Cosmology. The Dark Energy model of the universe, which won a Nobel Prize in 2011, may be completely wrong.", etc. For whatever reason, engineers really hate dark energy and will glom onto any fringe theory that appears to disprove it. Not to psychoanalyze too much, but it seems to be a topic where non-experts get to feel like they're smarter than those PhD cosmologists because they watched a Sabine Hossenfelder video. See literally any thread about dark energy (or dark matter, which elicits similar reactions) on HN.

deleted 2y ago
[deleted]

marcosdumay 2y ago

That is probably the same phenomenon that makes the most sketchy papers on nutrition science to appear on popular news. Sketchy things have the most interesting results. People that want entertaining news select for interesting results, and the sketchy ones get over-represented.

mmooss 2y ago

> why has it gotten such interest/points? For context: The top comment on most HN stories, especially research, tries to completely discredit the OP, often by finding flaws, and especially in statistical methods. Everything has flaws. I think people are interested in what is valuable and possible. Shakespeare's work has many flaws, but that's not what people focus on. Also, while you aren't responsible for all those other top comments, why should I believe yours? Usually I just ignore these comments (but I appreciate your curiosity).

JoeAltmaier 2y ago
Failures in statistical methods are not covered by 'everything has flaws'. Those are fatal, existential flaws. Something that is not likely true (though it is erroneously presented as likely of being true) are likely false.
- mmooss 2y ago
  Working backwards ... > Something that is not likely true (though it is erroneously presented as likely of being true) are likely false. That doesn't make sense to me. Because something isn't proven here, that doesn't make it more likely to be false; it's just uncertain. Poor evidence is not evidence either way. To say it's false, it would need to be proven false, with good evidence. If I assert, 'the universe is expanding because Pluto is further away today than yesterday', my argument wouldn't support the claim but that doesn't logically imply that the universe is not expanding. > Failures in statistical methods are not covered by 'everything has flaws'. Those are fatal, existential flaws. Why are errors in statistical methods -- if they exist here: we have a hot take by a random, anonymous Internet commenter (using a new account) against scientists who spent a long time on this work, and put their names and reputations on it -- somehow more fatal than other errors? For example, some statistical errors lead to weaker results, but results nonetheless. Some lead to results with a somewhat different meaning. (Some lead to stronger results.) We need to deal with imperfect information all the time and find value in it, or we would have almost no information. I spent last week solving a problem with several routers interacting; I had some clear data, some unreliable information, and some black-hole uncertainty; I had to work with what I had and solve the problem. The idea that science is exempt from that is a fantasy of non-scientists, of the religion of science.

SpaceManNabs 2y ago

I have mentioned this in two comments but HN gets really kooky when it comes to cosmology. A few years ago i saw a barrage of highly upvoted papers on MOND and de Broglie stuff. It really made me wonder what else gets posted here that is patently absurd but i dont have the prerequisite knowledge to filter it.

codethief 2y ago

I can't speak to the technical nature of the work, as I don't work in the field, but the last-named author, likely the advisor, seems to be a respectable researcher. > I have served on the committee of the International Society on General Relativity and Gravitation 2017-2022, and am a past President of the Australasian Society for General Relativity and Gravitation and a past President of the New Zealand Institute of Physics. I served 8 years on the editorial board of Classical and Quantum Gravity, 2012-2019. http://www2.phys.canterbury.ac.nz/~dlw24/ http://www2.phys.canterbury.ac.nz/~dlw24/ So while your criticism may well be justified, you make it sound like this is a fringe paper, which I don't think is the case. So instead of opening a meta discussion about what physics papers get posted & upvoted on HN and why (which is hardly novel), I'd be much more interested in how big the issues you're mentioning are and whether the results of the paper could be salvageable. I at least do think that Wiltshire's research is interesting and he has made good points[0] about the challenges of coarse-graining spacetime structures and where & why the underlying assumptions of LCDM might fail to hold. [0]: See e.g. the lecture notes on https://arxiv.org/abs/1311.3787 https://arxiv.org/abs/1311.3787

astro-cosmo-q 2y ago
I am certainly not suggesting anything negative about the character or reputation of the authors of the work. I think that work on alternatives to the accepted concordance cosmology model (LCDM) should definitely be explored, and David Wiltshire et al. should pursue this if they deem it promising. As an aside: You may not find this compelling, which is understandable, but I will note that the vast majority of cosmologists (very conservatively 95%) do not question the FLRW aspect of the cosmological concordance model (which is what the paper here does away with in the alternative timescape cosmology), even if they question other parts of it (i.e. by considering dynamical dark energy, neutrino interactions, etc.). I agree that timescape is an interesting idea, but it seems like only a few people have been working on it for over a decade now - unless you have a very (and I think unfairly) dim view of professional cosmologists, if there was a strong case to be made for the timescape model based in data, the greater community would have adopted it by now. Finally, and independent of the sociological points above, an answer to your question about specifics aside from the types of statistical/modeling issues of the type I mentioned above. Again this is not exactly my area and I have not done the analysis myself, so I will refrain from strong opinions. However, my immediate reaction to this is that it is easy to fit one particular dataset (in this case part of the Pantheon+ supernovae sample) with a more complicated model than LCDM, but often these types of models fail when other cosmological datasets are included. Considering such joint constraints with multiple datasets is not an “extra ask” of alternatives to LCDM - to be taken seriously any alternative model should hold up in a joint analysis of precision cosmological datasets (see, for example, the tests of a more seriously considered alternative model in the community, early dark energy [0]). These days, this is often the combination of Cosmic Microwave Background (CMB) Anisotropies, galaxy Baryon Acoustic Oscillations (BAO), and one of several supernovae samples - see e.g. [1] for a somewhat pedagogical overview. The failure to fit several datasets is a common issue e.g. with many MOND papers. In this case, the authors do not try to fit data to anything but a single (modified) supernovae dataset. I would expect that if they included a fit to CMB/BAO data, there would be trouble with the timescape model, as this is exactly what was found in an analysis of the timescape model in another state-of-the-art supernova dataset [2] - there the timescape model could not accommodate the supernova data when combined with CMB/BAO data [2]. [0] https://arxiv.org/abs/2302.09032 https://arxiv.org/abs/2302.09032 [1] https://arxiv.org/pdf/2007.08991 https://arxiv.org/pdf/2007.08991 [2] https://arxiv.org/pdf/2406.05048 https://arxiv.org/pdf/2406.05048

shepardrtc 2y ago

If this is the case then the paper is easily dismissed, correct? How is it being published if the flaws are immediate, obvious, and completely destructive to the idea being presented? I'm legitimately asking - I've published papers and reviewed them, and if I thought something was so thoroughly wrong, I certainly wouldn't give it the ok.

astro-cosmo-q 2y ago

A parenthetical remark just to clarify since I was reading quickly. Wrt the second point in parentheses about stretch factors - upon a second look it is not clear to me exactly what is happening since they say the factors are both fixed for each supernova but then also list the (global?) stretch and color factors in Table 1 (implying that they are varied).

mnau 2y ago

> And why has it gotten such interest/points? * I have seen incredible incompetency in my field. I have no illusions that it is better in other fields (plus personal stories from others about their fields). * Replication crisis. Generally speaking, it threw a massive shade on a lot of fields. * Generally speaking, the "trust the science" message during COVID did a massive damage. * Science news influencers. Dark energy is one of common themes of Sabine Hossenfelder and generally rather negative. * Dark energy is supposed make up ~70% of everything in universe. Yet basically nothing is know about it. That's a hard sell. And yes, every point has a massive asterisk/that's a massive simplification/incorrect. That doesn't help overall perception. Perception is everything on social media. Basically, if there was a theme, it would be "WTF are they doing?" Or if you want a more poetic version "Modern science has been a voyage into the unknown, with a lesson in humility waiting at every stop." There is something seductive about an idea that instead of using existing stuff, scientists just made something new and shiny and were wrong. Happens in our field every day. Your comment only reinforces this. This is a peer-reviewed paper in a reputable journal (Monthly Notices of the Royal Astronomical Society). Yet you are asserting a significant failures to use correct statistical practices. If you are correct, that is just another notch to the "WTF are they doing?" category.

SiempreViernes 2y ago

Maybe it's some deep cultural hangup the CS people got at university, there's certainly too much "I could have done physics" in the top personas of tech for it to be pure coincidence. https://www.youtube.com/watch?v=GmJI6qIqURA&t=1 https://www.youtube.com/watch?v=GmJI6qIqURA&t=1