3 ms·
That Science paper https://www.science.org/doi/10.1126/science.adq7055 https://www.science.org/doi/10.1126/science.adq7055 ought to be retracted, it is using
by PaulHoule 2y ago
That Science paper
https://www.science.org/doi/10.1126/science.adq7055 https://www.science.org/doi/10.1126/science.adq7055
ought to be retracted, it is using the same ad-hoc methods that we used to wrongly fit power laws to statistical distributions when I was a grad student 30 years ago. I wrote a trash paper too and I'm still angry about not feeling psychologically safe to bring the issue up.
A postdoc who was there wrote a paper many years later about the right way to do it
https://arxiv.org/abs/0706.1062 https://arxiv.org/abs/0706.1062
I don't think anybody read it. Note that the Science paper uses the "Pearson product moment correlation" which is "can only be used to measure the relationship between two variables which are both normally distributed" -- and there's nothing normally distributed here.
- dereify 2y agoOfc nobody reads papers saying your methods are trash; if they read it they simply ignore it. Fwiw Pearson correlation does not assume normally distributed variables, what people usually mean is homoscedascity (normality of residuals) is an assumption for parametric hypothesis testing.
- scarmig 2y agoCosma has a PowerPoint that may be more accessible to a general audience: https://www.stat.cmu.edu/~cshalizi/2010-10-18-Meetup.pdf https://www.stat.cmu.edu/~cshalizi/2010-10-18-Meetup.pdf
- PaulHoule 2y agoExactly, the thing that is "universal" about "universal power laws" is that the evidence is weak for them, not because people did a bad experiment, but because they pulled a data analysis method out of their ass. My one regret of my scientific career was this paper https://arxiv.org/abs/cond-mat/9512055 https://arxiv.org/abs/cond-mat/9512055 where my coauthor asked me "you want to be a theoretician, do you have an explanation?" but I didn't. (I did have some ideas that would have involved knowing more about the structure of the balls, doing more experiments, to give any real answer) Actually I had reservations about the analysis method (got different answers trying two reasonable things that I saw other people to do), it would have been a good paper had I actually addressed the tension but the coworker didn't want to go there. Another interesting thing about my paper was that segmentation was a problem as it was in that Science paper, I didn't do anywhere near the level of sensitivity analysis, but I could have gotten different answers if I'd defined integration windows around the pops differently. Probably the very loud pops also had echos, oh my god that problem was so bad when I tried to do the crumpling in an ordinary room and worse when I made my own small 'soundproof' box, it was better in the anechoic chamber I took the real measurements in. The mystery of explaining the universality is greatly diminished when you can throw out a large number of the papers though...
- gweinberg 2y agoIs your objection that maybe it only looks like a power law distribution for the more common "words", but if you go further out the fit isn't very good? Does it really mean anything if some other curve fits the data better?