3 ms·
Good faith is required to accept any type of reporting. Otherwise, your question can always be extended to "How do we know that they didn't just make it all up?
by gryson 7y ago
Good faith is required to accept any type of reporting. Otherwise, your question can always be extended to "How do we know that they didn't just make it all up?"
The researchers did not report conducting more comparisons and they did not adjust for this in their analysis, so we have to assume that they are telling the truth.
Of course, there's always the chance that they are being deceitful, so we have to reserve some skepticism and take into consideration related studies before forming any firm conclusions.
In this particular case, the correlations that they found have quite strong effect sizes with p < .001. Figures 2 and 3 are quite striking. These results are also supported by previous research. That doesn't mean the matter is settled by any means. It's just another piece in the puzzle. But there is no indication whatsoever that they are p-hacking or going on a fishing expedition.
- joe_the_user 7y agoI don't think you have to assume good faith at all in these situations. If someone comes up with a correlation of X statistic with Y statistic in a fashion seemingly coming from nowhere, they either say why they started studying the correlation or show the correlation is incredibly strong. I think that's pretty much what "extraordinary claims require extraordinary evidence" winds-up meaning.
- mattkrause 7y agoThe correlation is so large that I find it literally incredible. Could ramen restaurant prevalence really explain a quarter to a third of the variance in mortality (which is what r=0.5 to 0.6 implies)? If so, the damn things are an outright menace. And why is it so different from fast food, which is equally unhealthy but supposedly makes people live longer?
- perl4ever 7y agoWhat is the connection between a correlation and an explanation? I see none. Zero. Nada.
- mattkrause 7y agoIt is pretty common to interpret r^2 as the fraction of variance in one variable “explained by” the other. Here, ramen restaurant prevalence and mortality are reported to be correlated at r=0.5 to r=0.6, so r^2=0.25 to 0.36 Obviously, “explained” here doesn’t indicate something conceptual or causal—-it’s more like “accounted for”—-but the phrasing is probably too well established to change now. And that's sort of my point...It doesn't seem plausible to me that ramen would be that deadly, so I suspect there's some other unmodelled factor that drives both restaurant prevalence and mortality. For example, maybe ramen is more popular in poorer/more rural areas, which have worse medical care/outcomes.
- gryson 7y agoThe authors discuss this in the paper's Discussion. The prevalence of ramen restaurants is suggested as an indicator of general food preferences in the regions studied. Regions that have a higher prevalence of ramen restaurants also tend to have a higher salt intake in general. This can be seen by the increased use of soy sauce and miso compared to other regions in Japan (from personal experience, these regions are famous throughout Japan for their high-salt diets). So I'd say that the authors are very careful to clarify that they are not implying any sort of causal relationship between ramen restaurants and stroke mortality, and I don't think anyone reading the paper would assume that either.
- mattkrause 7y agoI did see that, and maybe Japanese regional cuisine varies a lot more than I'm imagining. However, I'm still not convinced that something higher-level is going on. For example, I think you could find a similar correlation between (say) soul food restaurants and diabetes prevalence in the US. Soul food tends to be starchy and fatty itself; you could also make a similar argument that the number of soul food restaurants is a proxy for food preferences in each region. However, soul food is especially popular in African-American communities. There are genetic factors that predispose African-Americans to diabetes. Soul food restaurants are probably more common in poorer areas. Diabetes is also linked to poverty: partly through diet, but also through things like access to preventative healthcare and management of pre-diabetes. Perhaps there still is some residual effect of the restaurants themselves, but it would be much weaker after adjusting for other factors.