4 ms·

While I'm not an expert on these type of studies, I am married to one. I often see these types of criticisms bantered about, but I'm fairly certain that these r

by enjo 16y ago

While I'm not an expert on these type of studies, I am married to one. I often see these types of criticisms bantered about, but I'm fairly certain that these researchers are MUCH more sophisticated in their controls than we give them credit for. There is an entire body of research dedicated to ascertaining the quality and reliability of surveys, how surveys are delivered, and the conclusions that can be drawn from them. In doing these studies those methodologies are generally well understood (and clearly cited) as part of the paper itself. While I know those of us who where educated in the 'hard' sciences (I'm a physicist by training) find it hard to buy into these... I've been quite impressed by how rigorous they really are. I haven't read this particular study, so I may be way off base... but the fact that the fundamentals of the study involved a survey do not make it bad or invalid science (as long as the study was conducted properly that is).

joe_the_user 16y ago
This isn't about how sophisticated researchers are. The limits to research into real world human behavior are pretty much inherent in the complexity and interrelatedness of said human behavior. Doing statistics is extremely hard because behaviors and conditions correlate with a many things at once, making the causal factor very hard to isolate. There have been a number of clever and rather specific ways in which research psychologists have gotten around these problems. But being an expert doesn't make you superman or give you unlimited bag of tricks dealing with this inherent difficulty. One obvious way that the study might be flawed could be that the respondents were more honest about how much TV they watched than about how much working-out they did, so TV-watched actually correlated better with exercise level than claimed level of exercise (and that's just one of hundreds possible problems). IE, until we know some specifics that researchers might have gotten around untruthness in research based on only survey data, we're entirely justified in being somewhat skeptical. "They're expert, they can do it" isn't enough.
chaostheory 16y ago
I can understand the use of surveys when it's not realistic to actually measure this stuff, but in the 2-3 decades since this survey was done we now have technology like this: http://www.polarusa.com/us-en/b2b_products/club_solutions/polar_cardio_gx/polar_cardio_gx/polar_team2_transmitter http://www.polarusa.com/us-en/b2b_products/club_solutions/po... There's no reason why this study shouldn't be updated with real data.

by enjo 16y ago

joe_the_user 16y ago

This isn't about how sophisticated researchers are. The limits to research into real world human behavior are pretty much inherent in the complexity and interrelatedness of said human behavior. Doing statistics is extremely hard because behaviors and conditions correlate with a many things at once, making the causal factor very hard to isolate. There have been a number of clever and rather specific ways in which research psychologists have gotten around these problems. But being an expert doesn't make you superman or give you unlimited bag of tricks dealing with this inherent difficulty. One obvious way that the study might be flawed could be that the respondents were more honest about how much TV they watched than about how much working-out they did, so TV-watched actually correlated better with exercise level than claimed level of exercise (and that's just one of hundreds possible problems). IE, until we know some specifics that researchers might have gotten around untruthness in research based on only survey data, we're entirely justified in being somewhat skeptical. "They're expert, they can do it" isn't enough.

chaostheory 16y ago

I can understand the use of surveys when it's not realistic to actually measure this stuff, but in the 2-3 decades since this survey was done we now have technology like this: http://www.polarusa.com/us-en/b2b_products/club_solutions/polar_cardio_gx/polar_cardio_gx/polar_team2_transmitter http://www.polarusa.com/us-en/b2b_products/club_solutions/po... There's no reason why this study shouldn't be updated with real data.