5 ms·

> Your hypothesis does not hold up to the data. Your data is not relevant to the hypothesis. The surveys were conducted in different time-frames (Canada: 2009-

> Your hypothesis does not hold up to the data. Your data is not relevant to the hypothesis. The surveys were conducted in different time-frames (Canada: 2009-2014, USA 2005-2010), which I shouldn't need to tell anyone why that dataset is not useful for rigorous comparison, but I will anyway: one surveys people who were just recently hit by a recession and tracks them throughout the economy's recovery; while the other surveys people who recently experienced massive economic growth (exuberance) and tracks them throughout their economy crashing. Both, Canda's and the USA's, economies were intertwined during the 2007-2009 financial crisis, and this should be accounted for -- but it isn't. Furthermore, the data DOES hold up to the GP's assertion: that as the unemployment rate goes up, the gap in life satisfaction between employed and unemployed approaches zero (chart 2). That is, 20% nationwide unemployment, unemployed and employed Canadians have the same life satisfaction (chart 1). Lastly, this is social sciences. We cannot conclude anything from the data without experimentation. At best, this data is observational reason to pursue deeper inquiry, at worst it's idealogical fodder to drive other people's agends by lying with statistics. Everyone does it. Statistics don't really mean anything on their own, and lay-people give them too much weight!

bigcohoneypot 7y ago
For those of us familiar with science, and not social science can you explain why we can't conclude anything from data without further experimentation? Or is this a claim about the scientific method as opposed to say how art is criticized.
- endothrowho333 7y ago
  It's not that you can't conclude anything from data, it's just that the things you can conclude from data are limited to only two "scientifically-sound" conclusions: >1). There is a relationship in your data >2). There isn't a relationship in your data From there, you can then setup further experiments to better understand why there is or isn't a relationship -- and perhaps find the opposite is true! It's the same reason you don't assume a theory is "true;" you don't conclude "that higher unemployment in Canada vis-a-vis the US is almost certainly driven by a lower supply of jobs, rather than as you posit a lower demand for employment" (GGP), because that's bad science! What the GGP did is create an inference -- one based on misread data and faulty assumptions about scientific methods -- and expressed it as de facto. Now, in order for his inference to have weight, he must either support it with further evidence (DIY meta-analysis) or carry out some other type of experiment to test said inference. Here's a very brief primer: https://socialresearchmethods.net/kb/concval.php https://socialresearchmethods.net/kb/concval.php
  - DubiousPusher 7y ago
    A very good example of this is how long it took scientists to be comfortable with positively saying that tobacco use caused lung cancer in people. (A fact tobacco companies greatly leveraged in their defense.) Huge surveys long showed a correlation between smokers and increased incidence of cancer. But it took decades of research to rule out confounding factors. For example, it may have been the case that industrial work was causing the cancer and industrial workers just happened to be more likely to be smokers. There is one tool that really nails causation fairly quickly which is a double blind controlled experiment. But usually in social science it's very hard to or even immoral to conduct such experiments. For example, assigning babies at random to be smokers or not for some period would be pretty hard to carry out and certainly be immoral if you thought the smoking may lead to cancer.
    - bigcohoneypot 7y ago
      You don't think this is an example where industry threw shade and we could have concluded this much earlier?
      - DubiousPusher 7y ago
        Genuinely no. Mostly I think the industry failed to corrupt the scientific process. Their attempts to do so were nakedly transparent. And they absolutely preyed upon the fact that real quality scientists were reluctant to definitively say the link was causal because they were being diligent about the fact that there was a preponderance of evidence that was correlative. They succeeded in corrupting the political process though. I don't think you should need 100% scientific certainty to begin regulatory action. Maybe 80% or 90% of the way is good enough. The industry succeeded in requiring 150% certainty before a public health response could begin.
  - kolbe 7y ago
    What would you say is better: him using an imperfect study, or you just expressing your opinion?
- dmwallin 7y ago
  It’s the equivalent of running an experiment and then looking for correlations. The likelihood of it being chance is drastically increased. There’s an important reason why you are supposed to come up with your hypothesis before. Social science has the issue that theres lots of data to review ex post facto and it’s hard to run rigorous experiments. This means you need extra rigor to avoid spurious correlations.
DubiousPusher 7y ago
Not to mention that people's personal moods do not necessarily reflect the quality of their material position.

bigcohoneypot 7y ago

For those of us familiar with science, and not social science can you explain why we can't conclude anything from data without further experimentation? Or is this a claim about the scientific method as opposed to say how art is criticized.

endothrowho333 7y ago
It's not that you can't conclude anything from data, it's just that the things you can conclude from data are limited to only two "scientifically-sound" conclusions: >1). There is a relationship in your data >2). There isn't a relationship in your data From there, you can then setup further experiments to better understand why there is or isn't a relationship -- and perhaps find the opposite is true! It's the same reason you don't assume a theory is "true;" you don't conclude "that higher unemployment in Canada vis-a-vis the US is almost certainly driven by a lower supply of jobs, rather than as you posit a lower demand for employment" (GGP), because that's bad science! What the GGP did is create an inference -- one based on misread data and faulty assumptions about scientific methods -- and expressed it as de facto. Now, in order for his inference to have weight, he must either support it with further evidence (DIY meta-analysis) or carry out some other type of experiment to test said inference. Here's a very brief primer: https://socialresearchmethods.net/kb/concval.php https://socialresearchmethods.net/kb/concval.php
- DubiousPusher 7y ago
  A very good example of this is how long it took scientists to be comfortable with positively saying that tobacco use caused lung cancer in people. (A fact tobacco companies greatly leveraged in their defense.) Huge surveys long showed a correlation between smokers and increased incidence of cancer. But it took decades of research to rule out confounding factors. For example, it may have been the case that industrial work was causing the cancer and industrial workers just happened to be more likely to be smokers. There is one tool that really nails causation fairly quickly which is a double blind controlled experiment. But usually in social science it's very hard to or even immoral to conduct such experiments. For example, assigning babies at random to be smokers or not for some period would be pretty hard to carry out and certainly be immoral if you thought the smoking may lead to cancer.
  - bigcohoneypot 7y ago
    You don't think this is an example where industry threw shade and we could have concluded this much earlier?
    - DubiousPusher 7y ago
      Genuinely no. Mostly I think the industry failed to corrupt the scientific process. Their attempts to do so were nakedly transparent. And they absolutely preyed upon the fact that real quality scientists were reluctant to definitively say the link was causal because they were being diligent about the fact that there was a preponderance of evidence that was correlative. They succeeded in corrupting the political process though. I don't think you should need 100% scientific certainty to begin regulatory action. Maybe 80% or 90% of the way is good enough. The industry succeeded in requiring 150% certainty before a public health response could begin.
- kolbe 7y ago
  What would you say is better: him using an imperfect study, or you just expressing your opinion?
dmwallin 7y ago
It’s the equivalent of running an experiment and then looking for correlations. The likelihood of it being chance is drastically increased. There’s an important reason why you are supposed to come up with your hypothesis before. Social science has the issue that theres lots of data to review ex post facto and it’s hard to run rigorous experiments. This means you need extra rigor to avoid spurious correlations.

DubiousPusher 7y ago

Not to mention that people's personal moods do not necessarily reflect the quality of their material position.