3 ms·

It's funny. Somehow I don't think the standard of evidence[RCTs] we get calls for in nutrition studies(for example), will be applied equally here. It's an epist

It's funny. Somehow I don't think the standard of evidence[RCTs] we get calls for in nutrition studies(for example), will be applied equally here. It's an epistemological double-standard that's rarely given serious consideration. I wouldn't give correlative studies much thought. It could be just as likely that people who were more anxious or depressed were found to be mobile phone users or any number of other things causing both anxiety and depression and increased mobile phone use. Critique of causal errors should be no different here than any other conclusion made from correlations.

11101010001100 4y ago
How do you propose to use RCTs to test the claim? As an aside, do you think that the data, not the hypothesis, is accurate (i.e., that mental health has declined in teens)?
- kelseyfrog 4y ago
  > How do you propose to use RCTs to test the claim? I'm not proposing to use RCTs to test the claim. I'm proposing that the conclusions be scrutinized to the same degree we scruitinize other claims made without RCTs. Nutrition studies are a great example that get routinely criticized here because they also rarely backed by RCTs. I'm proposing the same standard of critique. > As an aside, do you think that the data, not the hypothesis, is accurate (i.e., that mental health has declined in teens)? I haven't seen the data nor the data collection methodology. Even so, there are basic concerns that relate to health and specifically mental health that we can ask. Did reporting standards remain consistent throughout the study period? If we assume they did, this assumption becomes "baked" into the conclusions. Assumptions don't merely vanish once we reach a conclusion. Secondly, mental health(/illness) is both an umbrella term and one subject to operationalization errors more than say(extending the analogy) weight is. We would be suspicious if a nutritional study measuring health outcomes relied on self-reported bodyweight. We should be equally suspicious when mental health studies rely on self-reporting. I'm not arguing here that we should instead fMRI subjects, just that having an appropriate level of self-reporting suspicion is healthy, rational, skepticism. Finally, unlike weight, measuring mental health has unique and different operationalization characteristics. The connection between weight and physical health, as far as we can both agree, has a different associative-conceptual connection than self-harm admittance and mental health. We should reasonably rule out basic alternative hypotheses. Were there factors (economic, social, institutional) that could explain(even partially) the change in admittance rates? Did admittance standards change during the timeframe? Even a beneficial change, like awareness-raising campaigns could result in increased admittence. See Logic's song “1-800-273-8255”[1] as a good reason hotline calls increased after it's release that is arguably a good thing rather than indicative of a negative effect. At the end of the day, there is a lot of low-hanging fruit left to explore. Coupled with the previously mentioned skepticism, I wouldn't exactly put my faith in this. Let's be mature and say that's different from saying it's wrong. We're all invested in finding out the exact nature of this phenomenon. 1. https://thehill.com/changing-america/well-being/mental-health/585812-rapper-logics-song-with-suicide-prevention-hotline/ https://thehill.com/changing-america/well-being/mental-healt...
ouid 4y ago
Psychology is fundamentally not science. I don't mean to devalue it as a discipline, but it is incorrect to call it science for a pretty simple epistemological reason. Science depends upon the reproducibility of experiments. This means if I perform an experiment twice, then the second experiment (up to relativity) is done in the context of the experiment having already been done once, so the outcome of these experiments can be thought of as a fixed point of performing applying the have-done-the-experiment context (often with respect to all other experiments). Science is made up of these fixed points, which we can think of as the "set" of properties of the capital-W World. Fundamentally, however, what it means to observe the outcome of an experiment, is to be able to condition your behavior upon that outcome, so if your own behavior is the thing that you are observing, there aren't necessarily any fixed points of this process. This isn't merely a technicality either. For example: I measure that people experiencing insecurity about their performance are mostly pessimistic, and dub this phenomenon "impostor syndrome". I publish my results and, as a result, people assume that the insecurity they feel about their performance is explained by impostor syndrome, rather than being an honest evaluation of their performance. The next time I try to measure this phenomenon in the population, I might indeed find that it's the other way around, that people are optimistic about their performance, as a result of the previous experiment.
- kelseyfrog 4y ago
  I'm straying a bit off topic, but while agree on some of the premises, I reach a different conclusion. I agree with you, and probably more broadly that studying systems which aren't fully isolatable is particularly challenging, but I think there is more than a single simple explanation for this. One we need to get out of the way is that there are things under the psychology umbrella that are science and ones that aren't and what is and isn't science isn't simply topic-based, it's approach-based. For example, reaction-time is a psychological measurement. Is reaction time scientific? I think that's an ill-posed question. It's, like you mentioned, an epistomological question: specifically an ontological question[1]. In the sense that reaction-time is measurable and largely repeatable with respect to specific stimulus, yes, measuring it and analyzing the results can be a step in the scientific path toward knowledge. I'd find it surprising if most people disagreed with this. Let's take a more difficult example, is behavioral psychology science? Again, an ill-posed question. Can we ask scientific question of behavioral psychology? Sure; Does the intervention of CBT in anxiety-diagnosed subjects (as opposed to a non-intervention control) result in lower post-treatment hospital admittance? That's a valid scientific question. Does it say anything about how CBT works? No. It relates a treatment to a result, and if we want to be more specific, a treatment at a point in time, for a specific group. Anticipating the test-retest issue you mentioned above, it's being loose with assumptions and isolation. These things can either be controlled for, or assumed. We should be honest with ourselves that far too many times, they haven't been in practice, but let's not conclude from that that psychology has an essential characteristic of not being scientific. It has a social problem instead. I think we can all be more transparent about assumptions, conditions, and generalization, but I think that's a benefit, not a disadvantage. It enables us to be more precise in our ideas, concepts, and language and that's a good thing. Specifically regarding the fixed-point framing. It's an assumption of time-invariance. I'm not exactly an expert on the philosophy of science, but I'd be surprised not to find stronger and weaker versions of that assumption and that's a reasonable thing to be transparent about when communicating validity. 1. As an aside, it's funny that "Is reaction time scientific?" is itself not a scientific question.

11101010001100 4y ago

How do you propose to use RCTs to test the claim? As an aside, do you think that the data, not the hypothesis, is accurate (i.e., that mental health has declined in teens)?

kelseyfrog 4y ago
> How do you propose to use RCTs to test the claim? I'm not proposing to use RCTs to test the claim. I'm proposing that the conclusions be scrutinized to the same degree we scruitinize other claims made without RCTs. Nutrition studies are a great example that get routinely criticized here because they also rarely backed by RCTs. I'm proposing the same standard of critique. > As an aside, do you think that the data, not the hypothesis, is accurate (i.e., that mental health has declined in teens)? I haven't seen the data nor the data collection methodology. Even so, there are basic concerns that relate to health and specifically mental health that we can ask. Did reporting standards remain consistent throughout the study period? If we assume they did, this assumption becomes "baked" into the conclusions. Assumptions don't merely vanish once we reach a conclusion. Secondly, mental health(/illness) is both an umbrella term and one subject to operationalization errors more than say(extending the analogy) weight is. We would be suspicious if a nutritional study measuring health outcomes relied on self-reported bodyweight. We should be equally suspicious when mental health studies rely on self-reporting. I'm not arguing here that we should instead fMRI subjects, just that having an appropriate level of self-reporting suspicion is healthy, rational, skepticism. Finally, unlike weight, measuring mental health has unique and different operationalization characteristics. The connection between weight and physical health, as far as we can both agree, has a different associative-conceptual connection than self-harm admittance and mental health. We should reasonably rule out basic alternative hypotheses. Were there factors (economic, social, institutional) that could explain(even partially) the change in admittance rates? Did admittance standards change during the timeframe? Even a beneficial change, like awareness-raising campaigns could result in increased admittence. See Logic's song “1-800-273-8255”[1] as a good reason hotline calls increased after it's release that is arguably a good thing rather than indicative of a negative effect. At the end of the day, there is a lot of low-hanging fruit left to explore. Coupled with the previously mentioned skepticism, I wouldn't exactly put my faith in this. Let's be mature and say that's different from saying it's wrong. We're all invested in finding out the exact nature of this phenomenon. 1. https://thehill.com/changing-america/well-being/mental-health/585812-rapper-logics-song-with-suicide-prevention-hotline/ https://thehill.com/changing-america/well-being/mental-healt...

ouid 4y ago

Psychology is fundamentally not science. I don't mean to devalue it as a discipline, but it is incorrect to call it science for a pretty simple epistemological reason. Science depends upon the reproducibility of experiments. This means if I perform an experiment twice, then the second experiment (up to relativity) is done in the context of the experiment having already been done once, so the outcome of these experiments can be thought of as a fixed point of performing applying the have-done-the-experiment context (often with respect to all other experiments). Science is made up of these fixed points, which we can think of as the "set" of properties of the capital-W World. Fundamentally, however, what it means to observe the outcome of an experiment, is to be able to condition your behavior upon that outcome, so if your own behavior is the thing that you are observing, there aren't necessarily any fixed points of this process. This isn't merely a technicality either. For example: I measure that people experiencing insecurity about their performance are mostly pessimistic, and dub this phenomenon "impostor syndrome". I publish my results and, as a result, people assume that the insecurity they feel about their performance is explained by impostor syndrome, rather than being an honest evaluation of their performance. The next time I try to measure this phenomenon in the population, I might indeed find that it's the other way around, that people are optimistic about their performance, as a result of the previous experiment.

kelseyfrog 4y ago
I'm straying a bit off topic, but while agree on some of the premises, I reach a different conclusion. I agree with you, and probably more broadly that studying systems which aren't fully isolatable is particularly challenging, but I think there is more than a single simple explanation for this. One we need to get out of the way is that there are things under the psychology umbrella that are science and ones that aren't and what is and isn't science isn't simply topic-based, it's approach-based. For example, reaction-time is a psychological measurement. Is reaction time scientific? I think that's an ill-posed question. It's, like you mentioned, an epistomological question: specifically an ontological question[1]. In the sense that reaction-time is measurable and largely repeatable with respect to specific stimulus, yes, measuring it and analyzing the results can be a step in the scientific path toward knowledge. I'd find it surprising if most people disagreed with this. Let's take a more difficult example, is behavioral psychology science? Again, an ill-posed question. Can we ask scientific question of behavioral psychology? Sure; Does the intervention of CBT in anxiety-diagnosed subjects (as opposed to a non-intervention control) result in lower post-treatment hospital admittance? That's a valid scientific question. Does it say anything about how CBT works? No. It relates a treatment to a result, and if we want to be more specific, a treatment at a point in time, for a specific group. Anticipating the test-retest issue you mentioned above, it's being loose with assumptions and isolation. These things can either be controlled for, or assumed. We should be honest with ourselves that far too many times, they haven't been in practice, but let's not conclude from that that psychology has an essential characteristic of not being scientific. It has a social problem instead. I think we can all be more transparent about assumptions, conditions, and generalization, but I think that's a benefit, not a disadvantage. It enables us to be more precise in our ideas, concepts, and language and that's a good thing. Specifically regarding the fixed-point framing. It's an assumption of time-invariance. I'm not exactly an expert on the philosophy of science, but I'd be surprised not to find stronger and weaker versions of that assumption and that's a reasonable thing to be transparent about when communicating validity. 1. As an aside, it's funny that "Is reaction time scientific?" is itself not a scientific question.