6 ms·

I always get the feeling from reading academic material on psychology that people are trying to apply the scientific method on top of unscientific principles. I

I always get the feeling from reading academic material on psychology that people are trying to apply the scientific method on top of unscientific principles. It feels rather like alchemists trying to explain fire via phlogiston: an attempt to quantify high-level phenomena when you don't understand any of the lower levels. All the data comes from psychology students at a college in the US who either were able to use their participation for a course requirement or given extra credit. The method by which they generated and measured depletion seems adequate, but all the data is self reported based on how the test subjects felt. The statistics gives evidence linking the amount of generated depletion and the reported depletion, but not any insight into what mechanism actually causes depletion. What use is this? Why build a map when we cannot effectively observe the territory?

debacle 9y ago
Data collection in soft sciences definitely turns it into a Hardmode, but that doesn't mean all findings are bunk. https://vimeo.com/118188988 https://vimeo.com/118188988
ajross 9y ago
Modern experimental science is just garbage on the whole. Psychology has the disadvantage of a very poor theoretical foundation, making a lot of its pronouncements seem hokey and fabricated. But on the whole I don't think the prevalence of poor measurement and (espeically) bad statistics is any worse there than it is in, say, biology. Unreproducible experiments are the rule in modern science, not the exception.
- opportune 9y ago
  Depends on the discipline. I haven't really heard of frequent non-reproducible results in physics, chemistry, and geology. It seems to be more of a medicine-biology-psychology problem. Personally I think psychology shouldn't even be a separate field from neuroscience. Psychology is, after all, merely an emergent phenomenon from neuroscience, but culture can muddy whether psychological results are truly indicative of how humans think or rather of how a culture has conditioned us to behave in a certain way. I can't remember his name (and I hope someone does and links his wikipedia article), but there was a very well-known and controversial figure within the psychology/psychiatry community who believed that psychology was only slightly better than pseudo-science, and that oftentimes it merely studies behavior and pathologizes others based on our culture rather than science. I just think we don't know enough about neuroscience to really make psychology a very legitimate field of study - it's like trying to perform chemistry without the knowledge of electrons.
- jpeloquin 9y ago
  While theory indicates that many (or even most) published empirical findings are false [1], this doesn't make experimental science garbage in its entirety. Although the signal from individual papers is noisy, some results do get confirmed repeatedly by independent experiments over time. Those results are likely to be correct. Theory and multiple independent experimental methods should converge on the same result as work continues; if this doesn't happen, it's a red flag. Working scientists have to constantly estimate and re-estimate their confidence in a result, but progress still (so very slowly) gets made. Tracking research developments this way takes a lot of time and expertise. Sometimes unpublished knowledge or insider access are needed. Unfortunately, news reports about science rarely provide the context of prior work and competing hypotheses that is necessary to estimate the veracity of a result. Simple, sensational statements apparently get more clicks. This is not a good situation and leads to the impression that entire fields are garbage. Any effort to fix this communication problem is greatly appreciated. [1] http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 http://journals.plos.org/plosmedicine/article?id=10.1371/jou...
z55 9y ago
It might not seem as clear cut as other natural sciences (which also get fuzzy once you go into the details) but there is credibility in psychological reasoning. The mechanisms that cause a certain behavior are represented as models in psychological research. Such models must, by design, simplify the real mechanisms. But, as opposed to the real mechanism, they can be discussed and disproven, which is what apparently happened here.
- colemannugent 9y ago
  >It might not seem as clear cut as other natural sciences (which also get fuzzy once you go into the details) but there is credibility in psychological reasoning. Any "fuzziness" in the other natural sciences (thinking biology, chemistry, physics, astronomy, etc.) is usually limited by current technology or measurement methods. Newton's classical mechanics were observably true for a long time before their flaws started showing. In most situations, classical mechanics is all you need to accurately model a system. Is there any such well accepted theory that semi-accurately predicts psychological phenomena in a similar fashion? I agree that sometimes psychological findings seem credible because they align with our personal models of how we think people work, but this gives you nothing in terms of the scientific method. Just because a model is intuitive doesn't mean that it is any more true. >Such models must, by design, simplify the real mechanisms. What model? They essentially have a single claim backed by evidence. I don't see what predictive power their conclusion brings. We don't understand what "ego-depletion" is, how can we pretend to measure it? How can this "measurement" of something that we don't understand give us any more information about human psychology? What real mechanism have they suggested as an explanation for this phenomenon? I didn't see one.
- Retric 9y ago
  When papers drop below the 50/50 threshold and are more likely to be false then true, the entire discipline becomes worthless.
raverbashing 9y ago
Data collection on a lot of research papers today: quiz on Mechanical Turk
aetherson 9y ago
While most modern psychology seems to be, well, kind of bullshit, the problem isn't with the idea of observing behavior and trying to deduce underlying principles from that. How do you think that we arrived on those lower-level phenomena that you're lauding? It wasn't by getting an electron microscope and a particle collider from God and starting from there. And we disproved phlogisten without use of lower-level principles either -- just better measurement.
- colemannugent 9y ago
  Phlogiston was actually disproved by applying the lower level principle of weight to the supposed phlogiston reaction. The idea was that phlogiston was released when things burned. This was disproved when it was observed that some things gained weight when burned. Either the concept of weight was wrong, or the concept of phlogiston was wrong. The scientific method requires you to reduce your reasoning into the lowest-level possible discrete arguments. In this case, scientists were forced to abandon phlogiston as it worked on a very high level, but did not integrate with lower level observations. My gripe boils down to a desire of psychologists to provide sweeping high-level conclusions without even being able to describe possible lower-level phenomena.
  - canjobear 9y ago
    Sometimes you don't need knowledge of the lower-level phenomena. Darwin's arguments for evolution by natural selection were persuasive even though neither he nor contemporaries could conceive of what the chemical basis of gene transmission would look like.
    - colemannugent 9y ago
      But they did know that genes were transferred. Modern psychology feels to me like researchers all want to be the one who gets to the big revelation first and so they abandon the typical scientific pursuit of building a standard model. Newton came up with classical mechanics by simply observing mechanical systems. Darwin and friends came up with the idea of gene transmission by observing the offspring of plants and animals. The difference between them and modern psychologists is that psychologists make very little attempt to explain the underlying phenomena. Without any explanations, modern psychology is little more than a series of observations. Science is a process, and I feel like that you can't claim real knowledge of things if you can't explain them.
edilastic 9y ago
What the hell are you reading? By your reasoning no sciences should exist until we completely understand the mechanisms at a lower level. Plenty of phenomena are observed and even experimentally manipulated all the time without understanding mechanisms. This happens in lots of biology all the time--infectious disease being one prominent example but there are thousands of other examples you could pick. This also happens with various phenomena in physics and chemistry as well, and is routine in astronomy (minus the experiments perhaps). The undergrad student example is an armchair criticism, but think about it carefully: if the process you're studying shouldn't vary across populations, in a fundamental way, why does it matter if you use undergrads or 40 year old engineers? Ego depletion is not such an unreasonable hypothesis. People perceive many things as effortful, and become exhausted. Try running a marathon for example, and tell me there's no such thing as fatigue. A similar thing is also reported and observed for temptation. This is extremely common, so much so that people here are variously posting that the presence of the effect is obvious, and its absence is obvious. It's a bedrock idea invoked by all sorts of people all the time in explaining unhealthy behavior. It's also not unreasonable to assume, for example, that glucose in the brain is a limited resource, and that certain pathways are involved in certain types of tasks more, and might need rest. But you don't need to study the brain to examine whether depletion exists as a phenotype. The effect in question is about the depletion idea, period, not about the mechanism. And you don't need a mechanism to study whether or not it exists. Your final sentence betrays the problem in your reasoning: various lines of reasoning, including this paper but not limited to it, and involving all sorts of behavioral tasks, suggest that the depletion effect does not exist. So what insight is there to have in a mechanism that is not present? It's an illusion. This seems nontrivial to me.
- dang 9y ago
  > What the hell are you reading? Uncivil attacks will get you banned here. Please don't do that. Also, would you please stop bulk-creating HN accounts? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. HN is a community. If users don't have some consistent identity for others to relate to, we may as well have no usernames and no community at all. That would be quite a different kind of forum. Anonymity is fine, and throwaways for a specific purpose are ok—just not routinely. More here: https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix=false&page=0&query=by:dang%20community%20identity https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
rad88 9y ago
Science isn't driven by principles like that. Nothing is built from the ground up, and data that isn't formally understood or modeled is relied upon often. Though we think of it as a dead end today, the experiments and thinking of alchemists (like Isaac Newton) are part of the history of science and were essential to the development of e.g. chemistry and physics.

debacle 9y ago

Data collection in soft sciences definitely turns it into a Hardmode, but that doesn't mean all findings are bunk. https://vimeo.com/118188988 https://vimeo.com/118188988

ajross 9y ago

Modern experimental science is just garbage on the whole. Psychology has the disadvantage of a very poor theoretical foundation, making a lot of its pronouncements seem hokey and fabricated. But on the whole I don't think the prevalence of poor measurement and (espeically) bad statistics is any worse there than it is in, say, biology. Unreproducible experiments are the rule in modern science, not the exception.

opportune 9y ago
Depends on the discipline. I haven't really heard of frequent non-reproducible results in physics, chemistry, and geology. It seems to be more of a medicine-biology-psychology problem. Personally I think psychology shouldn't even be a separate field from neuroscience. Psychology is, after all, merely an emergent phenomenon from neuroscience, but culture can muddy whether psychological results are truly indicative of how humans think or rather of how a culture has conditioned us to behave in a certain way. I can't remember his name (and I hope someone does and links his wikipedia article), but there was a very well-known and controversial figure within the psychology/psychiatry community who believed that psychology was only slightly better than pseudo-science, and that oftentimes it merely studies behavior and pathologizes others based on our culture rather than science. I just think we don't know enough about neuroscience to really make psychology a very legitimate field of study - it's like trying to perform chemistry without the knowledge of electrons.
jpeloquin 9y ago
While theory indicates that many (or even most) published empirical findings are false [1], this doesn't make experimental science garbage in its entirety. Although the signal from individual papers is noisy, some results do get confirmed repeatedly by independent experiments over time. Those results are likely to be correct. Theory and multiple independent experimental methods should converge on the same result as work continues; if this doesn't happen, it's a red flag. Working scientists have to constantly estimate and re-estimate their confidence in a result, but progress still (so very slowly) gets made. Tracking research developments this way takes a lot of time and expertise. Sometimes unpublished knowledge or insider access are needed. Unfortunately, news reports about science rarely provide the context of prior work and competing hypotheses that is necessary to estimate the veracity of a result. Simple, sensational statements apparently get more clicks. This is not a good situation and leads to the impression that entire fields are garbage. Any effort to fix this communication problem is greatly appreciated. [1] http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 http://journals.plos.org/plosmedicine/article?id=10.1371/jou...

z55 9y ago

It might not seem as clear cut as other natural sciences (which also get fuzzy once you go into the details) but there is credibility in psychological reasoning. The mechanisms that cause a certain behavior are represented as models in psychological research. Such models must, by design, simplify the real mechanisms. But, as opposed to the real mechanism, they can be discussed and disproven, which is what apparently happened here.

colemannugent 9y ago
>It might not seem as clear cut as other natural sciences (which also get fuzzy once you go into the details) but there is credibility in psychological reasoning. Any "fuzziness" in the other natural sciences (thinking biology, chemistry, physics, astronomy, etc.) is usually limited by current technology or measurement methods. Newton's classical mechanics were observably true for a long time before their flaws started showing. In most situations, classical mechanics is all you need to accurately model a system. Is there any such well accepted theory that semi-accurately predicts psychological phenomena in a similar fashion? I agree that sometimes psychological findings seem credible because they align with our personal models of how we think people work, but this gives you nothing in terms of the scientific method. Just because a model is intuitive doesn't mean that it is any more true. >Such models must, by design, simplify the real mechanisms. What model? They essentially have a single claim backed by evidence. I don't see what predictive power their conclusion brings. We don't understand what "ego-depletion" is, how can we pretend to measure it? How can this "measurement" of something that we don't understand give us any more information about human psychology? What real mechanism have they suggested as an explanation for this phenomenon? I didn't see one.
Retric 9y ago
When papers drop below the 50/50 threshold and are more likely to be false then true, the entire discipline becomes worthless.

raverbashing 9y ago

Data collection on a lot of research papers today: quiz on Mechanical Turk

aetherson 9y ago

While most modern psychology seems to be, well, kind of bullshit, the problem isn't with the idea of observing behavior and trying to deduce underlying principles from that. How do you think that we arrived on those lower-level phenomena that you're lauding? It wasn't by getting an electron microscope and a particle collider from God and starting from there. And we disproved phlogisten without use of lower-level principles either -- just better measurement.

colemannugent 9y ago
Phlogiston was actually disproved by applying the lower level principle of weight to the supposed phlogiston reaction. The idea was that phlogiston was released when things burned. This was disproved when it was observed that some things gained weight when burned. Either the concept of weight was wrong, or the concept of phlogiston was wrong. The scientific method requires you to reduce your reasoning into the lowest-level possible discrete arguments. In this case, scientists were forced to abandon phlogiston as it worked on a very high level, but did not integrate with lower level observations. My gripe boils down to a desire of psychologists to provide sweeping high-level conclusions without even being able to describe possible lower-level phenomena.
- canjobear 9y ago
  Sometimes you don't need knowledge of the lower-level phenomena. Darwin's arguments for evolution by natural selection were persuasive even though neither he nor contemporaries could conceive of what the chemical basis of gene transmission would look like.
  - colemannugent 9y ago
    But they did know that genes were transferred. Modern psychology feels to me like researchers all want to be the one who gets to the big revelation first and so they abandon the typical scientific pursuit of building a standard model. Newton came up with classical mechanics by simply observing mechanical systems. Darwin and friends came up with the idea of gene transmission by observing the offspring of plants and animals. The difference between them and modern psychologists is that psychologists make very little attempt to explain the underlying phenomena. Without any explanations, modern psychology is little more than a series of observations. Science is a process, and I feel like that you can't claim real knowledge of things if you can't explain them.

edilastic 9y ago

What the hell are you reading? By your reasoning no sciences should exist until we completely understand the mechanisms at a lower level. Plenty of phenomena are observed and even experimentally manipulated all the time without understanding mechanisms. This happens in lots of biology all the time--infectious disease being one prominent example but there are thousands of other examples you could pick. This also happens with various phenomena in physics and chemistry as well, and is routine in astronomy (minus the experiments perhaps). The undergrad student example is an armchair criticism, but think about it carefully: if the process you're studying shouldn't vary across populations, in a fundamental way, why does it matter if you use undergrads or 40 year old engineers? Ego depletion is not such an unreasonable hypothesis. People perceive many things as effortful, and become exhausted. Try running a marathon for example, and tell me there's no such thing as fatigue. A similar thing is also reported and observed for temptation. This is extremely common, so much so that people here are variously posting that the presence of the effect is obvious, and its absence is obvious. It's a bedrock idea invoked by all sorts of people all the time in explaining unhealthy behavior. It's also not unreasonable to assume, for example, that glucose in the brain is a limited resource, and that certain pathways are involved in certain types of tasks more, and might need rest. But you don't need to study the brain to examine whether depletion exists as a phenotype. The effect in question is about the depletion idea, period, not about the mechanism. And you don't need a mechanism to study whether or not it exists. Your final sentence betrays the problem in your reasoning: various lines of reasoning, including this paper but not limited to it, and involving all sorts of behavioral tasks, suggest that the depletion effect does not exist. So what insight is there to have in a mechanism that is not present? It's an illusion. This seems nontrivial to me.

dang 9y ago
> What the hell are you reading? Uncivil attacks will get you banned here. Please don't do that. Also, would you please stop bulk-creating HN accounts? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. HN is a community. If users don't have some consistent identity for others to relate to, we may as well have no usernames and no community at all. That would be quite a different kind of forum. Anonymity is fine, and throwaways for a specific purpose are ok—just not routinely. More here: https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix=false&page=0&query=by:dang%20community%20identity https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...

rad88 9y ago

Science isn't driven by principles like that. Nothing is built from the ground up, and data that isn't formally understood or modeled is relied upon often. Though we think of it as a dead end today, the experiments and thinking of alchemists (like Isaac Newton) are part of the history of science and were essential to the development of e.g. chemistry and physics.