5 ms·

I don't see where they accounted for those who incorrectly believe they understand the topic vs those who actually understand. Unless they are claiming the pro

I don't see where they accounted for those who incorrectly believe they understand the topic vs those who actually understand. Unless they are claiming the process either: 1) detects how the brain correctly models Newtonian physics vs incorrectly. or 2) detects accurate knowledge from incorrect. I can't imagine this lol rich assertion would be the actual claim. But the title does vaguely imply it.

PascLeRasc 7y ago
With these kind of fMRI studies, there's usually a written assessment before or after the scan to verify your understanding of the concepts. They're making claim 1, that you can detect how learning happens. There's also usually a baseline scan before the class or whatever education method takes place. Source: I work in a very similar fMRI lab.
- joe_the_user 7y ago
  Sure, I believe you can rigorously find brain patterns that are correlated with understanding. The obvious, "correlation is not causation" applies, however. The problem is such pattern aren't any guarantee of understanding. Just "feeling confident" seems like such an easily correlated thing. More people who learn a subject sufficiently to do well on a test might "feel confident" but certain kinds of people might just feel confident without that actual learning.
  - elefanten 7y ago
    But they're testing the participants to see if they actually understand. Re-read gp
    - jgtrosh 7y ago
      They're testing understanding for the learning set, but applying a correlation to estimate understanding on the test set.
      - TrinaryWorksToo 7y ago
        What is scientific causation but correlation that works to an acceptable quantity of Sigma?
        Nasrudith 7y ago
        Causation also implies well sequential causality in some way and reproducibility. A subtle but crucial difference. If most people on the battlefield are covered in their own blood will soon die that doesn't mean gathering a gallon of a person's own blood through several donations and then throwing it at them at a Gettysburg musuem will kill them.
        mannykannot 7y ago
        As Feynman discussed in "The Character of Physical Law", there is more to it than that, but I think one can say that the empirical evidence underpinning all science can be characterized as correlation that passes various statistical tests.
        krisrm 7y ago
        Causal studies need to control for confounding variables, for one. There are all sorts of spurious non causal correlations with low p values.
- mannykannot 7y ago
  I have a related, but different, question about what the classifier is discriminating on. The article quotes Kraemer as saying “in the study, we found that when engineering students looked at images of real-world structures, the students would automatically apply their engineering knowledge, and would see the differences between structures such as whether it was a cantilever, truss or vertical load.” This makes me wonder whether the algorithm is discriminating on knowledge of engineering terminology, rather than what the subjects understand of the physics involved. I am curious as to how the non-engineer subjects who got the answer right compare to the engineers. Update: on reading the paper, it seems that the goal of the experiment was to distinguish between engineers and novices, and it conflates 'knowledge' with 'understanding', so the claims that the discrimination is on the basis the subjects' understanding of the issues appears to be editorial overreach.
pessimizer 7y ago
fMRI wrecked by impostor syndrome and Dunning-Kruger.
porky 7y ago
Also, it is a matter of state of mind and mood - you might not believe you understand the concept because you do have bigger picture, but you might also understand it as intended by the test, but not believe that you yourself understands. We have an image of ourselves

PascLeRasc 7y ago

With these kind of fMRI studies, there's usually a written assessment before or after the scan to verify your understanding of the concepts. They're making claim 1, that you can detect how learning happens. There's also usually a baseline scan before the class or whatever education method takes place. Source: I work in a very similar fMRI lab.

joe_the_user 7y ago
Sure, I believe you can rigorously find brain patterns that are correlated with understanding. The obvious, "correlation is not causation" applies, however. The problem is such pattern aren't any guarantee of understanding. Just "feeling confident" seems like such an easily correlated thing. More people who learn a subject sufficiently to do well on a test might "feel confident" but certain kinds of people might just feel confident without that actual learning.
- elefanten 7y ago
  But they're testing the participants to see if they actually understand. Re-read gp
  - jgtrosh 7y ago
    They're testing understanding for the learning set, but applying a correlation to estimate understanding on the test set.
    - TrinaryWorksToo 7y ago
      What is scientific causation but correlation that works to an acceptable quantity of Sigma?
      - Nasrudith 7y ago
        Causation also implies well sequential causality in some way and reproducibility. A subtle but crucial difference. If most people on the battlefield are covered in their own blood will soon die that doesn't mean gathering a gallon of a person's own blood through several donations and then throwing it at them at a Gettysburg musuem will kill them.
      - mannykannot 7y ago
        As Feynman discussed in "The Character of Physical Law", there is more to it than that, but I think one can say that the empirical evidence underpinning all science can be characterized as correlation that passes various statistical tests.
      - krisrm 7y ago
        Causal studies need to control for confounding variables, for one. There are all sorts of spurious non causal correlations with low p values.
mannykannot 7y ago
I have a related, but different, question about what the classifier is discriminating on. The article quotes Kraemer as saying “in the study, we found that when engineering students looked at images of real-world structures, the students would automatically apply their engineering knowledge, and would see the differences between structures such as whether it was a cantilever, truss or vertical load.” This makes me wonder whether the algorithm is discriminating on knowledge of engineering terminology, rather than what the subjects understand of the physics involved. I am curious as to how the non-engineer subjects who got the answer right compare to the engineers. Update: on reading the paper, it seems that the goal of the experiment was to distinguish between engineers and novices, and it conflates 'knowledge' with 'understanding', so the claims that the discrimination is on the basis the subjects' understanding of the issues appears to be editorial overreach.

pessimizer 7y ago

fMRI wrecked by impostor syndrome and Dunning-Kruger.

porky 7y ago

Also, it is a matter of state of mind and mood - you might not believe you understand the concept because you do have bigger picture, but you might also understand it as intended by the test, but not believe that you yourself understands. We have an image of ourselves