6 ms·

My previous job was at a startup doing BMI, for research. For the first time I had the chance to work with expensive neural signal measurement tools (mainly EEG

My previous job was at a startup doing BMI, for research. For the first time I had the chance to work with expensive neural signal measurement tools (mainly EEG for us, but some teams used fMRI). and quickly did I learn how absolute horrible the signal to noise ratio (SNR) was in this field. And how it was almost impossible to reproduce many published and well cited result. It was both exciting and jarring to talk with the neuroscientist, because they ofc knew about this and knew how to read the papers but the one doing more funding/business side ofc didn't really spend much time putting emphasis on that. One of the team presented a accepted paper that basically used Deep Learning (Attention) to predict images that a person was thinking of, from the fMRI signals. When I asked "but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact?" and there wasn't really any answer to that (or rather the publication didn't take that in to account, although that can be experimentally determined). Still, a month later I saw tech explore or some tech news writing an article about it, something like "AI can now read your brain" and the 1984 implications yada yada. So this is indeed something probably most practitioners, masters and PhD, realize relatively early. So now that someone says "you know mindfulness is proven to change your brainwaves?" I always add my story "yes, but the study was done with EEG, so I don't trust the scientific backing of it" (but anecdotally, it helps me)

deleted 10mo ago
[deleted]
D-Machine 10mo ago
The difference is that EEG can be used usefully in e.g. biofeedback training and the study of sleep phases, so there is in fact enough signal here for it to be broadly useful in some simple cases. It is not clear fMRI has enough signal for anything even as simple as these things though.
- j45 10mo ago
  I have been told QEEG can offer an additional perspective in neurofeedback, etc as well. fMRI's are being used in TBI/Concussion recovery that are study backed and seem to be delivering results.
  - D-Machine 10mo ago
    Yes, there are a few medical cases where fMRI makes good simple basic sense, and TBI/Concussion sounds immediately like one of those to me. I seem also to recall them being useful in some cases prior to brain surgeries and the like. This all makes sense because fMRI tracks metabolic activity via oxygenation changes, which is much more clearly and plausibly related to tissue health and recovery. In these cases, it is also most likely being used within-subject (i.e. longitudinally) to make comparisons to baselines, rather than in an attempt to make speculative inferences about the mind using groups of people, and likely is a simple comparison to baseline rather than bespoke statistical analyses relying on questionable assumptions about the BOLD response being related to overly-specific kinds of neural activity.
    - j45 10mo ago
      fMRI can track oxygenation changes, and indirectly where the blood flow is, or isn't, and perhaps some ideas on where to get it. All to say, this application might not fall in the 40%. I just find articles like these can't help but feel like they have an agenda to undermine something instead of simply acknowledge the kinds of things it is and isn't working for. There's no doubt these researchers have found something, but the need for sensationalistic headlines is well known in academia as well. Sometimes it's noticeable where the research is specific in scope, but the findings are more general and broad.
  - hirvi74 10mo ago
    > fMRI's are being used in TBI/Concussion recovery Interesting. Do you happen to have any more information on this topic? I ask because I was under the impression that concussions are a functional/metabolic injury and not a structural injury, therefore, concussions are not visible on any type of fMRI, CT Scan, etc.. Though, I haven't looked into this topic for almost half a decade, so I imagine things have likely progressed.
    - D-Machine 10mo ago
      Well fMRI (as opposed to MRI) is used precisely because it measures things directly related to metabolism and function. Not hard to find info on this stuff: https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&q=fMRI%20concussion%20TBI https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&q=fMRI%...
jtbayly 10mo ago
But none of this (signal/noise ratio, etc) is related to the topic of the article, which claims that even with good signal, blood flow is not useful to determine brain activity.
SubiculumCode 10mo ago
There are lots of reliable science done using EEG and fMRI; I believe you learned the wrong lesson here. The important thing is to treat motion and physiological sources of noise as a first-order problem that must be taken very seriously and requires strict data quality inclusion criterion. As far as deep learning in fMRI/EEG, your response about overfitting is too sweepingly broad to apply to the entire field. To put it succinctly, I think you have overfit your conclusions on the amount of data you have seen
- D-Machine 10mo ago
  I would argue in fact almost all fMRI research is unreliable, and formally so (test-retest reliabilities are in fact quite miserable: see my post below). https://news.ycombinator.com/item?id=46289133 https://news.ycombinator.com/item?id=46289133 EDIT: The reason being, with reliabilities as bad as these, it is obvious almost all fMRI studies are massively underpowered, and you really need to have hundreds or even up to a thousand participants to detect effects with any statistical reliability. Very few fMRI studies ever have even close to these numbers (https://www.nature.com/articles/s42003-018-0073-z https://www.nature.com/articles/s42003-018-0073-z).
  - SubiculumCode 10mo ago
    Yes on many of those fronts, although not all those papers support your conclusion. The field did/does too often use tasks with to few trials, with to few participants. That always frustrated me as my advisor rightly insisted we collect hundreds of participants for each study, while others would collect 20 and publish 10x faster than us.
    - D-Machine 10mo ago
      Yes, well "almost all" is vague and needs to be qualified. Sample sizes have improved over the past decade for sure. I'm not sure if they have grown on median meaningfully, because there are still way too many low-N studies, but you do see studies now that are at least plausibly "large enough" more frequently. More open data has also helped here. EDIT: And kudos to you and your advisor here. EDIT2: I will also say that a lot of the research on fMRI methods is very solid and often quite reproducible. I.e. papers that pioneer new analytic methods and/or investigate pipelines and such. There is definitely a lot of fMRI research telling us a lot of interesting and likely reliable things about fMRI, but there is very little fMRI research that is telling us anything reliably generalizable about people or cognition.
caycep 10mo ago
There's fancier ML studies on EEG signal but probably not consistent enough for clinical work. For now, the one thing EEG can reliably tell is if you're having a seizure or not, if you're delirious (or in a coma) or not, or if you're asleep.
ErroneousBosh 10mo ago
> When I asked "but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact?" So here you say quite a mouthful. If you train it on a pattern it'll see that pattern everywhere - think about the early "Deep Dream" trippy-dogs-pictures nonsense that was pervasive about eight or nine years ago. I repaired a couple of cameras for someone who was working with a large university hospital about 15 years ago, where they were using admittedly 2010s-era "Deep Learning" to analyse biopsy scans for signs of cancer. It worked brilliantly, at least with the training materials, incredible hit rate, not too terrible false positive rate (no biggie, you're just trying to decide if you want to investigate further), really low false negative rate (if there was cancer it would spot it, for sure, and you don't want to miss that). But in real-world patient data it went completely mental. The sample data was real-world patient data, too, but on "uncontrolled" patients, it was detecting cancer all over the place. It also detected cancer in pictures of the Oncology department lino floor, it detected cancer in a picture of a guy's ID badge, it detected cancer in a closeup of my car tyre, and it detected cancer in a photo of a grey overcast sky. Aw no. Now what? Well, that's why I looked at the camera for them. They'd photographed the biopsies with one camera on site, from "real patients", but a lot of the "clear" biopsies were from other sites. You're ahead of me now, aren't you? The "Deep Learning" system had in fact trained itself on a speck of shit on the sensor of one of the cameras, the one used for most of the "has cancer" biopsies and most of the "real patient under test" biopsies. If that little blob of about a dozen slightly darker pixels was present, then it must be cancer because that's what the grown-ups told it. The actual picture content was largely irrelevant because the blob was consistent across all of them. I'm not too keen on AI in healthcare, not as a definitive "go/no-go" test thing.
Plutoberth 10mo ago
I'm not sure I understand. Wouldn't any prediction result above statistical random (in the image mind reading study) be significant? If the study was performed correctly I don't really need to know much about fMRI to tell whether it's an interesting result or not.
- ladberg 10mo ago
  The study misleading claimed to produce images from brainwaves. In reality, they effectively built a combination of classifier from brainwaves to one of a few predetermined classifications of images shown (still cool, but less impressive) and a neural net to reproduce images it was trained on given a classification (boring).
selfmodruntime 10mo ago
90% of papers I read in computer science / computer security speak of software written or AI models they trained that are nowhere to be found. Not on git nor via email to the authors.
canjobear 10mo ago
> but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact? You test your DL decoder on held-out data. This is the common practice.
aardvark92 10mo ago
Saw the same thing first hand with Pathology data. Image analysis is far more straightforward problem than fMRI, but sorry, I do not trust your AI model that matches our pathologist’s scoring with 98.5% accuracy. Our pathologists are literally guesstimating these numbers and can vary by like 10-20% just based on the phase of the moon, whether the pathologist ate lunch yet, what slides he looked at earlier that day…that’s not even accounting for inter-pathologist variation… Also saw this irl with a particular NGS diagnostic. This model was initially 99% accurate, P.I. smelled BS, had the grad student crunch the numbers again, 96% accurate, published it, built a company around this product —-> boom, 2 years later it was retracted because the data was a lot of amplified noise, spurious hits, overfitting. I don’t know jack compared to the average HN contributor, but even I can smell the BS from a mile away in some of these biomedical AI models. Peer review is broken for highly-interdisciplinary research like this.
pedalpete 10mo ago
I think you're throwing the baby out with the bathwater, while also pointing to the missing pieces in our understanding of the brain and consciousness. I also work in the field, specifically with sleep slow-wave enhancement. Blood flow as a proxy for brain activity I always felt was a weak measure, as the brain activity involved across all manner of operating our biological systems, so is the increased blood flow measured in fMRI a response to cognition, or autonomic activity? What does that oxydation mean. EEG is similarly flawed when we try to equate "brainwaves" to emotions and consciousness. I think we're almost better off measuring HRV, a much simpler measure, and more reliable. I'm fascinated that so many people who discuss brainwaves think of them as actual "waves", when it is just how we plot electrical activity which creates a visual wave like pattern. However, and this is specifically related to our work in sleep, we can detect slow-waves (I dislike that term, it's the synchronous firing of neurons) and we are able to stimulate this restorative brain function through sensory perception during sleep, and even create slow-waves in a lab using TMS. Research linked on our website [1] I agree the industry needs to stop projecting what we hope we're seeing with what is actually being measured, and we don't understand enough about how the brain works, but I think completely throwing away any brain related measures we have is going too far. 1 - https://affectablesleep.com/how-it-works#research https://affectablesleep.com/how-it-works#research

deleted 10mo ago

[deleted]

D-Machine 10mo ago

The difference is that EEG can be used usefully in e.g. biofeedback training and the study of sleep phases, so there is in fact enough signal here for it to be broadly useful in some simple cases. It is not clear fMRI has enough signal for anything even as simple as these things though.

j45 10mo ago
I have been told QEEG can offer an additional perspective in neurofeedback, etc as well. fMRI's are being used in TBI/Concussion recovery that are study backed and seem to be delivering results.
- D-Machine 10mo ago
  Yes, there are a few medical cases where fMRI makes good simple basic sense, and TBI/Concussion sounds immediately like one of those to me. I seem also to recall them being useful in some cases prior to brain surgeries and the like. This all makes sense because fMRI tracks metabolic activity via oxygenation changes, which is much more clearly and plausibly related to tissue health and recovery. In these cases, it is also most likely being used within-subject (i.e. longitudinally) to make comparisons to baselines, rather than in an attempt to make speculative inferences about the mind using groups of people, and likely is a simple comparison to baseline rather than bespoke statistical analyses relying on questionable assumptions about the BOLD response being related to overly-specific kinds of neural activity.
  - j45 10mo ago
    fMRI can track oxygenation changes, and indirectly where the blood flow is, or isn't, and perhaps some ideas on where to get it. All to say, this application might not fall in the 40%. I just find articles like these can't help but feel like they have an agenda to undermine something instead of simply acknowledge the kinds of things it is and isn't working for. There's no doubt these researchers have found something, but the need for sensationalistic headlines is well known in academia as well. Sometimes it's noticeable where the research is specific in scope, but the findings are more general and broad.
- hirvi74 10mo ago
  > fMRI's are being used in TBI/Concussion recovery Interesting. Do you happen to have any more information on this topic? I ask because I was under the impression that concussions are a functional/metabolic injury and not a structural injury, therefore, concussions are not visible on any type of fMRI, CT Scan, etc.. Though, I haven't looked into this topic for almost half a decade, so I imagine things have likely progressed.
  - D-Machine 10mo ago
    Well fMRI (as opposed to MRI) is used precisely because it measures things directly related to metabolism and function. Not hard to find info on this stuff: https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&q=fMRI%20concussion%20TBI https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&q=fMRI%...

jtbayly 10mo ago

But none of this (signal/noise ratio, etc) is related to the topic of the article, which claims that even with good signal, blood flow is not useful to determine brain activity.

SubiculumCode 10mo ago

There are lots of reliable science done using EEG and fMRI; I believe you learned the wrong lesson here. The important thing is to treat motion and physiological sources of noise as a first-order problem that must be taken very seriously and requires strict data quality inclusion criterion. As far as deep learning in fMRI/EEG, your response about overfitting is too sweepingly broad to apply to the entire field. To put it succinctly, I think you have overfit your conclusions on the amount of data you have seen

D-Machine 10mo ago
I would argue in fact almost all fMRI research is unreliable, and formally so (test-retest reliabilities are in fact quite miserable: see my post below). https://news.ycombinator.com/item?id=46289133 https://news.ycombinator.com/item?id=46289133 EDIT: The reason being, with reliabilities as bad as these, it is obvious almost all fMRI studies are massively underpowered, and you really need to have hundreds or even up to a thousand participants to detect effects with any statistical reliability. Very few fMRI studies ever have even close to these numbers (https://www.nature.com/articles/s42003-018-0073-z https://www.nature.com/articles/s42003-018-0073-z).
- SubiculumCode 10mo ago
  Yes on many of those fronts, although not all those papers support your conclusion. The field did/does too often use tasks with to few trials, with to few participants. That always frustrated me as my advisor rightly insisted we collect hundreds of participants for each study, while others would collect 20 and publish 10x faster than us.
  - D-Machine 10mo ago
    Yes, well "almost all" is vague and needs to be qualified. Sample sizes have improved over the past decade for sure. I'm not sure if they have grown on median meaningfully, because there are still way too many low-N studies, but you do see studies now that are at least plausibly "large enough" more frequently. More open data has also helped here. EDIT: And kudos to you and your advisor here. EDIT2: I will also say that a lot of the research on fMRI methods is very solid and often quite reproducible. I.e. papers that pioneer new analytic methods and/or investigate pipelines and such. There is definitely a lot of fMRI research telling us a lot of interesting and likely reliable things about fMRI, but there is very little fMRI research that is telling us anything reliably generalizable about people or cognition.

caycep 10mo ago

There's fancier ML studies on EEG signal but probably not consistent enough for clinical work. For now, the one thing EEG can reliably tell is if you're having a seizure or not, if you're delirious (or in a coma) or not, or if you're asleep.

ErroneousBosh 10mo ago

> When I asked "but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact?" So here you say quite a mouthful. If you train it on a pattern it'll see that pattern everywhere - think about the early "Deep Dream" trippy-dogs-pictures nonsense that was pervasive about eight or nine years ago. I repaired a couple of cameras for someone who was working with a large university hospital about 15 years ago, where they were using admittedly 2010s-era "Deep Learning" to analyse biopsy scans for signs of cancer. It worked brilliantly, at least with the training materials, incredible hit rate, not too terrible false positive rate (no biggie, you're just trying to decide if you want to investigate further), really low false negative rate (if there was cancer it would spot it, for sure, and you don't want to miss that). But in real-world patient data it went completely mental. The sample data was real-world patient data, too, but on "uncontrolled" patients, it was detecting cancer all over the place. It also detected cancer in pictures of the Oncology department lino floor, it detected cancer in a picture of a guy's ID badge, it detected cancer in a closeup of my car tyre, and it detected cancer in a photo of a grey overcast sky. Aw no. Now what? Well, that's why I looked at the camera for them. They'd photographed the biopsies with one camera on site, from "real patients", but a lot of the "clear" biopsies were from other sites. You're ahead of me now, aren't you? The "Deep Learning" system had in fact trained itself on a speck of shit on the sensor of one of the cameras, the one used for most of the "has cancer" biopsies and most of the "real patient under test" biopsies. If that little blob of about a dozen slightly darker pixels was present, then it must be cancer because that's what the grown-ups told it. The actual picture content was largely irrelevant because the blob was consistent across all of them. I'm not too keen on AI in healthcare, not as a definitive "go/no-go" test thing.

Plutoberth 10mo ago

I'm not sure I understand. Wouldn't any prediction result above statistical random (in the image mind reading study) be significant? If the study was performed correctly I don't really need to know much about fMRI to tell whether it's an interesting result or not.

ladberg 10mo ago
The study misleading claimed to produce images from brainwaves. In reality, they effectively built a combination of classifier from brainwaves to one of a few predetermined classifications of images shown (still cool, but less impressive) and a neural net to reproduce images it was trained on given a classification (boring).

selfmodruntime 10mo ago

90% of papers I read in computer science / computer security speak of software written or AI models they trained that are nowhere to be found. Not on git nor via email to the authors.

canjobear 10mo ago

> but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact? You test your DL decoder on held-out data. This is the common practice.

aardvark92 10mo ago

Saw the same thing first hand with Pathology data. Image analysis is far more straightforward problem than fMRI, but sorry, I do not trust your AI model that matches our pathologist’s scoring with 98.5% accuracy. Our pathologists are literally guesstimating these numbers and can vary by like 10-20% just based on the phase of the moon, whether the pathologist ate lunch yet, what slides he looked at earlier that day…that’s not even accounting for inter-pathologist variation… Also saw this irl with a particular NGS diagnostic. This model was initially 99% accurate, P.I. smelled BS, had the grad student crunch the numbers again, 96% accurate, published it, built a company around this product —-> boom, 2 years later it was retracted because the data was a lot of amplified noise, spurious hits, overfitting. I don’t know jack compared to the average HN contributor, but even I can smell the BS from a mile away in some of these biomedical AI models. Peer review is broken for highly-interdisciplinary research like this.

pedalpete 10mo ago

I think you're throwing the baby out with the bathwater, while also pointing to the missing pieces in our understanding of the brain and consciousness. I also work in the field, specifically with sleep slow-wave enhancement. Blood flow as a proxy for brain activity I always felt was a weak measure, as the brain activity involved across all manner of operating our biological systems, so is the increased blood flow measured in fMRI a response to cognition, or autonomic activity? What does that oxydation mean. EEG is similarly flawed when we try to equate "brainwaves" to emotions and consciousness. I think we're almost better off measuring HRV, a much simpler measure, and more reliable. I'm fascinated that so many people who discuss brainwaves think of them as actual "waves", when it is just how we plot electrical activity which creates a visual wave like pattern. However, and this is specifically related to our work in sleep, we can detect slow-waves (I dislike that term, it's the synchronous firing of neurons) and we are able to stimulate this restorative brain function through sensory perception during sleep, and even create slow-waves in a lab using TMS. Research linked on our website [1] I agree the industry needs to stop projecting what we hope we're seeing with what is actually being measured, and we don't understand enough about how the brain works, but I think completely throwing away any brain related measures we have is going too far. 1 - https://affectablesleep.com/how-it-works#research https://affectablesleep.com/how-it-works#research