10 ms·

New AI System Predicts Seizures

71 points by riffraff 7y ago

brenden2 7y ago
The sample size is 22 patients. Also, I've noticed the definition of the word "AI" has grown to encompass pretty much any type of software that does something with data.
- leoh 7y ago
  I think it's ok to call this AI, but I agree that it's a little misleading. I think a better way to describe much of AI these days is pattern recognition. Incredible pattern recognition, to be clear, but pattern recognition. Agreed about the sample size. It's not clear from the article whether the high accuracy was based on testing against the training set, too. I assume not, as that would be exceptionally sloppy. But it's not clear.
- denhaus 7y ago
  Good catch, I don’t think they mention that in the main article (at least in reader view). In this case I guess the “AI” label is relatively well applied (compared to the scope of ALL things I’ve seen it haphazardly slapped on), but yeah, still - people need to stop calling any kind of data analysis “AI”.
- etbebl 7y ago
  What, do you think that's too low? Sounds pretty normal for this type of study. Of course, that does mean the results don't necessarily generalize to the whole population, which is why you do eventually need much larger sample sizes when you're working toward FDA approval.
  - brenden2 7y ago
    It seems a little premature to proclaim "Near-Perfect Accuracy" on such a small sample size. It sounds to me like they may have overfit their model.
  - ska 7y ago
    > don't necessarily generalize to the whole population In my experience it is far more likely to mean "doesn't generalize well, and moving to a reasonable training set will reveal flaws in the model.
- Scea91 7y ago
  Also accuracy is not a good measure for evaluating detection of rare events such as seizures. (https://arxiv.org/pdf/1812.01388.pdf https://arxiv.org/pdf/1812.01388.pdf)
- freeflight 7y ago
  > Also, I've noticed the definition of the word "AI" has grown to encompass pretty much any type of software that does something with data. Afaik AGI research is still a separate thing, so it's not really that misleading to use "AI" for applications like this were machine learning is involved, resulting in a very specialized "artificial intelligence" that can spot otherwise hidden patterns.
- hodgesrm 7y ago
  > Also, I've noticed the definition of the word "AI" has grown to encompass pretty much any type of software that does something with data. That's definitely the case in conferences like O'Reilly Strata. They seem to be 100% focused on AI/ML, which seems really limiting. I'm starting to skip attendance until things get a bit more balanced.
leoh 7y ago
Some questions not addressed by the article: * what is the input to the model? It sounds like it's EEG, but how practical is it to collect those data on a day-to-day basis? * how far ahead of time are seizures predicted? In other words -- how practical is this research?
- tuukkah 7y ago
  "predict the occurrence of seizures up to one hour before onset" "We are currently working on the design of an efficient hardware [device] that deploys this algorithm, considering many issues like system size, power consumption, and latency to be suitable for practical application in a comfortable way to the patient"
siftrics 7y ago
"A false positive rate of 0.004 per hour" is a sly way of putting "a false positive once every 10 days."
- freeflight 7y ago
  That's 3 false positives in a month, depending on how many seizures a patient suffers from, this might actually not that bad. Afaik a false positive doesn't have any direct consequences except "be careful and keep medication ready"?
sanchezdev 7y ago
Time will tell, but as a machine learning engineer, when you see results this good it's more probable that a mistake was made. They could be reporting the training error on an overfit model or data leakage could be occuring due to an improper train-test spilt of the data. Also, it is definitely appropriate to use the term AI in this case. AI is not a technical term so it's really in the eye of the beholder, but I think it's safe to say that ML is a subset of AI. Perhaps people are conflating AI with AGI?
- hwbehrens 7y ago
  It's a 14-person training set with an 8-person test set, so my guess is that it can pretty accurately predict seizures in the small group of people it is trained on. Whether the model could be kernelized for a useful general deployment is unclear. It still requires many electrodes attached to the scalp, so there is a still a ways to go before it can be integrated into a watch, for example.
  - gnarcoregrizz 7y ago
    I assume it’ll end up in an implant like RNS
    - est31 7y ago
      Yeah watches can't even detect whether you are sleeping or not. The products in the market are mainly accelerometer based and aren't really reliable if you e.g. are awake but don't move.
  - m0zg 7y ago
    They acknowledge this in the article - the system will have to be transfer learned for every patient. Which IMO is ideal, but you will need training/validation data, which in this case sounds like it'd be extremely expensive. Furthermore, the system could get automatically better over time _for that patient_, if properly designed and fed clean training samples.
- ska 7y ago
moioci 7y ago
Work proceeds on natural intelligence system for predicting seizures: https://www.nature.com/articles/s41598-019-40721-4 https://www.nature.com/articles/s41598-019-40721-4
matthewdgreen 7y ago
How does this become a treatment, assuming the results actually hold up? Is it possible to prevent seizures with a timely dose of a powerful epilepsy medication? Are there portable EEG rigs that can produce sufficiently powerful readings?
- tuukkah 7y ago
  At least it lets you get into a safe place and position. Perhaps you can even get someone to watch over and be ready to call 911 if it comes to that. Edit: Perhaps this hints at some medication that is not currently viable but could be administered if the seisure was predicted: "Notably, seizures are controllable with medication in up to 70 percent of these patients."
  - yomly 7y ago
    Correct - a friend had a seizure on the stairs and was lucky to get away with just a cut lip...
- gnarcoregrizz 7y ago
  I assume it’d be used in an implant like the RNS, which is used to treat (mostly) focal seizures. It’s a small device implanted in your skull to record localized EEG and provide neurostimulation to quell burgeoning seizures. Currently, it’s periodically adjusted and individualized by neurologists based on the the EEG it records. Maybe this could speed up or automate that process.
- petra 7y ago
  It's possible to make anti epileptic drugs work rapidly, some in 30 minutes - but it requires an injection.
black_puppydog 7y ago
Actual paper, linked from the article: https://ieeexplore.ieee.org/document/8765420 https://ieeexplore.ieee.org/document/8765420 "Efficient Epileptic Seizure Prediction Based on Deep Learning"
peter303 7y ago
Its interesting that service animals like dogs or monkeys can sometimes be trained to anticipate seizures. And in many of these cases its not clear what the signal is- a change in activity, odor etc.

71 points by riffraff 7y ago

The sample size is 22 patients. Also, I've noticed the definition of the word "AI" has grown to encompass pretty much any type of software that does something with data.

leoh 7y ago
I think it's ok to call this AI, but I agree that it's a little misleading. I think a better way to describe much of AI these days is pattern recognition. Incredible pattern recognition, to be clear, but pattern recognition. Agreed about the sample size. It's not clear from the article whether the high accuracy was based on testing against the training set, too. I assume not, as that would be exceptionally sloppy. But it's not clear.
denhaus 7y ago
Good catch, I don’t think they mention that in the main article (at least in reader view). In this case I guess the “AI” label is relatively well applied (compared to the scope of ALL things I’ve seen it haphazardly slapped on), but yeah, still - people need to stop calling any kind of data analysis “AI”.
etbebl 7y ago
What, do you think that's too low? Sounds pretty normal for this type of study. Of course, that does mean the results don't necessarily generalize to the whole population, which is why you do eventually need much larger sample sizes when you're working toward FDA approval.
- brenden2 7y ago
  It seems a little premature to proclaim "Near-Perfect Accuracy" on such a small sample size. It sounds to me like they may have overfit their model.
- ska 7y ago
  > don't necessarily generalize to the whole population In my experience it is far more likely to mean "doesn't generalize well, and moving to a reasonable training set will reveal flaws in the model.
Scea91 7y ago
Also accuracy is not a good measure for evaluating detection of rare events such as seizures. (https://arxiv.org/pdf/1812.01388.pdf https://arxiv.org/pdf/1812.01388.pdf)
freeflight 7y ago
> Also, I've noticed the definition of the word "AI" has grown to encompass pretty much any type of software that does something with data. Afaik AGI research is still a separate thing, so it's not really that misleading to use "AI" for applications like this were machine learning is involved, resulting in a very specialized "artificial intelligence" that can spot otherwise hidden patterns.
hodgesrm 7y ago
> Also, I've noticed the definition of the word "AI" has grown to encompass pretty much any type of software that does something with data. That's definitely the case in conferences like O'Reilly Strata. They seem to be 100% focused on AI/ML, which seems really limiting. I'm starting to skip attendance until things get a bit more balanced.

leoh 7y ago

Some questions not addressed by the article: * what is the input to the model? It sounds like it's EEG, but how practical is it to collect those data on a day-to-day basis? * how far ahead of time are seizures predicted? In other words -- how practical is this research?

tuukkah 7y ago
"predict the occurrence of seizures up to one hour before onset" "We are currently working on the design of an efficient hardware [device] that deploys this algorithm, considering many issues like system size, power consumption, and latency to be suitable for practical application in a comfortable way to the patient"

siftrics 7y ago

"A false positive rate of 0.004 per hour" is a sly way of putting "a false positive once every 10 days."

freeflight 7y ago
That's 3 false positives in a month, depending on how many seizures a patient suffers from, this might actually not that bad. Afaik a false positive doesn't have any direct consequences except "be careful and keep medication ready"?

sanchezdev 7y ago

Time will tell, but as a machine learning engineer, when you see results this good it's more probable that a mistake was made. They could be reporting the training error on an overfit model or data leakage could be occuring due to an improper train-test spilt of the data. Also, it is definitely appropriate to use the term AI in this case. AI is not a technical term so it's really in the eye of the beholder, but I think it's safe to say that ML is a subset of AI. Perhaps people are conflating AI with AGI?

hwbehrens 7y ago
It's a 14-person training set with an 8-person test set, so my guess is that it can pretty accurately predict seizures in the small group of people it is trained on. Whether the model could be kernelized for a useful general deployment is unclear. It still requires many electrodes attached to the scalp, so there is a still a ways to go before it can be integrated into a watch, for example.
- gnarcoregrizz 7y ago
  I assume it’ll end up in an implant like RNS
  - est31 7y ago
    Yeah watches can't even detect whether you are sleeping or not. The products in the market are mainly accelerometer based and aren't really reliable if you e.g. are awake but don't move.
- m0zg 7y ago
  They acknowledge this in the article - the system will have to be transfer learned for every patient. Which IMO is ideal, but you will need training/validation data, which in this case sounds like it'd be extremely expensive. Furthermore, the system could get automatically better over time _for that patient_, if properly designed and fed clean training samples.
ska 7y ago

moioci 7y ago

Work proceeds on natural intelligence system for predicting seizures: https://www.nature.com/articles/s41598-019-40721-4 https://www.nature.com/articles/s41598-019-40721-4

matthewdgreen 7y ago

How does this become a treatment, assuming the results actually hold up? Is it possible to prevent seizures with a timely dose of a powerful epilepsy medication? Are there portable EEG rigs that can produce sufficiently powerful readings?

tuukkah 7y ago
At least it lets you get into a safe place and position. Perhaps you can even get someone to watch over and be ready to call 911 if it comes to that. Edit: Perhaps this hints at some medication that is not currently viable but could be administered if the seisure was predicted: "Notably, seizures are controllable with medication in up to 70 percent of these patients."
- yomly 7y ago
  Correct - a friend had a seizure on the stairs and was lucky to get away with just a cut lip...
gnarcoregrizz 7y ago
I assume it’d be used in an implant like the RNS, which is used to treat (mostly) focal seizures. It’s a small device implanted in your skull to record localized EEG and provide neurostimulation to quell burgeoning seizures. Currently, it’s periodically adjusted and individualized by neurologists based on the the EEG it records. Maybe this could speed up or automate that process.
petra 7y ago
It's possible to make anti epileptic drugs work rapidly, some in 30 minutes - but it requires an injection.

black_puppydog 7y ago

Actual paper, linked from the article: https://ieeexplore.ieee.org/document/8765420 https://ieeexplore.ieee.org/document/8765420 "Efficient Epileptic Seizure Prediction Based on Deep Learning"

peter303 7y ago

Its interesting that service animals like dogs or monkeys can sometimes be trained to anticipate seizures. And in many of these cases its not clear what the signal is- a change in activity, odor etc.