6 ms·

Apple is researching ways to identify signs of childhood autism through iPhones

31 points by MihikaBasu 5y ago

CivBase 5y ago
Here's the WSJ article mentioned in the first paragraph, although it is pay-walled: https://www.wsj.com/articles/apple-wants-iphones-to-help-detect-depression-cognitive-decline-sources-say-11632216601?mod=hp_lead_pos3 https://www.wsj.com/articles/apple-wants-iphones-to-help-det...
- Zak 5y ago
  Apple Is Working on iPhone Features to Help Detect Depression, Cognitive Decline Company is working with UCLA, Biogen to see if sensitive data like facial expressions, typing metrics could signal mental-health concerns Apple Inc. is working on technology to help diagnose depression and cognitive decline, aiming for tools that could expand the scope of its burgeoning health portfolio, according to people familiar with the matter and documents reviewed by The Wall Street Journal. Using an array of sensor data that includes mobility, physical activity, sleep patterns, typing behavior and more, researchers hope they can tease out digital signals associated with the target conditions so that algorithms can be created to detect them reliably, the people said. Apple hopes that would become the basis for unique features for its devices, according to the people and documents. The efforts spring from research partnerships that Apple has announced with the University of California, Los Angeles, which is studying stress, anxiety and depression, and pharmaceutical company Biogen Inc., BIIB -0.92% which is studying mild cognitive impairment. “Seabreeze” is Apple’s code name for the UCLA project and “Pi” is the code name for the Biogen project, according to the people and documents. Biogen launched its study on Monday, but many of the details about Apple’s studies haven’t been reported. The pharmaceutical company received approval this summer from the U.S. Food and Drug Administration for a new drug to treat mild cognitive impairment. Representatives for Apple, Biogen and UCLA declined to comment. Much of Apple’s past health work has focused on features for its smartwatch. The mental-health and cognitive-decline research leverages more-sensitive iPhone data and shows Apple’s health unit is now looking at features for the company’s flagship product, according to people familiar with the plans. The research projects are still at early stages, and may never lead to new device features, the people said. While prior academic studies have shown some evidence that people with certain mental-health conditions use their digital devices differently than others, it remains to be seen if reliable algorithms can be created to detect the conditions, according to researchers. Apple also has a research project with Duke University that aims to create an algorithm to detect childhood autism, including to use the iPhone’s camera to observe how young children focus, according to the documents and people familiar with the work. The promise of detection tools for people with severe neurological or mood disorders is that you could intervene early and potentially prevent worse outcomes, said Faraz Hussain, the lead developer at BiAffect, an academic research project at the University of Illinois Chicago that uses typing data from smartphones to predict moods. “It’s the ability to peek inside how our mind is functioning instead of relying on self-reports that are often subjective, using digital exhaust from our daily lives that would otherwise be lost.” Even though the effort is at an early stage, top Apple executives are excited about the possibility. Chief Operating Officer Jeff Williams, who oversees Apple’s health unit, has spoken enthusiastically to employees about the company’s potential to address surging rates of depression and anxiety as well as other brain disorders, according to people who have heard him talk about the efforts. If they are successful, Apple and its partners could improve the detection of the conditions, which affect tens of millions of people world-wide. But the extent of user tracking that may be required could spark privacy concerns. To address them, Apple aims for algorithms that work on users’ devices and don’t send the data to Apple servers, the documents show. Much of Apple’s past health work has focused on features for its smartwatch. Achieving its health ambitions could depend on consumers’ trusting Apple with sensitive data, underscoring why privacy is a business imperative for the company. The new research comes as Apple faces intense scrutiny over how it planned to access user data to warn authorities of child pornography, which raised new concerns about the company’s commitment to privacy after years of battles against governments seeking to access their customers’ devices. The company has said privacy remains paramount, arguing the anti-child-pornography initiative is meant to guard against intrusion while also combating the exploitation of children. Apple has said it is slowing the effort. Scores of companies are chasing the opportunity in the broader digital behavioral-health sector, where startups are on pace this year to nearly double the record $2.7 billion in financing raised in 2020, according to Rock Health, a venture-investment and advisory firm. The pandemic drove an increase in mental-health-related complaints. The percentage of adults reporting anxiety or depression-related symptoms reached 41% in January, according to the Kaiser Family Foundation, nearly quadruple the early-2019 figure. Mild cognitive impairment, which can develop into dementia, affects around 5 million Americans over 60 years old, estimates the Alzheimer’s Association. Apple has developed previous health features after working with researchers. For example, work at Stanford University showed that its smartwatch could identify an irregular heart rhythm known as atrial fibrillation. Soon after Apple added a watch feature to do so. UCLA has said its research studying signs of stress, anxiety and depression began with a pilot phase last fall tracking Apple Watch and iPhone data from 150 people and will continue with a main phase tracking similar data for 3,000 people starting this year. UCLA researchers will track data from the iPhone’s video camera, keyboard and audio sensors, and data from the watch related to movement, vital signs and sleep, according to the documents and people familiar with the study. The data that may be used includes analysis of participants’ facial expressions, how they speak, the pace and frequency of their walks, sleep patterns, and heart and respiration rates. They may also measure the speed of their typing, frequency of their typos and content of what they type, among other data points, according to the people familiar with the research and the documents. Each bit of data could give researchers clues about device users’ emotions, concentration, energy level, state of mind and more, according to the people and the documents. To compare this data with other measures of stress, anxiety or depression, the researchers are having participants complete questionnaires about how they feel, according to people familiar with the work and the documents. They are also looking at the amount of the stress hormone cortisol in follicles of participants’ hair, according to the documents and one of these people. If the research finds that any of that data correlates with relevant mental-health conditions, the hope is to turn those signals into an app or feature that could warn people they might be at risk and prompt them to seek care, according to the documents and people familiar with the work. Biogen and Apple said in January they are collaborating on a study to use the iPhone and Apple Watch to track cognitive function over time and identify mild cognitive impairment, a condition that can develop into Alzheimer’s. The two-year study aims to follow about 20,000 participants—half of them at high risk of cognitive impairment—and will use device data in a way similar to the UCLA mental-health research, according to the documents and people familiar with the matter. The work follows a 2019 feasibility study that showed that 31 adults with cognitive impairment exhibited different behavior on their Apple devices than healthy older adults. Biogen is collaborating on the study because it hopes it can help Apple develop an iPhone feature to detect mild cognitive impairment early and encourage relevant users to seek care earlier, according to a person familiar with the matter. The company will compare the data against standard tests of brain health including traditional cognitive assessments and scans that track plaque buildup in the brain, according to the documents and the person familiar with the work. Write to Rolfe Winkler at rolfe.winkler@wsj.com
  - macintux 5y ago
    I somehow doubt HN wants to be in the copyright violation business.
    - pomian 5y ago
      Although it's nice to be be able to read the article. Maybe one could quote a few highlights, as a usable alternative to archive link etc.
    - Zak 5y ago
      Commenting with the text of paywalled articles has been common practice here for years.
macintux 5y ago
A remarkably cynical and negative take, but at least the headline was sanitized here on HN. Tying this to the CSAM misfire is quite a stretch.
- azinman2 5y ago
  The skepticism is not well motivated when it’s clearly a health feature. There is far more valuable information already on phones without everyone stopping using them as is. It feels like sensationalism designed for clicks, which seems to be the Achilles heel in today’s society.
mistrial9 5y ago
enabling creativity, Apple's original value proposition, comes full circle to relentless categorization and surveillance, the end-game of markets
bastawhiz 5y ago
I don't see anything here about productizing the research they're doing. Nor is there any indication of how it actually works (on-device or other). There's also a lot of words about medical data and monitoring children, but it would seem wild to me that Apple somehow gets classified as a healthcare provider based on their current offering. It would seem that the technology mentioned looks at videos and makes a determination based on that. Does analyzing a video make it medical data? What about the other "medical data" that Apple has? Besides complying with COPPA, what data do they have that's not explicitly opted into? I simply can't see past the alarmist rhetoric of this post. Especially when the cookie banner doesn't give an option to opt-out, with the option "Ahh, that's alright". Only a little ironic.
sumthinprofound 5y ago
From the article: > Apple has partnered with Duke University to research the third brain using advanced medical technology powered by Artificial Intelligence and Machine Learning based on people and documents. What? From the WSJ article > Apple has a third brain-related research partnership with Duke University that it hasn’t disclosed. That makes more sense.
- gliptic 5y ago
  Did they misread "third (brain-related research partnership)" as "(third brain)-related research partnership". That's funny.
  - sumthinprofound 5y ago
    I believe so.
- MBCook 5y ago
  Honestly this is a pretty poor article that keeps trying to push click-bait links with the phone scanning dust up.
- jaredsohn 5y ago
  Amusingly after a Google search, I see that 'third brain' is a thing (a paper gave that name to the epidermis, another gave it to the placenta, is a Spy Kids thing, is a consulting company, is in the name of a book.)
g42gregory 5y ago
For me, Apple lost the trust. So yes, I don’t not give Apple a benefit of the doubt on this one. The most generous explanation I could come up with is that this is an effort to introduce “good benefits” that could come from the surveillance. That’s of course my take on Apple corporate. I am sure the team within Apple, who is working on this, means well and kudos to them.
deleted 5y ago
[deleted]
breakfastduck 5y ago
Apple is researching ways their health features and machine learning implementations can detect health conditions. Who would have thought.

31 points by MihikaBasu 5y ago

Here's the WSJ article mentioned in the first paragraph, although it is pay-walled: https://www.wsj.com/articles/apple-wants-iphones-to-help-detect-depression-cognitive-decline-sources-say-11632216601?mod=hp_lead_pos3 https://www.wsj.com/articles/apple-wants-iphones-to-help-det...

Zak 5y ago
Apple Is Working on iPhone Features to Help Detect Depression, Cognitive Decline Company is working with UCLA, Biogen to see if sensitive data like facial expressions, typing metrics could signal mental-health concerns Apple Inc. is working on technology to help diagnose depression and cognitive decline, aiming for tools that could expand the scope of its burgeoning health portfolio, according to people familiar with the matter and documents reviewed by The Wall Street Journal. Using an array of sensor data that includes mobility, physical activity, sleep patterns, typing behavior and more, researchers hope they can tease out digital signals associated with the target conditions so that algorithms can be created to detect them reliably, the people said. Apple hopes that would become the basis for unique features for its devices, according to the people and documents. The efforts spring from research partnerships that Apple has announced with the University of California, Los Angeles, which is studying stress, anxiety and depression, and pharmaceutical company Biogen Inc., BIIB -0.92% which is studying mild cognitive impairment. “Seabreeze” is Apple’s code name for the UCLA project and “Pi” is the code name for the Biogen project, according to the people and documents. Biogen launched its study on Monday, but many of the details about Apple’s studies haven’t been reported. The pharmaceutical company received approval this summer from the U.S. Food and Drug Administration for a new drug to treat mild cognitive impairment. Representatives for Apple, Biogen and UCLA declined to comment. Much of Apple’s past health work has focused on features for its smartwatch. The mental-health and cognitive-decline research leverages more-sensitive iPhone data and shows Apple’s health unit is now looking at features for the company’s flagship product, according to people familiar with the plans. The research projects are still at early stages, and may never lead to new device features, the people said. While prior academic studies have shown some evidence that people with certain mental-health conditions use their digital devices differently than others, it remains to be seen if reliable algorithms can be created to detect the conditions, according to researchers. Apple also has a research project with Duke University that aims to create an algorithm to detect childhood autism, including to use the iPhone’s camera to observe how young children focus, according to the documents and people familiar with the work. The promise of detection tools for people with severe neurological or mood disorders is that you could intervene early and potentially prevent worse outcomes, said Faraz Hussain, the lead developer at BiAffect, an academic research project at the University of Illinois Chicago that uses typing data from smartphones to predict moods. “It’s the ability to peek inside how our mind is functioning instead of relying on self-reports that are often subjective, using digital exhaust from our daily lives that would otherwise be lost.” Even though the effort is at an early stage, top Apple executives are excited about the possibility. Chief Operating Officer Jeff Williams, who oversees Apple’s health unit, has spoken enthusiastically to employees about the company’s potential to address surging rates of depression and anxiety as well as other brain disorders, according to people who have heard him talk about the efforts. If they are successful, Apple and its partners could improve the detection of the conditions, which affect tens of millions of people world-wide. But the extent of user tracking that may be required could spark privacy concerns. To address them, Apple aims for algorithms that work on users’ devices and don’t send the data to Apple servers, the documents show. Much of Apple’s past health work has focused on features for its smartwatch. Achieving its health ambitions could depend on consumers’ trusting Apple with sensitive data, underscoring why privacy is a business imperative for the company. The new research comes as Apple faces intense scrutiny over how it planned to access user data to warn authorities of child pornography, which raised new concerns about the company’s commitment to privacy after years of battles against governments seeking to access their customers’ devices. The company has said privacy remains paramount, arguing the anti-child-pornography initiative is meant to guard against intrusion while also combating the exploitation of children. Apple has said it is slowing the effort. Scores of companies are chasing the opportunity in the broader digital behavioral-health sector, where startups are on pace this year to nearly double the record $2.7 billion in financing raised in 2020, according to Rock Health, a venture-investment and advisory firm. The pandemic drove an increase in mental-health-related complaints. The percentage of adults reporting anxiety or depression-related symptoms reached 41% in January, according to the Kaiser Family Foundation, nearly quadruple the early-2019 figure. Mild cognitive impairment, which can develop into dementia, affects around 5 million Americans over 60 years old, estimates the Alzheimer’s Association. Apple has developed previous health features after working with researchers. For example, work at Stanford University showed that its smartwatch could identify an irregular heart rhythm known as atrial fibrillation. Soon after Apple added a watch feature to do so. UCLA has said its research studying signs of stress, anxiety and depression began with a pilot phase last fall tracking Apple Watch and iPhone data from 150 people and will continue with a main phase tracking similar data for 3,000 people starting this year. UCLA researchers will track data from the iPhone’s video camera, keyboard and audio sensors, and data from the watch related to movement, vital signs and sleep, according to the documents and people familiar with the study. The data that may be used includes analysis of participants’ facial expressions, how they speak, the pace and frequency of their walks, sleep patterns, and heart and respiration rates. They may also measure the speed of their typing, frequency of their typos and content of what they type, among other data points, according to the people familiar with the research and the documents. Each bit of data could give researchers clues about device users’ emotions, concentration, energy level, state of mind and more, according to the people and the documents. To compare this data with other measures of stress, anxiety or depression, the researchers are having participants complete questionnaires about how they feel, according to people familiar with the work and the documents. They are also looking at the amount of the stress hormone cortisol in follicles of participants’ hair, according to the documents and one of these people. If the research finds that any of that data correlates with relevant mental-health conditions, the hope is to turn those signals into an app or feature that could warn people they might be at risk and prompt them to seek care, according to the documents and people familiar with the work. Biogen and Apple said in January they are collaborating on a study to use the iPhone and Apple Watch to track cognitive function over time and identify mild cognitive impairment, a condition that can develop into Alzheimer’s. The two-year study aims to follow about 20,000 participants—half of them at high risk of cognitive impairment—and will use device data in a way similar to the UCLA mental-health research, according to the documents and people familiar with the matter. The work follows a 2019 feasibility study that showed that 31 adults with cognitive impairment exhibited different behavior on their Apple devices than healthy older adults. Biogen is collaborating on the study because it hopes it can help Apple develop an iPhone feature to detect mild cognitive impairment early and encourage relevant users to seek care earlier, according to a person familiar with the matter. The company will compare the data against standard tests of brain health including traditional cognitive assessments and scans that track plaque buildup in the brain, according to the documents and the person familiar with the work. Write to Rolfe Winkler at rolfe.winkler@wsj.com
- macintux 5y ago
  I somehow doubt HN wants to be in the copyright violation business.
  - pomian 5y ago
    Although it's nice to be be able to read the article. Maybe one could quote a few highlights, as a usable alternative to archive link etc.
  - Zak 5y ago
    Commenting with the text of paywalled articles has been common practice here for years.

macintux 5y ago

A remarkably cynical and negative take, but at least the headline was sanitized here on HN. Tying this to the CSAM misfire is quite a stretch.

azinman2 5y ago
The skepticism is not well motivated when it’s clearly a health feature. There is far more valuable information already on phones without everyone stopping using them as is. It feels like sensationalism designed for clicks, which seems to be the Achilles heel in today’s society.

mistrial9 5y ago

enabling creativity, Apple's original value proposition, comes full circle to relentless categorization and surveillance, the end-game of markets

bastawhiz 5y ago

I don't see anything here about productizing the research they're doing. Nor is there any indication of how it actually works (on-device or other). There's also a lot of words about medical data and monitoring children, but it would seem wild to me that Apple somehow gets classified as a healthcare provider based on their current offering. It would seem that the technology mentioned looks at videos and makes a determination based on that. Does analyzing a video make it medical data? What about the other "medical data" that Apple has? Besides complying with COPPA, what data do they have that's not explicitly opted into? I simply can't see past the alarmist rhetoric of this post. Especially when the cookie banner doesn't give an option to opt-out, with the option "Ahh, that's alright". Only a little ironic.

sumthinprofound 5y ago

From the article: > Apple has partnered with Duke University to research the third brain using advanced medical technology powered by Artificial Intelligence and Machine Learning based on people and documents. What? From the WSJ article > Apple has a third brain-related research partnership with Duke University that it hasn’t disclosed. That makes more sense.

gliptic 5y ago
Did they misread "third (brain-related research partnership)" as "(third brain)-related research partnership". That's funny.
- sumthinprofound 5y ago
  I believe so.
MBCook 5y ago
Honestly this is a pretty poor article that keeps trying to push click-bait links with the phone scanning dust up.
jaredsohn 5y ago
Amusingly after a Google search, I see that 'third brain' is a thing (a paper gave that name to the epidermis, another gave it to the placenta, is a Spy Kids thing, is a consulting company, is in the name of a book.)

g42gregory 5y ago

For me, Apple lost the trust. So yes, I don’t not give Apple a benefit of the doubt on this one. The most generous explanation I could come up with is that this is an effort to introduce “good benefits” that could come from the surveillance. That’s of course my take on Apple corporate. I am sure the team within Apple, who is working on this, means well and kudos to them.

deleted 5y ago

[deleted]

breakfastduck 5y ago

Apple is researching ways their health features and machine learning implementations can detect health conditions. Who would have thought.