4 ms·

I would be curious whether this is actually classifying something that typically corresponds with sex, like hormone levels. In that case people with hormonal di

I would be curious whether this is actually classifying something that typically corresponds with sex, like hormone levels. In that case people with hormonal disorders would potentially be mis-classified, and someone photographed pre/during puberty might also be mis-classified. Since the paper mentions both neural and vascular tissue being represented in retinal photos, it seems like the levels of various hormones in the individual's blood could potentially also generate a mis-classification if they (for example) cause blood vessels in the eye to expand or contract. The mention that foveal pathology causes the model to mispredict suggests it would probably have issues in these cases too, I think. I wonder what actual values they were trying to predict with this analysis? Based on the paper, I get the impression they were trying to do something more interesting and they got the best data for sex.

cassonmars 5y ago
I wonder this too, and they could get a pretty solid answer if they incorporated transgender folks’ data into the set since they’re actively keeping their hormones in the desired ranges of their gender identity.
- esyir 5y ago
  Generally you'd start with the common, easier problem before you delve into the abnormal cases. There are probably better ways to do that, like looking at bloodwork.
  - lukeschlather 5y ago
    That's fine for training, but for testing the model delving into abnormal cases seems required. If there's noticeable difference on abnormal cases that gives a lot of insight into what your model is actually testing.
    - cassonmars 5y ago
      All observations about the use of the term “abnormal” aside, I agree the idea of specifically isolating the hormone variable by using groups of people who naturally fit that range and groups of people that use medication to mirror that range would at least indicate whether this classifier is picking up on sexual dimorphism or vascular effects from hormonal differences (which also would potentially impact not only transgender people, but intersex people — who make up around 3% of the population)
spuz 5y ago
They were specifically trying to classify sex because it is something that experts cannot already do: > While our deep learning model was specifcally designed for the task of sex prediction, we emphasize that this task has no inherent clinical utility. Instead, we aimed to demonstrate that AutoML could classify these images independent of salient retinal features being known to domain experts, that is, retina specialists cannot readily perform this task.

cassonmars 5y ago

I wonder this too, and they could get a pretty solid answer if they incorporated transgender folks’ data into the set since they’re actively keeping their hormones in the desired ranges of their gender identity.

esyir 5y ago
Generally you'd start with the common, easier problem before you delve into the abnormal cases. There are probably better ways to do that, like looking at bloodwork.
- lukeschlather 5y ago
  That's fine for training, but for testing the model delving into abnormal cases seems required. If there's noticeable difference on abnormal cases that gives a lot of insight into what your model is actually testing.
  - cassonmars 5y ago
    All observations about the use of the term “abnormal” aside, I agree the idea of specifically isolating the hormone variable by using groups of people who naturally fit that range and groups of people that use medication to mirror that range would at least indicate whether this classifier is picking up on sexual dimorphism or vascular effects from hormonal differences (which also would potentially impact not only transgender people, but intersex people — who make up around 3% of the population)

spuz 5y ago

They were specifically trying to classify sex because it is something that experts cannot already do: > While our deep learning model was specifcally designed for the task of sex prediction, we emphasize that this task has no inherent clinical utility. Instead, we aimed to demonstrate that AutoML could classify these images independent of salient retinal features being known to domain experts, that is, retina specialists cannot readily perform this task.