3 ms·

This is a really impressive result and an interesting result to apply ML to. Thank you for sharing, OP. I'm just wondering if there are any real world applicati

This is a really impressive result and an interesting result to apply ML to. Thank you for sharing, OP. I'm just wondering if there are any real world applications of why you'd want to tell the sex of a person by a retinal photograph? It seems like a bit of a useless skill to have?

qayxc 5y ago
I think this more an example how black-box models are basically useless for clinical research. The authors aren't aware of any distinguishing retinal features between male and female eyes and the model itself has no explanatory power. Could be a Clever Hans situation where the model exploits meta information of some kind in the absence of actual features. It could just as well mean that there are indeed distinguishing features that are compromised in the presence of foveal pathology. The authors note that another study using manually selected features identified three features that are indicative of genetical sex. These features yielded about 0.78 AUROC accuracy measure. Compared to the presented model's AUROC accuracy of 0.93 that's only 19% worse and these 19% additional accuracy may point to a combination of the already identified features or one or more additional features. I personally find this paper rather pointless. It stops at the point where actual progress could be made and things would get interesting - why didn't the authors evaluate the previously known features on the model's matches to measure their significance? This could have told them whether their black-box was relying on the same set of features as the ones identified by previous work, for example.
- Blikkentrekker 5y ago
  > I think this more an example how black-box models are basically useless for clinical research. A result different from the null hypothesis is useles. Let us say the machine could not succeed greater than chance, it would be a case of cosmic bad luck in that case, that all social and biological factors cancel each other out. Thus, in the case of the null hypothesis being confirmed by this, one may conclude that in all likelihood retinal patterns have no sexual divergence. But, machines very rarely find such a null hypothesis in such cases, and that might be in no small part because of all the extra factors that such models latch on to. I for one would be more interested in an obvious nonce test to see if the machine can find something: see if the a.i. can find a retinal difference between, say, the poor and the rich. If it can with high accuracy what retinae are poor, and what are rich, we might have a somewhat interesting situation.

qayxc 5y ago

I think this more an example how black-box models are basically useless for clinical research. The authors aren't aware of any distinguishing retinal features between male and female eyes and the model itself has no explanatory power. Could be a Clever Hans situation where the model exploits meta information of some kind in the absence of actual features. It could just as well mean that there are indeed distinguishing features that are compromised in the presence of foveal pathology. The authors note that another study using manually selected features identified three features that are indicative of genetical sex. These features yielded about 0.78 AUROC accuracy measure. Compared to the presented model's AUROC accuracy of 0.93 that's only 19% worse and these 19% additional accuracy may point to a combination of the already identified features or one or more additional features. I personally find this paper rather pointless. It stops at the point where actual progress could be made and things would get interesting - why didn't the authors evaluate the previously known features on the model's matches to measure their significance? This could have told them whether their black-box was relying on the same set of features as the ones identified by previous work, for example.

Blikkentrekker 5y ago
> I think this more an example how black-box models are basically useless for clinical research. A result different from the null hypothesis is useles. Let us say the machine could not succeed greater than chance, it would be a case of cosmic bad luck in that case, that all social and biological factors cancel each other out. Thus, in the case of the null hypothesis being confirmed by this, one may conclude that in all likelihood retinal patterns have no sexual divergence. But, machines very rarely find such a null hypothesis in such cases, and that might be in no small part because of all the extra factors that such models latch on to. I for one would be more interested in an obvious nonce test to see if the machine can find something: see if the a.i. can find a retinal difference between, say, the poor and the rich. If it can with high accuracy what retinae are poor, and what are rich, we might have a somewhat interesting situation.