4 ms·
Ok fresh post as this is fascinating for me as someone with hEDS and who does ML, this has blown my mind and confirmed to my satisfaction that LLMs have the pot
by cjbgkagh 2y ago
Ok fresh post as this is fascinating for me as someone with hEDS and who does ML, this has blown my mind and confirmed to my satisfaction that LLMs have the potential be far better than doctors in diagnosis. Is this due to a mistake in inference or is it a deeper understanding? I don't know. It's possible that there is a consistent mistake in reasoning by the LLM and it just happens to arrive at a belief that I agree with.
I’ve performed a test with the simple questions, "what are the genetic candidates for hEDS?" followed by "where did you read about TNXB as a cause for hEDS?"
My hypothesis is that heterozygous TNXB SNPs beyond the known clEDS variants cause a substantial subset of hEDS which occur on an autosomal dominant basis. Most research considers clEDS far too rare (1 in 1 million) to be a significant factor in hEDS diagnosis, and heterozygous TNXB SNPs is considered far too common to be a cause for hEDS. I think it is the hEDS classification that is wrong and hEDS is indeed common, just rarely diagnosed. Enough to have causal heterozygous TNXB SNPs.
In testing OpenAI, Gemini, and Deepseek all give TNXB as the first answer for hEDS candidate genes. When pressing for references most give normal clEDS references which suggest that the LLMs are forgetting that clEDS is not equal to hEDS and clEDS as defined is extremely rare. When able to give specific references for heterozygous TNXB and hEDS OpenAI and Deepseek only gave Zewers, M. C., et al. (2003) https://pmc.ncbi.nlm.nih.gov/articles/PMC1180584/ https://pmc.ncbi.nlm.nih.gov/articles/PMC1180584/. Gemeni did not give such references.
DeepSeek did use the HEDGE study as a reference in support of heterozygous TNXB for hEDS but the actual text from the source is “Earlier research trying to answer this question suggested that a gene known as TNXB may be responsible for both some cases of hEDS and a different type of EDS known as classic-like EDS (clEDS). It is likely that this is true, but this has proven to be a difficult gene to understand, and at best it represents a very small percentage of hEDS patients.” (https://www.ehlers-danlos.com/heds-hsd-research/#1694081826099-339fe523-0fcf https://www.ehlers-danlos.com/heds-hsd-research/#16940818260...) Possibly again in reference to the original Zewers paper.
It could be that the Zewers paper is referenced by many other papers as a plausible hypothesis without providing contradictory evidence, perhaps the LLMs see this and give the paper much more weight because of that. Is the LLM incorrectly including it due to confusion with clEDS? Are doctors incorrectly excluding it for the same reason? I would love to know.