4 ms·

Unlike what you might think, this is not a sequencing based method, but something based on dna sticking to small parts of a chip coated with complimentary seque

Unlike what you might think, this is not a sequencing based method, but something based on dna sticking to small parts of a chip coated with complimentary sequences. This method is super error prone, more importantly the error rate is dependent on who does it and with what level of care. So yes it's not going to be repeatable!

tonto 8y ago
You do not offer any citation as to why this, the chip based genotyping, leads to misinterpretation and the article does not suggest this is the reason either. Without getting into it, the article says >An entire DNA sample is made up of about three billion parts, but companies that provide ancestry tests look at about 700,000 of those to spot genetic differences. >According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical. >Still, none of the five companies provided the same ancestry breakdown for the twins.
benj111 8y ago
Ok this is new to me. So these people getting results and finding out they aren't related to their parents, are the results suspect? Ditto the results about genes associated with dementia or heart disease? I'm aware of the companies having disclaimers, I thought that was on the basis of them not really know what the genes did, not that the test itself was rubbish.
- nyolfen 8y ago
  he is dramatically overstating the error rate
- testvox 8y ago
  He is just wrong, the twins in this case got the same "results" as far as the genotyping is concerned. The differences were in the analysis provided.
  - benj111 8y ago
    That would explain the differences between the companies, not between the twins, no? Is it a case of them adding too many decimal places to the percentages? Making statistics out of noise?
nkrumm 8y ago
It's true that these companies are not offering "sequencing" per se but rather are genotyping commonly polymorphic SNPs using an array or other genotyping method. These SNP arrays, however, are exceedingly accurate and reproducible in their calls. A common medium-density SNP array used for human genetics is the Illumina CytoSNP-850K [1], which routinely gets >99.5% accuracy on calls. This is at least as good, or better than, whole genome sequencing. The challenge in ancestry assignment is not determining what is in the sample, but rather in the interpretation and assignment of ancestral haplotypes. [1] https://www.illumina.com/content/dam/illumina-marketing/documents/products/datasheets/datasheet_CytoSNP850K_POP.pdf https://www.illumina.com/content/dam/illumina-marketing/docu...
- yorwba 8y ago
  Since the test results for the twins in the article were 99.6% identical, that seems to match what you say is the usual accuracy of SNP arrays. However, I don't think that can be called exceedingly accurate and reproducible if you consider the intended application. >99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers.
  - Scaevolus 8y ago
    If the practical difference is 38% vs 39% "Eastern European", who cares?
  - bigiain 8y ago
    ">99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers." Or potentially put people in jail based on these results...
  - jhall1468 8y ago
    If you feel that way about 99.5% accuracy, I'd advise that there are a number of consumer products you absolutely shouldn't buy, including: kitchen scales (small ones), measuring cups, store brand spirit levels, etc. You can get considerably better accuracy, you just have to be willing to fork over $1500 rather than $60. How much are you willing to pay to know if your 31% German vs 30%?
Obi_Juan_Kenobi 8y ago
DNA microarrays do have limitations vs. RNAseq, but they are accurate and repeatable when used properly. https://en.wikipedia.org/wiki/DNA_microarray https://en.wikipedia.org/wiki/DNA_microarray Calling them 'super error prone' is simply inaccurate. There was a good 5-10 years where microarrays were the go-to method for genomic investigation. Since RNA-seq has supplanted it, there has been no revolution in overturning those old results. At best, RNA-seq reveals what microarrays cannot: novel sequences, specific sequences that have difficulty hybridizing, etc. The variation between these services is due primarily to how the data is interpreted and presented to customers, not the reads themselves.
- nkrumm 8y ago
  Note that these companies are not doing DNA microarrays in the sense you're describing (which is for quantifying gene expression by mRNA level). RNA-seq is (now) arguably better for this purpose as well (better dynamic range, primarily), but again this is not what any of the ancestry or personalized genetics companies are doing. In this case "SNP microarray" is a "digital" (or binary) readout of the genomic DNA sequence; specifically the array has attached to it sequences that match both the "reference allele" (e.g., a "G") and the "alternate allele" (e.g., "A" or some other base). Note that across the world's population, most sites in the human genome are "bi-allelic" meaning that there's really only two variants at each site, though of course it is possible to have tri-allelic or even fully polymorphic (A, T, C, or G) sites. For the purposes of ancestry, bi-allelic identification of commonly polymorphic sites is adequate, though the more sites and the rarer those sites are in world populations, the more specific you can be re: ancestry. This is why the gold standard for ancestry and haplotyping is whole genome sequencing-- you simply get far more informative sites to map to other populations.
ken 8y ago
It's fair that a cheap test might not be super accurate, but then why are they stating figures like "13%" if the margin of error is at least 10 percentage points? That's just plain misleading. When my doctor asks how tall I am, if I'm 6'1", then 6'1" is a good answer. 6' (or even better: about 6') can also be an acceptable answer, as it's an approximation. I wouldn't say I'm 6'8", and then try to rationalize it by saying my tape measure is cheap, and anyway it's still right to about 10%. Reporting what you don't know can be just as important as reporting what you do know. It doesn't matter how good your laboratory method is if you can't report results in a way that doesn't mislead customers.

tonto 8y ago

You do not offer any citation as to why this, the chip based genotyping, leads to misinterpretation and the article does not suggest this is the reason either. Without getting into it, the article says >An entire DNA sample is made up of about three billion parts, but companies that provide ancestry tests look at about 700,000 of those to spot genetic differences. >According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical. >Still, none of the five companies provided the same ancestry breakdown for the twins.

benj111 8y ago

Ok this is new to me. So these people getting results and finding out they aren't related to their parents, are the results suspect? Ditto the results about genes associated with dementia or heart disease? I'm aware of the companies having disclaimers, I thought that was on the basis of them not really know what the genes did, not that the test itself was rubbish.

nyolfen 8y ago
he is dramatically overstating the error rate
testvox 8y ago
He is just wrong, the twins in this case got the same "results" as far as the genotyping is concerned. The differences were in the analysis provided.
- benj111 8y ago
  That would explain the differences between the companies, not between the twins, no? Is it a case of them adding too many decimal places to the percentages? Making statistics out of noise?

nkrumm 8y ago

It's true that these companies are not offering "sequencing" per se but rather are genotyping commonly polymorphic SNPs using an array or other genotyping method. These SNP arrays, however, are exceedingly accurate and reproducible in their calls. A common medium-density SNP array used for human genetics is the Illumina CytoSNP-850K [1], which routinely gets >99.5% accuracy on calls. This is at least as good, or better than, whole genome sequencing. The challenge in ancestry assignment is not determining what is in the sample, but rather in the interpretation and assignment of ancestral haplotypes. [1] https://www.illumina.com/content/dam/illumina-marketing/documents/products/datasheets/datasheet_CytoSNP850K_POP.pdf https://www.illumina.com/content/dam/illumina-marketing/docu...

yorwba 8y ago
Since the test results for the twins in the article were 99.6% identical, that seems to match what you say is the usual accuracy of SNP arrays. However, I don't think that can be called exceedingly accurate and reproducible if you consider the intended application. >99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers.
- Scaevolus 8y ago
  If the practical difference is 38% vs 39% "Eastern European", who cares?
- bigiain 8y ago
  ">99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers." Or potentially put people in jail based on these results...
- jhall1468 8y ago
  If you feel that way about 99.5% accuracy, I'd advise that there are a number of consumer products you absolutely shouldn't buy, including: kitchen scales (small ones), measuring cups, store brand spirit levels, etc. You can get considerably better accuracy, you just have to be willing to fork over $1500 rather than $60. How much are you willing to pay to know if your 31% German vs 30%?

Obi_Juan_Kenobi 8y ago

DNA microarrays do have limitations vs. RNAseq, but they are accurate and repeatable when used properly. https://en.wikipedia.org/wiki/DNA_microarray https://en.wikipedia.org/wiki/DNA_microarray Calling them 'super error prone' is simply inaccurate. There was a good 5-10 years where microarrays were the go-to method for genomic investigation. Since RNA-seq has supplanted it, there has been no revolution in overturning those old results. At best, RNA-seq reveals what microarrays cannot: novel sequences, specific sequences that have difficulty hybridizing, etc. The variation between these services is due primarily to how the data is interpreted and presented to customers, not the reads themselves.

nkrumm 8y ago
Note that these companies are not doing DNA microarrays in the sense you're describing (which is for quantifying gene expression by mRNA level). RNA-seq is (now) arguably better for this purpose as well (better dynamic range, primarily), but again this is not what any of the ancestry or personalized genetics companies are doing. In this case "SNP microarray" is a "digital" (or binary) readout of the genomic DNA sequence; specifically the array has attached to it sequences that match both the "reference allele" (e.g., a "G") and the "alternate allele" (e.g., "A" or some other base). Note that across the world's population, most sites in the human genome are "bi-allelic" meaning that there's really only two variants at each site, though of course it is possible to have tri-allelic or even fully polymorphic (A, T, C, or G) sites. For the purposes of ancestry, bi-allelic identification of commonly polymorphic sites is adequate, though the more sites and the rarer those sites are in world populations, the more specific you can be re: ancestry. This is why the gold standard for ancestry and haplotyping is whole genome sequencing-- you simply get far more informative sites to map to other populations.

ken 8y ago

It's fair that a cheap test might not be super accurate, but then why are they stating figures like "13%" if the margin of error is at least 10 percentage points? That's just plain misleading. When my doctor asks how tall I am, if I'm 6'1", then 6'1" is a good answer. 6' (or even better: about 6') can also be an acceptable answer, as it's an approximation. I wouldn't say I'm 6'8", and then try to rationalize it by saying my tape measure is cheap, and anyway it's still right to about 10%. Reporting what you don't know can be just as important as reporting what you do know. It doesn't matter how good your laboratory method is if you can't report results in a way that doesn't mislead customers.