6 ms·

I recently went to see George Church and Siddhartha Mukherjee talk about genetic manipulation at Pioneer Works here in Brooklyn. Of course CRISPR/cas-9 came up

I recently went to see George Church and Siddhartha Mukherjee talk about genetic manipulation at Pioneer Works here in Brooklyn. Of course CRISPR/cas-9 came up for part of the discussion, but one question I wanted to ask and didn't get a chance to is: what is the error rate on the technique? I can't imagine that it is zero, and every article I read about CRISPR/cas-9 seems to leave that part out. Can anyone speak to that?

jfarlow 10y ago
I don't think the 'technique' is settled enough to have an answer to your question yet. There are a lot of different 'errors' that can happen, each of which has entire sections of researchers working to ameliorate them. The floor to error is that a wild-type Cas9 protein with a naively generated guide RNA, and a payload DNA along with it, just dropped into a petri-dish of cells correctly edits some single-digit percent of cells. That's the crudest of experiments a grad student might do on their first try. And that it is that robust (in a biological laboratory) is precisely why it is so powerful. Single-digit percent activity on an unoptimized first try is actually pretty amazing in the context of biology.
- mtalantikite 10y ago
  Thanks, that's helpful to know. I'm actually thinking about within the context of that edit of the cells. Is it a binary thing where either the cell is edited or it isn't, or can we have an edit with some error rate in regards to what actually was pasted in? The cutting seems highly specific, you need some sort of homology between your guide RNA and the target sequence, right? It's the pasting I'm mainly curious about. All the pop science articles seem to make it out to be a flawless system, and the engineer in me is highly skeptical of that narrative.
  - jfarlow 10y ago
    The cutting is mostly specific (given you have a reasonable target sequence - a pure 'GGGGGGGGGGGGGG' sequence is just chemically problematic anyway). The cutting splits the DNA in half completely. It has to rejoin itself (by a not-well understood mechanism termed 'NHEJ'). This rejoining of the DNA is the kind of 'well it works (mostly, sometimes!)' part that is actively being better understood. Cas9 really has no relevance to this part of the process - unless it is used as a platform to attach DNA-repair machinery onto. Like in all things biology, concentration and time matter. So if you leave Cas9 on 24/7 at very high concentrations with nothing to do, it will go ahead and cut all over the place. The goal is now to bring to bear everything else we know about biological expression to put the Cas9 homing system in its place, only when it should be in place. We already know pretty good mechanisms to make sure Cas9 only turns on under certain conditions, at certain times, is inhibited under other conditions, is otherwise made even more specific. Those are engineering 'details' at this point. And by 'details' I mean you have a huge academic and corporate effort underway currently figuring out how most efficiently and most accurately control Cas9's function. Common results with a naive wild-type Cas9 (which we have moved on from in a lot of ways): (no change): ====== --> ====== (unhealed breakage): ====== --> === & === (forwards insert): ====== --> ===(insert)=== (backwards insert): ====== --> ===(tresni)=== (tandem insert): ====== --> ===(insert)(insert)(...)=== (off-target insert): ====== --> =(insert)===== (intermediate failure: ====== --> ===(insert) & === || .... There are also very powerful uses for Cas9 that do not cleave DNA at all. Sticking a gigantic 'BREAK()' command onto Cas9 (that cannot cut DNA) can be very useful where the natural break command has been lost.
  - tstactplsignore 10y ago
    I've worked on this exact problem before. There are two kinds of possible errors: 1. On site errors- small errors that occur at the spot you intended to edit. Occur maybe 30% of the time- usually will not break anything, but worth thinking about. What we know so far is that these are not random and it will be possible to rationally design crispr sequences to reduce (or even purposefully cause) specific on site errors. 2. Off target errors- this is when Cas9 cuts the genome at the wrong spot- this is the main error people are worried about. Computational efforts in guide design have attempted to reduce this, and will get more advanced as we learn more about the binding kinetics of Cas9. These also occur non-randomly, at highly quantifiable frequencies, and are due to biophysical rules. Also, newly discovered and engineered Cas variants have substantially better specificity and will work to further reduce this problem. Also off target effects are usually not problematic and can be tested for (will be bred out of crispr-ed crops)- however, if you're going to do CRISPR in live humans it is likely that regulatory bodies will require a demonstrated very low probability of off target effects. Basically, errors are an issue to spend a lot of time thinking about when designing a crispr system intended as a human therapeutic. For most scientists, however, errors are not terribly important or common and they don't need anything beyond a bit of rational guide design for their experiments.
kbenson 10y ago
You got me wondering, so I did a quick google search. A little over a year ago an advance was made to take it to a 60% success rate[1]. I'm not sure if any other advanceshave been made. Additionally, this[2] site seems to note multiple different success rates based on project goals. Another article[3] notes: Why some guide RNAs work well, setting up Cas9 to cut and disable a gene nearly 100 percent of the time, while others bind but seldom or never knock out the gene, has been a puzzle since the technique was invented by Jennifer Doudna of UC Berkeley and Emmanuelle Charpentier of Umea University in 2012. The cutting efficiency varies with the type of cell and the particular cell line... 1: http://news.berkeley.edu/2016/01/20/advance-improves-cutting-and-pasting-with-crispr-cas9-gene-editing/ http://news.berkeley.edu/2016/01/20/advance-improves-cutting... 2: http://ko.cwru.edu/services/crispr_cas.shtml#successrates http://ko.cwru.edu/services/crispr_cas.shtml#successrates 3: https://phys.org/news/2016-08-crispr-cas9-genes-disrupt-dna.html https://phys.org/news/2016-08-crispr-cas9-genes-disrupt-dna....
gwern 10y ago
I've been interested in the same question. It's hard to say because the genetics papers are very technical indeed, and the error rate is going to depend on how you define it, what kinds of edits you were trying to make, how you measure/detect errors (as well as how reliable your process is there), whether you're doing it to in vitro cells or embryos or whole organisms (in increasing order of difficulty), and, since CRISPR has been advancing so fast, when the work described in a paper was done (something done in 2014 and published in 2016 is going to be much less relevant than work done in 2016 but not yet published). For example, kbenson quotes a 60% figure, but that is for number of successful edits as opposed to non-edits or edits somewhere else; which one you care about is going to depend on your purpose. For 'edits somewhere else', which are new mutations which might be harmful, as far as I can tell as an interested layman, in terms of the last category of errors, 'off-target' edits, the state of the CRISPR art is a low, effectively near zero mutation rate: - "High-fidelity CRISPR-Cas9 nucleases with no detectable genome-wide off-target effects" Kleinstiver et al 2016, https://www.gwern.net/docs/genetics/2016-kleinstiver.pdf https://www.gwern.net/docs/genetics/2016-kleinstiver.pdf - "Rationally engineered Cas9 nucleases with improved specificity", Slaymaker et al 2016 https://www.gwern.net/docs/genetics/2016-slaymaker.pdf https://www.gwern.net/docs/genetics/2016-slaymaker.pdf - Church, April 2016: "Indeed, the latest versions of gene-editing enzymes have zero detectable off-target activities." http://www.wsj.com/articles/should-heritable-gene-editing-be-used-on-humans-1460340173 http://www.wsj.com/articles/should-heritable-gene-editing-be... - Church, June 2016 "Church: In practice, when we introduced our first CRISPR in 2013,19 it was about 5% off target. In other words, CRISPR would edit five treated cells out of 100 in the wrong place in the genome. Now, we can get down to about one error per 6 trillion cells...Fahy: Just how efficient is CRISPR at editing targeted genes? Church: Without any particular tricks, you can get anywhere up to, on the high end, into the range of 50% to 80% or more of targeted genes actually getting edited in the intended way. Fahy: Why not 100%? Church: We don't really know, but over time, we're getting closer and closer to 100%, and I suspect that someday we will get to 100%. Fahy: Can you get a higher percentage of successful gene edits by dosing with CRISPR more than once? Church: Yes, but there are limits." http://www.lifeextension.com/Lpages/2016/CRISPR/index http://www.lifeextension.com/Lpages/2016/CRISPR/index In human embryos specifically, the published work is below state-of-the-art (unsurprising, given the taboo) but shows increasingly good performance in terms of making the desired edit and also not making undesired edits: - Liang et al 2015 "CRISPR/Cas9-mediated gene editing in human tripronuclear zygotes" http://link.springer.com/article/10.1007/s13238-015-0153-5%20/fulltext.html http://link.springer.com/article/10.1007/s13238-015-0153-5%2... http://www.nature.com/news/chinese-scientists-genetically-modify-human-embryos-1.17378 http://www.nature.com/news/chinese-scientists-genetically-mo... - Kang et al 2016, "Introducing precise genetic modifications into human 3PN embryos by CRISPR/Cas-mediated genome editing" https://www.gwern.net/docs/genetics/2016-kang.pdf https://www.gwern.net/docs/genetics/2016-kang.pdf http://www.nature.com/news/second-chinese-team-reports-gene-editing-in-human-embryos-1.19718 http://www.nature.com/news/second-chinese-team-reports-gene-... - Komor et al 2016, "Programmable editing of a target base in genomic DNA without double-stranded DNA cleavage" https://ase.tufts.edu/chemistry/kumar/jc/pdf/Liu_2016.pdf https://ase.tufts.edu/chemistry/kumar/jc/pdf/Liu_2016.pdf - http://www.nature.com/news/chinese-scientists-to-pioneer-first-human-crispr-trial-1.20302 http://www.nature.com/news/chinese-scientists-to-pioneer-fir... - "CRISPR/Cas9-mediated gene editing in human zygotes using Cas9 protein" Tang et al 2017 https://www.gwern.net/docs/genetics/2017-tang.pdf https://www.gwern.net/docs/genetics/2017-tang.pdf : no observed off-target mutations (but very small sample size); efficiency of 20%, 50%, and 100%
dnautics 10y ago
the error rate on cutting is low enough that "it is no longer the issue". CRISPR is not the whole story for any given genetic manipulation project. Here's generally how it works: 1) cut DNA (this is the part that CRISPR does) 2) add replacement DNA 3) kill everything that doesn't have the replacement DNA in there 4) optional: repeat the process to 'clean up' your modification [0] Basically in the pre-CRISPR era, if you, say, wanted to make a transgenic mouse, you had to take some embryonic mouse stem cells and just add DNA and hope that it found its way into the "right place". If you were adding a totally new gene - usually not a big deal because you kind of don't care where it winds up, shoot first, do your experiment, ask questions later. If you were knocking out or replacing a gene, (aka specific location addressing) it is a big deal. Suddenly step 3), while necessary, is not sufficient to guarantee successful DNA modification. Furthermore "checking" is really hard, You need to implant the modified stem cell into a mouse embryo, create a chimeric mouse (a mouse that has cells from two genomes), hope to hell that the cell you implanted into the embryo randomly got chosen to turn into an ovary, and then take the children of the mice which should have the gene modification. And then you check the genome of the mice and it turns out that it's all wrong and you have to start from scratch (literally had a friend who unluckily spent the first 5 years of grad school repeating this process about 10 times IIRC). This gets really complicated when step 1) is inefficient, or cuts in the wrong place, or has a spontaneous integration preferentially in the wrong place - you off the bat take a huge hit (a hit on the order of 90%, if you were lucky). With CRISPR, you basically derisk step 1), pushing the 'hard part' of molecular biology elsewhere. Suddenly your step 3) selections instead of being a low-yield, probabilistic crapshoot, are near-quantitatively correct. Since it's also the first step, suddenly the one or two 'actually hard' parts of molecular biology, like in the case of the mice the complicated process of generating chimerics, etc, can really be tackled head on by sheer numbers much more easily. [0] in order to do 3) you usually need some extra stuff (resistance genes) that you might not want in your 'final work', so you might have to come up with a second stage to 'clean this up'. Keep in mind that Church and Mukherjee have not really ever 'been in the trenches' actually doing these things (church was a structural biochemist and later a yeast geneticist - yeast are easy peasy). Their grad students and postdocs have.

jfarlow 10y ago

I don't think the 'technique' is settled enough to have an answer to your question yet. There are a lot of different 'errors' that can happen, each of which has entire sections of researchers working to ameliorate them. The floor to error is that a wild-type Cas9 protein with a naively generated guide RNA, and a payload DNA along with it, just dropped into a petri-dish of cells correctly edits some single-digit percent of cells. That's the crudest of experiments a grad student might do on their first try. And that it is that robust (in a biological laboratory) is precisely why it is so powerful. Single-digit percent activity on an unoptimized first try is actually pretty amazing in the context of biology.

mtalantikite 10y ago
Thanks, that's helpful to know. I'm actually thinking about within the context of that edit of the cells. Is it a binary thing where either the cell is edited or it isn't, or can we have an edit with some error rate in regards to what actually was pasted in? The cutting seems highly specific, you need some sort of homology between your guide RNA and the target sequence, right? It's the pasting I'm mainly curious about. All the pop science articles seem to make it out to be a flawless system, and the engineer in me is highly skeptical of that narrative.
- jfarlow 10y ago
  The cutting is mostly specific (given you have a reasonable target sequence - a pure 'GGGGGGGGGGGGGG' sequence is just chemically problematic anyway). The cutting splits the DNA in half completely. It has to rejoin itself (by a not-well understood mechanism termed 'NHEJ'). This rejoining of the DNA is the kind of 'well it works (mostly, sometimes!)' part that is actively being better understood. Cas9 really has no relevance to this part of the process - unless it is used as a platform to attach DNA-repair machinery onto. Like in all things biology, concentration and time matter. So if you leave Cas9 on 24/7 at very high concentrations with nothing to do, it will go ahead and cut all over the place. The goal is now to bring to bear everything else we know about biological expression to put the Cas9 homing system in its place, only when it should be in place. We already know pretty good mechanisms to make sure Cas9 only turns on under certain conditions, at certain times, is inhibited under other conditions, is otherwise made even more specific. Those are engineering 'details' at this point. And by 'details' I mean you have a huge academic and corporate effort underway currently figuring out how most efficiently and most accurately control Cas9's function. Common results with a naive wild-type Cas9 (which we have moved on from in a lot of ways): (no change): ====== --> ====== (unhealed breakage): ====== --> === & === (forwards insert): ====== --> ===(insert)=== (backwards insert): ====== --> ===(tresni)=== (tandem insert): ====== --> ===(insert)(insert)(...)=== (off-target insert): ====== --> =(insert)===== (intermediate failure: ====== --> ===(insert) & === || .... There are also very powerful uses for Cas9 that do not cleave DNA at all. Sticking a gigantic 'BREAK()' command onto Cas9 (that cannot cut DNA) can be very useful where the natural break command has been lost.
- tstactplsignore 10y ago
  I've worked on this exact problem before. There are two kinds of possible errors: 1. On site errors- small errors that occur at the spot you intended to edit. Occur maybe 30% of the time- usually will not break anything, but worth thinking about. What we know so far is that these are not random and it will be possible to rationally design crispr sequences to reduce (or even purposefully cause) specific on site errors. 2. Off target errors- this is when Cas9 cuts the genome at the wrong spot- this is the main error people are worried about. Computational efforts in guide design have attempted to reduce this, and will get more advanced as we learn more about the binding kinetics of Cas9. These also occur non-randomly, at highly quantifiable frequencies, and are due to biophysical rules. Also, newly discovered and engineered Cas variants have substantially better specificity and will work to further reduce this problem. Also off target effects are usually not problematic and can be tested for (will be bred out of crispr-ed crops)- however, if you're going to do CRISPR in live humans it is likely that regulatory bodies will require a demonstrated very low probability of off target effects. Basically, errors are an issue to spend a lot of time thinking about when designing a crispr system intended as a human therapeutic. For most scientists, however, errors are not terribly important or common and they don't need anything beyond a bit of rational guide design for their experiments.

kbenson 10y ago

You got me wondering, so I did a quick google search. A little over a year ago an advance was made to take it to a 60% success rate[1]. I'm not sure if any other advanceshave been made. Additionally, this[2] site seems to note multiple different success rates based on project goals. Another article[3] notes: Why some guide RNAs work well, setting up Cas9 to cut and disable a gene nearly 100 percent of the time, while others bind but seldom or never knock out the gene, has been a puzzle since the technique was invented by Jennifer Doudna of UC Berkeley and Emmanuelle Charpentier of Umea University in 2012. The cutting efficiency varies with the type of cell and the particular cell line... 1: http://news.berkeley.edu/2016/01/20/advance-improves-cutting-and-pasting-with-crispr-cas9-gene-editing/ http://news.berkeley.edu/2016/01/20/advance-improves-cutting... 2: http://ko.cwru.edu/services/crispr_cas.shtml#successrates http://ko.cwru.edu/services/crispr_cas.shtml#successrates 3: https://phys.org/news/2016-08-crispr-cas9-genes-disrupt-dna.html https://phys.org/news/2016-08-crispr-cas9-genes-disrupt-dna....

gwern 10y ago

I've been interested in the same question. It's hard to say because the genetics papers are very technical indeed, and the error rate is going to depend on how you define it, what kinds of edits you were trying to make, how you measure/detect errors (as well as how reliable your process is there), whether you're doing it to in vitro cells or embryos or whole organisms (in increasing order of difficulty), and, since CRISPR has been advancing so fast, when the work described in a paper was done (something done in 2014 and published in 2016 is going to be much less relevant than work done in 2016 but not yet published). For example, kbenson quotes a 60% figure, but that is for number of successful edits as opposed to non-edits or edits somewhere else; which one you care about is going to depend on your purpose. For 'edits somewhere else', which are new mutations which might be harmful, as far as I can tell as an interested layman, in terms of the last category of errors, 'off-target' edits, the state of the CRISPR art is a low, effectively near zero mutation rate: - "High-fidelity CRISPR-Cas9 nucleases with no detectable genome-wide off-target effects" Kleinstiver et al 2016, https://www.gwern.net/docs/genetics/2016-kleinstiver.pdf https://www.gwern.net/docs/genetics/2016-kleinstiver.pdf - "Rationally engineered Cas9 nucleases with improved specificity", Slaymaker et al 2016 https://www.gwern.net/docs/genetics/2016-slaymaker.pdf https://www.gwern.net/docs/genetics/2016-slaymaker.pdf - Church, April 2016: "Indeed, the latest versions of gene-editing enzymes have zero detectable off-target activities." http://www.wsj.com/articles/should-heritable-gene-editing-be-used-on-humans-1460340173 http://www.wsj.com/articles/should-heritable-gene-editing-be... - Church, June 2016 "Church: In practice, when we introduced our first CRISPR in 2013,19 it was about 5% off target. In other words, CRISPR would edit five treated cells out of 100 in the wrong place in the genome. Now, we can get down to about one error per 6 trillion cells...Fahy: Just how efficient is CRISPR at editing targeted genes? Church: Without any particular tricks, you can get anywhere up to, on the high end, into the range of 50% to 80% or more of targeted genes actually getting edited in the intended way. Fahy: Why not 100%? Church: We don't really know, but over time, we're getting closer and closer to 100%, and I suspect that someday we will get to 100%. Fahy: Can you get a higher percentage of successful gene edits by dosing with CRISPR more than once? Church: Yes, but there are limits." http://www.lifeextension.com/Lpages/2016/CRISPR/index http://www.lifeextension.com/Lpages/2016/CRISPR/index In human embryos specifically, the published work is below state-of-the-art (unsurprising, given the taboo) but shows increasingly good performance in terms of making the desired edit and also not making undesired edits: - Liang et al 2015 "CRISPR/Cas9-mediated gene editing in human tripronuclear zygotes" http://link.springer.com/article/10.1007/s13238-015-0153-5%20/fulltext.html http://link.springer.com/article/10.1007/s13238-015-0153-5%2... http://www.nature.com/news/chinese-scientists-genetically-modify-human-embryos-1.17378 http://www.nature.com/news/chinese-scientists-genetically-mo... - Kang et al 2016, "Introducing precise genetic modifications into human 3PN embryos by CRISPR/Cas-mediated genome editing" https://www.gwern.net/docs/genetics/2016-kang.pdf https://www.gwern.net/docs/genetics/2016-kang.pdf http://www.nature.com/news/second-chinese-team-reports-gene-editing-in-human-embryos-1.19718 http://www.nature.com/news/second-chinese-team-reports-gene-... - Komor et al 2016, "Programmable editing of a target base in genomic DNA without double-stranded DNA cleavage" https://ase.tufts.edu/chemistry/kumar/jc/pdf/Liu_2016.pdf https://ase.tufts.edu/chemistry/kumar/jc/pdf/Liu_2016.pdf - http://www.nature.com/news/chinese-scientists-to-pioneer-first-human-crispr-trial-1.20302 http://www.nature.com/news/chinese-scientists-to-pioneer-fir... - "CRISPR/Cas9-mediated gene editing in human zygotes using Cas9 protein" Tang et al 2017 https://www.gwern.net/docs/genetics/2017-tang.pdf https://www.gwern.net/docs/genetics/2017-tang.pdf : no observed off-target mutations (but very small sample size); efficiency of 20%, 50%, and 100%

dnautics 10y ago

the error rate on cutting is low enough that "it is no longer the issue". CRISPR is not the whole story for any given genetic manipulation project. Here's generally how it works: 1) cut DNA (this is the part that CRISPR does) 2) add replacement DNA 3) kill everything that doesn't have the replacement DNA in there 4) optional: repeat the process to 'clean up' your modification [0] Basically in the pre-CRISPR era, if you, say, wanted to make a transgenic mouse, you had to take some embryonic mouse stem cells and just add DNA and hope that it found its way into the "right place". If you were adding a totally new gene - usually not a big deal because you kind of don't care where it winds up, shoot first, do your experiment, ask questions later. If you were knocking out or replacing a gene, (aka specific location addressing) it is a big deal. Suddenly step 3), while necessary, is not sufficient to guarantee successful DNA modification. Furthermore "checking" is really hard, You need to implant the modified stem cell into a mouse embryo, create a chimeric mouse (a mouse that has cells from two genomes), hope to hell that the cell you implanted into the embryo randomly got chosen to turn into an ovary, and then take the children of the mice which should have the gene modification. And then you check the genome of the mice and it turns out that it's all wrong and you have to start from scratch (literally had a friend who unluckily spent the first 5 years of grad school repeating this process about 10 times IIRC). This gets really complicated when step 1) is inefficient, or cuts in the wrong place, or has a spontaneous integration preferentially in the wrong place - you off the bat take a huge hit (a hit on the order of 90%, if you were lucky). With CRISPR, you basically derisk step 1), pushing the 'hard part' of molecular biology elsewhere. Suddenly your step 3) selections instead of being a low-yield, probabilistic crapshoot, are near-quantitatively correct. Since it's also the first step, suddenly the one or two 'actually hard' parts of molecular biology, like in the case of the mice the complicated process of generating chimerics, etc, can really be tackled head on by sheer numbers much more easily. [0] in order to do 3) you usually need some extra stuff (resistance genes) that you might not want in your 'final work', so you might have to come up with a second stage to 'clean this up'. Keep in mind that Church and Mukherjee have not really ever 'been in the trenches' actually doing these things (church was a structural biochemist and later a yeast geneticist - yeast are easy peasy). Their grad students and postdocs have.