8 ms·

Lol. If it would take 8.5 yrs to review, it's probably god awful, and should never ever ever be used to convict someone of such a crime. My prediction: this fi

by breck 6y ago

Lol. If it would take 8.5 yrs to review, it's probably god awful, and should never ever ever be used to convict someone of such a crime. My prediction: this firm will probably try to get removed from the case, rather than open source their shitty code. Source: I've worked on MATLAB codebases for various genomics research projects in the past.

nickodell 6y ago
>My prediction: this firm will probably try to get removed from the case, rather than open source their shitty code. That isn't necessarily their choice. The prosecutors will make the decision about whether to withdraw the DNA evidence. They probably won't, given that they would need to give the defendant a new trial, which could lead to an accused murderer getting off. A bad look for any prosecutor. More to the point, if the firm withdraws from any case where their credibility is questioned, what does that say to law enforcement agencies who are thinking about using their software?
- bdavisx 6y ago
  It would also give every person convicted using their software an incentive to open an appeal.
  - Spivak 6y ago
    I like how this is considered a bad thing. Like we can’t let this guy point out that he’s being convicted by an unauditable black box that suddenly isn’t worth using if it has to stand up to scrutiny because then everyone would want to. The horror. Like I’m actually kinda shocked this is the reality. I would have assumed that DNA evidence would have some blessed methodologies and tools/algorithms, with a strict definition of what constitutes a match or partial match specifically so this wouldn’t happen.
    - hluska 6y ago
      This is one of these scary areas where reality matches my teenaged experiences playing Shadowrun. I used to hope that the brutal dystopia we played through was just fun. Now I’m seeing that the present needs a word even more brutal than dystopia. :(
      - alchemism 6y ago
        Kafkatopia
        hluska 6y ago
        You nailed it! Thanks friend, that’s some wicked writing and thinking.
    - belorn 6y ago
      Here in Sweden, there is a legal practice that you can't find someone guilty based on DNA evidence alone. Probabilistic evidence is nice to point law enforcement in a direction, but there is always a risk of false positives. In this case we are also dealing with probabilistic genotyping involving DNA Mixtures with DNA from several individual contributors, and most likely degraded DNA. It is the tool the police can use when other more traditional methods is not possible because of the mixture. That should mean the qualitative value of the DNA evidence is lower, requiring even stronger additional evidence from other sources.
      - MereInterest 6y ago
        Interesting. What does Swedish law consider non-probabilistic evidence? Even something like eye-witness testimony I would consider to be probabilistic, given how easy it is to manipulate memories, even unintentionally.
        acct776 6y ago
        Clear videographic, use of a PIN that only the accused had access to, etc
        gwd 6y ago
        How do you prove that only the accused had access to a PIN? Surely that's probabilistic as well?
      - Blikkentrekker 6y ago
        In the U.S.A., a man can be convicted upon the word of a single witness, even if the defence poked significant holes into the reliability of said witness. What can happen in the U.S.A. is that one lone man says “I saw the defendant do it.”; the defence attorney can point out that the witness was drunk at the time, that he has motive to lie, that he initially reported another story to the police and only later settled on this story, and what ever else to render him completely unreliable. The jury can nevertheless return a verdict of guilty, and there are no grounds for appeal then, as it is the power of the jury to decide who is “reliable”, and it is not required to explain it's thought process at all. What a shocking development that such would result into a criminal justice system where a defendant's race and gender plays such a factor.
    - Blikkentrekker 6y ago
      I do not find this reality worse at all than people being convicted upon the black box testimony of blood splatter analysts, which is simply an expert testifying that in his conclusion the blood indicated such-and-that. Or of course, that the U.S.A. permits conviction based on the sworn testimony of a single eye witness, which is noteably unreliable. All of these are black boxes that are routinely meant to convict. — it would not surprise me if such software were far more reliable than human eye witness accounts, but if there's one thing I noticed, it's that a man is seldom afraid of bad matters, he is only afraid of bad matters produced by new technology; far worse matters can stay, so long as they be ancient enough.
- gidan 6y ago
  That would be implying that the prosecutor would prefer taking the life of an innocent rather than having it hurt his career, making the prosecutor kind of a criminal.
  - nickodell 6y ago
    >taking the life of an innocent The prosecutor isn't unilaterally deciding whether the DNA evidence is valid. There will be a public hearing where both the prosecution and defense show evidence about the validity of the DNA evidence, and a court will rule based on that evidence.
    - bscphil 6y ago
      Yeah, the system is in a pretty horrific state when you have to count on prosecutors' restraint for anything. Granted, we are in such a state, but it's beneficial not to just accept that as the status quo.
    - afthonos 6y ago
      You should read up on the rates of plea bargaining, as well as the methods prosecutors use to push defendants to do so, which include: - Not revealing all information they are required to. - Parallel construction (see above) - Overcharging, with the goal of making the plea more palatable than the cost/risk of defending multiple absurd charges. - Lying to you while getting to throw you in jail if you lie to them. As a result, only 5% of federal cases go to trial. None of behaviors these are rare. If your understanding of the legal system is based on popular culture, as most people’s is, it is basically law enforcement propaganda that has little relationship to reality.
      - nickodell 6y ago
        Believe it or not, I was already aware of all of those things, having followed a number of criminal defense blogs. If you read the article and appellate decision which is linked, it says what I just said: >On Wednesday, the appellate court sided with the defense [PDF] and sent the case back to a lower court directing the judge to compel Cybergenetics to make the TrueAllele code available to the defense team.
- ahepp 6y ago
  My understanding is that (some) law enforcement agencies have been more than happy to drop cases rather than subject investigative tools to proper scrutiny[0]. They have no qualms resorting to "parallel construction"[1], and simply using the inadmissible (sometimes illegal) evidence to find admissible evidence. [0] https://arstechnica.com/tech-policy/2015/04/fbi-would-rather-prosecutors-drop-cases-than-disclose-stingray-details/ https://arstechnica.com/tech-policy/2015/04/fbi-would-rather... [1] https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
  - canadianfella 6y ago
    You seem to imply there is a problem with parallel construction.
  - nickodell 6y ago
    Stingrays are more useful as an investigative tool than an evidentiary tool. DNA is the other way around.
Aachen 6y ago
> If it would take 8.5 yrs to review, it's probably god awful, and should never ever ever be used to convict someone of such a crime. It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well. Getting the opportunity to look for bugs is essential in my opinion, but it needn't take multiple years. Focus on the parts you doubt, similar to what you'd do if you were reviewing the scientific method used in analog evidence. Of course, the two aren't identical. Validating scientific methods and validating a program is different in that the program is proprietary and the science (usually) merely behind a paywall. The latter can then be replicated by others and becomes established. The former will only ever be seen by that company and doesn't become established. So scrutiny is necessary, but after a couple cases that used an identical version, requiring access without articulating particular doubts would unduly delay the case. It doesn't seem unreasonable to start trusting the program after a bunch of defendants had experts look at it and found no way to cast doubt on its result. If you don't think software of 180k lines can be used in court under such circumstances because it would take too long to review, we should throw out pretty much all software anywhere in the judicial system. (That's not what you said, but some of the replies including yours hint at that.)
- vkou 6y ago
  > It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well There are a few important differences between a generally accepted method, and some Matlab black-box that you feed an input into, and it prints out 'guilty' and 'not guilty'. 1. The former is based on centuries of peer review, where the best ideas eventually get selected for. The latter is an externally un-reviewed application, which encapsulates the best of whatever we could ship by Thursday. 2. You can call an expert witness to the stand, and ask them questions about the state of the art of <some evidence based method>. You can ask them why. You can ask them about how certain one should be about their statements. You can't cross-examine a black box. The actual solution to your quandary is to require that forensic analysis services must pass an annual, independent, double-blind analysis of the accuracy of their methods, before they are used in a courtroom - and that the results of those audits are made available to the defense. It's one thing for a man in a lab coat to take the microphone and say that their methods are accurate 'to within one in a million'. It's quite another to see an audit, where 100 samples were sent in for analysis over six weeks, and only 92 of them were analysed correctly. A jury might still convict on the basis of that 92% accuracy, but only if other meaningful evidence points against the defendant. Unfortunately, the reality of forensic science in 2021 is that most of it is sloppy bunk, with no assurances of accuracy.
knolan 6y ago
170,000 lines of Matlab code for a project is not a good sign. Unless they’re also including the source of various Matlab toolboxes which are already tested by the Mathworks. It’s such a high-level language it’s hard to imagine what the hell they’re doing with all that code. It’s probably mostly useless cruft from GUIDE.
- sdenton4 6y ago
  My guess is a bit of each: The company high-balling the LoC estimate to try to impress/scare the judge, but prooobably also has a truly terrible codebase.
- breischl 6y ago
  How that number was probably arrived at... PHB: Hey, how many lines of code do we have? CodeMonkey: You want a high estimate or a low estimate? PHB: High CodeMonkey: Well, including unit tests, comments, whitespace, build scripts, integration test harness... 170k At least I hope they have enough testing code to be signficant...
- h0l0cube 6y ago
  Having seen Matlab code that's been exported to C, it's mostly bloated by static arrays, and there's quite a lot of redundant functions. So maybe the LOC count is from a C export?
- willis936 6y ago
  I’ve made 4 decent sized gui based tools before. I would never use GUIDE. At minimum it uses eval. If you want a terrible gui-based gui creation interface: labview awaits.
  - knolan 6y ago
    I compiled eval once for fun.
    - willis936 6y ago
      A friend of mine did that. They were able to run arbitrary .m files from an executable. As far as I know, this doesn't violate any terms of use. I wonder what keeps someone from just downloading the freely available MATLAB runtime and running a compiled eval wrapper and suddenly have a freely available version of MATLAB. Also just because eval can be compiled doesn't mean it should. It will forever be a security risk and I will not write code that uses it in good conscious. Fortunately, Mathworks has provided good alternatives. My personal favorite has been variable field names. It really opens up a lot of elegant coding. https://www.mathworks.com/help/matlab/matlab_prog/string-evaluation.html https://www.mathworks.com/help/matlab/matlab_prog/string-eva...
Blikkentrekker 6y ago
It indeed doesn't seem fair to them that they are forced to share their trade secrets simply because the prosecutors elected to use their software.
- hansel_der 6y ago
  it doesn't seem fair to sentence someone (to death?) based on a unaudited secret sauce tool. the company can always retract their product if they want to keep it a trade secret.
  - Blikkentrekker 6y ago
    You assume it to be one or the other. The real solution would obviously that the prosecutors cannot enter into evidence the conclusions of any closed source software. Of course, this only displaces the issue, what of the black box c.p.u.'s whereupon this software ran?

by breck 6y ago

nickodell 6y ago

>My prediction: this firm will probably try to get removed from the case, rather than open source their shitty code. That isn't necessarily their choice. The prosecutors will make the decision about whether to withdraw the DNA evidence. They probably won't, given that they would need to give the defendant a new trial, which could lead to an accused murderer getting off. A bad look for any prosecutor. More to the point, if the firm withdraws from any case where their credibility is questioned, what does that say to law enforcement agencies who are thinking about using their software?

bdavisx 6y ago
It would also give every person convicted using their software an incentive to open an appeal.
- Spivak 6y ago
  I like how this is considered a bad thing. Like we can’t let this guy point out that he’s being convicted by an unauditable black box that suddenly isn’t worth using if it has to stand up to scrutiny because then everyone would want to. The horror. Like I’m actually kinda shocked this is the reality. I would have assumed that DNA evidence would have some blessed methodologies and tools/algorithms, with a strict definition of what constitutes a match or partial match specifically so this wouldn’t happen.
  - hluska 6y ago
    This is one of these scary areas where reality matches my teenaged experiences playing Shadowrun. I used to hope that the brutal dystopia we played through was just fun. Now I’m seeing that the present needs a word even more brutal than dystopia. :(
    - alchemism 6y ago
      Kafkatopia
      - hluska 6y ago
        You nailed it! Thanks friend, that’s some wicked writing and thinking.
  - belorn 6y ago
    Here in Sweden, there is a legal practice that you can't find someone guilty based on DNA evidence alone. Probabilistic evidence is nice to point law enforcement in a direction, but there is always a risk of false positives. In this case we are also dealing with probabilistic genotyping involving DNA Mixtures with DNA from several individual contributors, and most likely degraded DNA. It is the tool the police can use when other more traditional methods is not possible because of the mixture. That should mean the qualitative value of the DNA evidence is lower, requiring even stronger additional evidence from other sources.
    - MereInterest 6y ago
      Interesting. What does Swedish law consider non-probabilistic evidence? Even something like eye-witness testimony I would consider to be probabilistic, given how easy it is to manipulate memories, even unintentionally.
      - acct776 6y ago
        Clear videographic, use of a PIN that only the accused had access to, etc
        gwd 6y ago
        How do you prove that only the accused had access to a PIN? Surely that's probabilistic as well?
    - Blikkentrekker 6y ago
      In the U.S.A., a man can be convicted upon the word of a single witness, even if the defence poked significant holes into the reliability of said witness. What can happen in the U.S.A. is that one lone man says “I saw the defendant do it.”; the defence attorney can point out that the witness was drunk at the time, that he has motive to lie, that he initially reported another story to the police and only later settled on this story, and what ever else to render him completely unreliable. The jury can nevertheless return a verdict of guilty, and there are no grounds for appeal then, as it is the power of the jury to decide who is “reliable”, and it is not required to explain it's thought process at all. What a shocking development that such would result into a criminal justice system where a defendant's race and gender plays such a factor.
  - Blikkentrekker 6y ago
    I do not find this reality worse at all than people being convicted upon the black box testimony of blood splatter analysts, which is simply an expert testifying that in his conclusion the blood indicated such-and-that. Or of course, that the U.S.A. permits conviction based on the sworn testimony of a single eye witness, which is noteably unreliable. All of these are black boxes that are routinely meant to convict. — it would not surprise me if such software were far more reliable than human eye witness accounts, but if there's one thing I noticed, it's that a man is seldom afraid of bad matters, he is only afraid of bad matters produced by new technology; far worse matters can stay, so long as they be ancient enough.
gidan 6y ago
That would be implying that the prosecutor would prefer taking the life of an innocent rather than having it hurt his career, making the prosecutor kind of a criminal.
- nickodell 6y ago
  >taking the life of an innocent The prosecutor isn't unilaterally deciding whether the DNA evidence is valid. There will be a public hearing where both the prosecution and defense show evidence about the validity of the DNA evidence, and a court will rule based on that evidence.
  - bscphil 6y ago
    Yeah, the system is in a pretty horrific state when you have to count on prosecutors' restraint for anything. Granted, we are in such a state, but it's beneficial not to just accept that as the status quo.
  - afthonos 6y ago
    You should read up on the rates of plea bargaining, as well as the methods prosecutors use to push defendants to do so, which include: - Not revealing all information they are required to. - Parallel construction (see above) - Overcharging, with the goal of making the plea more palatable than the cost/risk of defending multiple absurd charges. - Lying to you while getting to throw you in jail if you lie to them. As a result, only 5% of federal cases go to trial. None of behaviors these are rare. If your understanding of the legal system is based on popular culture, as most people’s is, it is basically law enforcement propaganda that has little relationship to reality.
    - nickodell 6y ago
      Believe it or not, I was already aware of all of those things, having followed a number of criminal defense blogs. If you read the article and appellate decision which is linked, it says what I just said: >On Wednesday, the appellate court sided with the defense [PDF] and sent the case back to a lower court directing the judge to compel Cybergenetics to make the TrueAllele code available to the defense team.
ahepp 6y ago
My understanding is that (some) law enforcement agencies have been more than happy to drop cases rather than subject investigative tools to proper scrutiny[0]. They have no qualms resorting to "parallel construction"[1], and simply using the inadmissible (sometimes illegal) evidence to find admissible evidence. [0] https://arstechnica.com/tech-policy/2015/04/fbi-would-rather-prosecutors-drop-cases-than-disclose-stingray-details/ https://arstechnica.com/tech-policy/2015/04/fbi-would-rather... [1] https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
- canadianfella 6y ago
  You seem to imply there is a problem with parallel construction.
- nickodell 6y ago
  Stingrays are more useful as an investigative tool than an evidentiary tool. DNA is the other way around.

Aachen 6y ago

> If it would take 8.5 yrs to review, it's probably god awful, and should never ever ever be used to convict someone of such a crime. It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well. Getting the opportunity to look for bugs is essential in my opinion, but it needn't take multiple years. Focus on the parts you doubt, similar to what you'd do if you were reviewing the scientific method used in analog evidence. Of course, the two aren't identical. Validating scientific methods and validating a program is different in that the program is proprietary and the science (usually) merely behind a paywall. The latter can then be replicated by others and becomes established. The former will only ever be seen by that company and doesn't become established. So scrutiny is necessary, but after a couple cases that used an identical version, requiring access without articulating particular doubts would unduly delay the case. It doesn't seem unreasonable to start trusting the program after a bunch of defendants had experts look at it and found no way to cast doubt on its result. If you don't think software of 180k lines can be used in court under such circumstances because it would take too long to review, we should throw out pretty much all software anywhere in the judicial system. (That's not what you said, but some of the replies including yours hint at that.)

vkou 6y ago
> It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well There are a few important differences between a generally accepted method, and some Matlab black-box that you feed an input into, and it prints out 'guilty' and 'not guilty'. 1. The former is based on centuries of peer review, where the best ideas eventually get selected for. The latter is an externally un-reviewed application, which encapsulates the best of whatever we could ship by Thursday. 2. You can call an expert witness to the stand, and ask them questions about the state of the art of <some evidence based method>. You can ask them why. You can ask them about how certain one should be about their statements. You can't cross-examine a black box. The actual solution to your quandary is to require that forensic analysis services must pass an annual, independent, double-blind analysis of the accuracy of their methods, before they are used in a courtroom - and that the results of those audits are made available to the defense. It's one thing for a man in a lab coat to take the microphone and say that their methods are accurate 'to within one in a million'. It's quite another to see an audit, where 100 samples were sent in for analysis over six weeks, and only 92 of them were analysed correctly. A jury might still convict on the basis of that 92% accuracy, but only if other meaningful evidence points against the defendant. Unfortunately, the reality of forensic science in 2021 is that most of it is sloppy bunk, with no assurances of accuracy.

knolan 6y ago

170,000 lines of Matlab code for a project is not a good sign. Unless they’re also including the source of various Matlab toolboxes which are already tested by the Mathworks. It’s such a high-level language it’s hard to imagine what the hell they’re doing with all that code. It’s probably mostly useless cruft from GUIDE.

sdenton4 6y ago
My guess is a bit of each: The company high-balling the LoC estimate to try to impress/scare the judge, but prooobably also has a truly terrible codebase.
breischl 6y ago
How that number was probably arrived at... PHB: Hey, how many lines of code do we have? CodeMonkey: You want a high estimate or a low estimate? PHB: High CodeMonkey: Well, including unit tests, comments, whitespace, build scripts, integration test harness... 170k At least I hope they have enough testing code to be signficant...
h0l0cube 6y ago
Having seen Matlab code that's been exported to C, it's mostly bloated by static arrays, and there's quite a lot of redundant functions. So maybe the LOC count is from a C export?
willis936 6y ago
I’ve made 4 decent sized gui based tools before. I would never use GUIDE. At minimum it uses eval. If you want a terrible gui-based gui creation interface: labview awaits.
- knolan 6y ago
  I compiled eval once for fun.
  - willis936 6y ago
    A friend of mine did that. They were able to run arbitrary .m files from an executable. As far as I know, this doesn't violate any terms of use. I wonder what keeps someone from just downloading the freely available MATLAB runtime and running a compiled eval wrapper and suddenly have a freely available version of MATLAB. Also just because eval can be compiled doesn't mean it should. It will forever be a security risk and I will not write code that uses it in good conscious. Fortunately, Mathworks has provided good alternatives. My personal favorite has been variable field names. It really opens up a lot of elegant coding. https://www.mathworks.com/help/matlab/matlab_prog/string-evaluation.html https://www.mathworks.com/help/matlab/matlab_prog/string-eva...

Blikkentrekker 6y ago

It indeed doesn't seem fair to them that they are forced to share their trade secrets simply because the prosecutors elected to use their software.

hansel_der 6y ago
it doesn't seem fair to sentence someone (to death?) based on a unaudited secret sauce tool. the company can always retract their product if they want to keep it a trade secret.
- Blikkentrekker 6y ago
  You assume it to be one or the other. The real solution would obviously that the prosecutors cannot enter into evidence the conclusions of any closed source software. Of course, this only displaces the issue, what of the black box c.p.u.'s whereupon this software ran?