6 ms·

That's not what he said at all and your comment dips into "the elite few know more than you do" territory, particularly with the unnecessary dig at Web developm

That's not what he said at all and your comment dips into "the elite few know more than you do" territory, particularly with the unnecessary dig at Web development. His only mention of TDD was "even if you don't," yet you jumped on the acronym TDD and took him to task for it. What I read him as saying (and agree with) is that he's surprised that GnuTLS isn't tested against known-bad certificates in a simple integration test that doesn't require a Ph.D. to set up, as you imply.

tptacek 13y ago
What "known-bad" certificate? You appear to be suffering from hindsight bias. We know what the bad certs look like now, but they're idiosyncratically related to a very, very specific bug: having the test case that checks for this bug implies that you don't have the bug to begin with. The fallacy that you could have easily written a test case for this bug appears neatly encapsulates the weakness of "TDD" as a mitigation for programming flaws.
- jsmthrowaway 13y ago
  My point wasn't the specificity of the argument, merely that you had misinterpreted the argument and gone after its originator misguidedly, to the conversation's detriment, something you're continuing to do with me by accusing me of hindsight bias. Your résumé is well-known around here and you don't have to set the terms of the discussion about every security-related issue on Hacker News, particularly when it's this heavy-handed and you get recognition upvotes. All I'm saying.
  - tptacek 13y ago
    I no longer have any idea what you're talking about. If you want to pose an argument about how TDD applies to building things like TLS stacks, I will read, consider, and respond. Debating TDD for TLS stacks isn't a waste of time, even if I do think the debate has a foregone conclusion. If you want to talk about me personally, I'm not interested.
    - jsmthrowaway 13y ago
      Okay, small words: Him: Can't you test for this somehow? Even without TDD... You: TDD is bad because math and wall of text. Go read a book and realize why you can't, Web hacker. Me: You jumped on TDD unnecessarily, there. You: Now I'm going to redirect the argument to you! You're browbeating anybody that comments here, rather unnecessarily, almost as a display of expertise.
      - tptacek 13y ago
        I'll address the sliver of this comment that is actually relevant to the thread: the distinction between "testing" and "TDD" is not important to the point I'm making.
        jsmthrowaway 13y ago
        Reading your original reply, I walk away with these four points from your four paragraphs, now with the knowledge that wherever you typed "TDD" you could mean "testing" as well. - Testing distributed cryptography is difficult. - Read a book. - You're a web developer and don't know the first thing about testing TLS stacks, clearly. - Testing does not improve systems security, as evidenced by Ruby on Rails: Rails uses testing and it's insecure. Seriously, re-read your comment. More than half of it is just unnecessary and dilutes what little point you've made into something unrecognizable in the snark.
        sillysaurus3 13y ago
        Testing does not improve systems security, as evidenced by Ruby on Rails: Rails uses testing and it's insecure. This is an important and valuable point. He wasn't discouraging new ideas, but rather pointing out how hard it is to make new ideas practical in the security arena.
        pg_is_a_butt 13y ago
        you're not making any point at all... you appear to be an idiot. if code can test a certificate, a certificate could trivially be created to test the code. "wah wah wah... so many different tests... it's too hard... wah wah wah" go suck a duck, lady. it's 2014. automated testing is a thing. it's dead simple. you're dumb.
rmc 13y ago
There are many ways for a cert to be bad.
cube13 13y ago
>What I read him as saying (and agree with) is that he's surprised that GnuTLS isn't tested against known-bad certificates in a simple integration test that doesn't require a Ph.D. to set up, as you imply. Define "known-bad" in a general enough way that a specific test can be created to cover the entire range of "bad" certs. That's quite difficult, and, probably isn't realistically possible to go through all the "known-bad" if you want your tests to run quickly. Realistically, all you can do is have regression tests to make sure that the found bugs aren't repeated in future releases.
- jsmthrowaway 13y ago
  My thoughts on this are more about the previous TLS disclosure, not this one, where an integration test that says "this invalid certificate should fail validation" probably would have caught it. But, this isn't my argument to make as I don't know enough about it, I just want to see the actual coherent argument made instead of a side tangent as we're seeing here.
  - tptacek 13y ago
    What integration test? Writing that integration test implies that you know the bug is a possibility. Once you consider that possibility, a glance at the code tells you if it's there or not. It's not like this bug is likely to regress, either.
    - jsmthrowaway 13y ago
      Again, I'm talking about the "goto fail" one, not this one, in which a simple test of the library against an invalid certificate would have caught at least one case that I recall. And I think it's pretty obvious that "you might erroneously validate a bad certificate" is a bug that's, you know, a given.
      - tptacek 13y ago
        Feature: *Certificate Validation* In order to *keep NSA from reading my emai* As a *TLS X.509 validation library* I want *to never erroneously validate a certificate* What are the "Scenarios"? Given: *???* And: *???* When: *???* Then: *the certificate should be rejected* Remember, if we're switching topics to the "goto fail" bug: that bug didn't affect every instance of certificate validation. You had to be in a particular set of ciphersuites.

tptacek 13y ago

What "known-bad" certificate? You appear to be suffering from hindsight bias. We know what the bad certs look like now, but they're idiosyncratically related to a very, very specific bug: having the test case that checks for this bug implies that you don't have the bug to begin with. The fallacy that you could have easily written a test case for this bug appears neatly encapsulates the weakness of "TDD" as a mitigation for programming flaws.

jsmthrowaway 13y ago
My point wasn't the specificity of the argument, merely that you had misinterpreted the argument and gone after its originator misguidedly, to the conversation's detriment, something you're continuing to do with me by accusing me of hindsight bias. Your résumé is well-known around here and you don't have to set the terms of the discussion about every security-related issue on Hacker News, particularly when it's this heavy-handed and you get recognition upvotes. All I'm saying.
- tptacek 13y ago
  I no longer have any idea what you're talking about. If you want to pose an argument about how TDD applies to building things like TLS stacks, I will read, consider, and respond. Debating TDD for TLS stacks isn't a waste of time, even if I do think the debate has a foregone conclusion. If you want to talk about me personally, I'm not interested.
  - jsmthrowaway 13y ago
    Okay, small words: Him: Can't you test for this somehow? Even without TDD... You: TDD is bad because math and wall of text. Go read a book and realize why you can't, Web hacker. Me: You jumped on TDD unnecessarily, there. You: Now I'm going to redirect the argument to you! You're browbeating anybody that comments here, rather unnecessarily, almost as a display of expertise.
    - tptacek 13y ago
      I'll address the sliver of this comment that is actually relevant to the thread: the distinction between "testing" and "TDD" is not important to the point I'm making.
      - jsmthrowaway 13y ago
        Reading your original reply, I walk away with these four points from your four paragraphs, now with the knowledge that wherever you typed "TDD" you could mean "testing" as well. - Testing distributed cryptography is difficult. - Read a book. - You're a web developer and don't know the first thing about testing TLS stacks, clearly. - Testing does not improve systems security, as evidenced by Ruby on Rails: Rails uses testing and it's insecure. Seriously, re-read your comment. More than half of it is just unnecessary and dilutes what little point you've made into something unrecognizable in the snark.
        sillysaurus3 13y ago
        Testing does not improve systems security, as evidenced by Ruby on Rails: Rails uses testing and it's insecure. This is an important and valuable point. He wasn't discouraging new ideas, but rather pointing out how hard it is to make new ideas practical in the security arena.
      - pg_is_a_butt 13y ago
        you're not making any point at all... you appear to be an idiot. if code can test a certificate, a certificate could trivially be created to test the code. "wah wah wah... so many different tests... it's too hard... wah wah wah" go suck a duck, lady. it's 2014. automated testing is a thing. it's dead simple. you're dumb.

rmc 13y ago

There are many ways for a cert to be bad.

cube13 13y ago

>What I read him as saying (and agree with) is that he's surprised that GnuTLS isn't tested against known-bad certificates in a simple integration test that doesn't require a Ph.D. to set up, as you imply. Define "known-bad" in a general enough way that a specific test can be created to cover the entire range of "bad" certs. That's quite difficult, and, probably isn't realistically possible to go through all the "known-bad" if you want your tests to run quickly. Realistically, all you can do is have regression tests to make sure that the found bugs aren't repeated in future releases.

jsmthrowaway 13y ago
My thoughts on this are more about the previous TLS disclosure, not this one, where an integration test that says "this invalid certificate should fail validation" probably would have caught it. But, this isn't my argument to make as I don't know enough about it, I just want to see the actual coherent argument made instead of a side tangent as we're seeing here.
- tptacek 13y ago
  What integration test? Writing that integration test implies that you know the bug is a possibility. Once you consider that possibility, a glance at the code tells you if it's there or not. It's not like this bug is likely to regress, either.
  - jsmthrowaway 13y ago
    Again, I'm talking about the "goto fail" one, not this one, in which a simple test of the library against an invalid certificate would have caught at least one case that I recall. And I think it's pretty obvious that "you might erroneously validate a bad certificate" is a bug that's, you know, a given.
    - tptacek 13y ago
      Feature: *Certificate Validation* In order to *keep NSA from reading my emai* As a *TLS X.509 validation library* I want *to never erroneously validate a certificate* What are the "Scenarios"? Given: *???* And: *???* When: *???* Then: *the certificate should be rejected* Remember, if we're switching topics to the "goto fail" bug: that bug didn't affect every instance of certificate validation. You had to be in a particular set of ciphersuites.