5 ms·

> I don't do this out of principle, but because I've never managed to get an implementation of a primitive right without it. My journey looked a bit like this.

> I don't do this out of principle, but because I've never managed to get an implementation of a primitive right without it. My journey looked a bit like this. First implement all the thing! Cool it compiles! I'm tired… Okay, let's do the test vectors, because crypto-critical-yada-yada. Aw crap. It was all broken. Okay, let's dig it up… By the way, one worry I have right now is that I may not have enough test vectors. Do they hit all the code paths? Do they sufficiently test for overflow? That's why I display the number of test vectors involve in each tests when running the test suite.

tptacek 10y ago
Are you maybe using the term "test vectors" differently than we are? The test vectors for a crypto construction are just a series of test transforms, provided to make sure you implemented the same transform function as everyone else. They're not a suite of security checks. Most of the most serious misuse errors with crypto constructions don't have anything like "test vectors"; that's what was so exciting about Google's Wycheproof.
- loup-vaillant 10y ago
  > Are you maybe using the term "test vectors" differently than we are? I don't think so. Check out the test suite in my tarball, there are files named "vectors", with sets of inputs and expected outputs. In addition, I have written consistency checks for all constructions (encryption-decryption roundtrip yields the original plaintext, legitimate messages are accepted, forgeries are rejected…) I feel I should do more, but I'm not sure what exactly.
  - tptacek 10y ago
    Right, then, so these aren't really security tests. I'm suggesting you reword that bit on your page; passing test vectors doesn't make something more trustworthy; it's table stakes. I'm not saying you shouldn't write tests for misuse cases! You should do that.
    - loup-vaillant 10y ago
      Okay, what's a security test? How is it any different from a mere correctness test? Can you suggest some link to read up, or some tool to use?
      - arnarbi 10y ago
        See e.g. https://github.com/google/wycheproof https://github.com/google/wycheproof Fuzzing tests can also uncover weird bugs and edge cases that can lead to vulnerabilities.

tptacek 10y ago

Are you maybe using the term "test vectors" differently than we are? The test vectors for a crypto construction are just a series of test transforms, provided to make sure you implemented the same transform function as everyone else. They're not a suite of security checks. Most of the most serious misuse errors with crypto constructions don't have anything like "test vectors"; that's what was so exciting about Google's Wycheproof.

loup-vaillant 10y ago
> Are you maybe using the term "test vectors" differently than we are? I don't think so. Check out the test suite in my tarball, there are files named "vectors", with sets of inputs and expected outputs. In addition, I have written consistency checks for all constructions (encryption-decryption roundtrip yields the original plaintext, legitimate messages are accepted, forgeries are rejected…) I feel I should do more, but I'm not sure what exactly.
- tptacek 10y ago
  Right, then, so these aren't really security tests. I'm suggesting you reword that bit on your page; passing test vectors doesn't make something more trustworthy; it's table stakes. I'm not saying you shouldn't write tests for misuse cases! You should do that.
  - loup-vaillant 10y ago
    Okay, what's a security test? How is it any different from a mere correctness test? Can you suggest some link to read up, or some tool to use?
    - arnarbi 10y ago
      See e.g. https://github.com/google/wycheproof https://github.com/google/wycheproof Fuzzing tests can also uncover weird bugs and edge cases that can lead to vulnerabilities.