4 ms·

Comparing multiple implementations is a great way to find bugs. Even running the same code with different compilers or platforms helps find bugs. It doesn't fi

by illumen 12y ago

Comparing multiple implementations is a great way to find bugs. Even running the same code with different compilers or platforms helps find bugs. It doesn't find cases where they are both wrong, but often finds cases where one of them is wrong. It is a very common testing strategy with maths libraries.

nickik 12y ago
You might be intrested in this [1]. Its a 31C3 talk about exactly this. There idea is to write a runable spec (that only focues on correctness, instead of speed and things like that) that then can be used to test against real world application. There idea is to do this against all the MAJOR interfaces that we all depend on TCP/IP, x86 memory model and stuff like that. [1] http://events.ccc.de/congress/2014/Fahrplan/events/6574.html http://events.ccc.de/congress/2014/Fahrplan/events/6574.html (the talk is on youtube or on ccc video streaming side)
- Anderkent 12y ago
  The talk on youtube: https://www.youtube.com/watch?v=ca0DWaV9uNc https://www.youtube.com/watch?v=ca0DWaV9uNc (I couldn't find a link on that page)
  - aw3c2 12y ago
    That 'CCCen' account is an imposter and evil. Please do not link to it or the companion 'CCCdeVideos'. Some background on this issue: https://events.ccc.de/2015/01/03/the-youtube-and-stream-dump-problem/ https://events.ccc.de/2015/01/03/the-youtube-and-stream-dump... Here is the official CCC channel: https://www.youtube.com/user/mediacccde https://www.youtube.com/user/mediacccde The official recording: http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_saal_1_-_201412301245_-_why_are_computers_so_and_what_can_we_do_about_it_-_peter_sewell.html#video http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_... The official recording on their official YT channel: https://www.youtube.com/watch?v=MBIHPLFmcgA https://www.youtube.com/watch?v=MBIHPLFmcgA
- edwintorok 12y ago
  It would help if the videos were automatically linked from the event page: http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_saal_1_-_201412301245_-_why_are_computers_so_and_what_can_we_do_about_it_-_peter_sewell.html#video http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_...
- pjc50 12y ago
  Runnable/verifiable specs are a good idea. A colleague of mine did some great work with compiling the H265 & VP9 specifications to produce a test suite, in a similar manner to fuzzing: http://www.argondesign.com/products/2014/may/27/streamsvp9/ http://www.argondesign.com/products/2014/may/27/streamsvp9/
- pilsetnieks 12y ago
  > instead of speed Not when developing crypto. You have to look at execution speed, too, to prevent timing attacks.
  - raverbashing 12y ago
    Execution speed variance It doesn't matter if it takes 1s or 10s, but that it takes 10s every time (to verify a password, for example)
  - azdle 12y ago
    Why would that matter in the case of testing for correctness? I thought these 'runable spec' implementations were only to verify that the output of the 'real' implementation is correct and thus would never even be used anywhere that could/would be attacked. Am I not understanding the purpose correctly?
  - Igglyboo 12y ago
    When testing for correctness why would you care about timing attacks? You would obviously have multiple tests covering correctness, performance, security, etc.

by illumen 12y ago

nickik 12y ago

You might be intrested in this [1]. Its a 31C3 talk about exactly this. There idea is to write a runable spec (that only focues on correctness, instead of speed and things like that) that then can be used to test against real world application. There idea is to do this against all the MAJOR interfaces that we all depend on TCP/IP, x86 memory model and stuff like that. [1] http://events.ccc.de/congress/2014/Fahrplan/events/6574.html http://events.ccc.de/congress/2014/Fahrplan/events/6574.html (the talk is on youtube or on ccc video streaming side)

Anderkent 12y ago
The talk on youtube: https://www.youtube.com/watch?v=ca0DWaV9uNc https://www.youtube.com/watch?v=ca0DWaV9uNc (I couldn't find a link on that page)
- aw3c2 12y ago
  That 'CCCen' account is an imposter and evil. Please do not link to it or the companion 'CCCdeVideos'. Some background on this issue: https://events.ccc.de/2015/01/03/the-youtube-and-stream-dump-problem/ https://events.ccc.de/2015/01/03/the-youtube-and-stream-dump... Here is the official CCC channel: https://www.youtube.com/user/mediacccde https://www.youtube.com/user/mediacccde The official recording: http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_saal_1_-_201412301245_-_why_are_computers_so_and_what_can_we_do_about_it_-_peter_sewell.html#video http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_... The official recording on their official YT channel: https://www.youtube.com/watch?v=MBIHPLFmcgA https://www.youtube.com/watch?v=MBIHPLFmcgA
edwintorok 12y ago
It would help if the videos were automatically linked from the event page: http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_saal_1_-_201412301245_-_why_are_computers_so_and_what_can_we_do_about_it_-_peter_sewell.html#video http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_...
pjc50 12y ago
Runnable/verifiable specs are a good idea. A colleague of mine did some great work with compiling the H265 & VP9 specifications to produce a test suite, in a similar manner to fuzzing: http://www.argondesign.com/products/2014/may/27/streamsvp9/ http://www.argondesign.com/products/2014/may/27/streamsvp9/
pilsetnieks 12y ago
> instead of speed Not when developing crypto. You have to look at execution speed, too, to prevent timing attacks.
- raverbashing 12y ago
  Execution speed variance It doesn't matter if it takes 1s or 10s, but that it takes 10s every time (to verify a password, for example)
- azdle 12y ago
  Why would that matter in the case of testing for correctness? I thought these 'runable spec' implementations were only to verify that the output of the 'real' implementation is correct and thus would never even be used anywhere that could/would be attacked. Am I not understanding the purpose correctly?
- Igglyboo 12y ago
  When testing for correctness why would you care about timing attacks? You would obviously have multiple tests covering correctness, performance, security, etc.