5 ms·

It's funny, I just assume open source security oriented development would follow far more rigorous testing and verification than it apparently does. I guess it

It's funny, I just assume open source security oriented development would follow far more rigorous testing and verification than it apparently does. I guess it's no different from every other kind of development - testing and verification are not the fun part. Human incentives being what they are, this keeps playing out. Bounties are probably the only practical way to reverse those incentives because making that cash makes testing and verification fun again.

Alupis 12y ago
Even at big shops... testing is not fun and not done as much as you'd [want to] think.
- jacquesm 12y ago
  Proposal: When something is completed according to the devs a bonus is posted. For every vulnerability discovered by all the other devs in the company a portion of the bonus gets paid out. If an 'x' period has passed and no vulnerabilities have been discovered the remainder is paid out to the original devs.
  - contingencies 12y ago
    Your proposal is interesting in an office dynamic sense, however I feel that the problem is really time pressure, not a lack of quality pressure. Additional carrot (financial reward) for programmers is always good, but perhaps a more meaningful range of attacks would only be made by experienced external consultants... and the size of the fund must then be large enough to interest them. For fun, I just duckduckwent 'make a game of it' and got some amusing responses in the context of this suggestion. Kids can make a game out of almost anything! See more about gross motor activities, outdoor play and pool noodles. Whatever you'd like to improve about your life, try making it into a game. Challenge yourself. Make up your own rules. Then play it to win. When the family gets so big that you no longer can buy holiday gifts for everyone, you have to find an alternative. For many, that turns the gift exchange into a rousing game. "We used to buy for everyone and I enjoyed that, but in the last couple of years, we had to face the fact ... I decided to put together my own Iron Rations and make a game of it. You are welcome to play. I hope you have some helpful ideas to add to the stash. The rules are: Everything must be shelf safe with an expiration date or best by date of at least two years. Principal Skinner: Oh, licking envelopes can be fun! All you have to do is make a game of it. Bart: What kind of game? Principal Skinner: Well, for example, you could see how many you could lick in an hour, then try to break that record.
    - bayonetz 12y ago
      Perhaps but there are so many more actually fun "games" than testing your code that you are again fighting natural incentives - people will pick the most fun game when given the choice of many, yes?
      - contingencies 12y ago
        With reference to the parent's suggestion, 'winning' the game means getting paid. Hence game theory :)
  - bayonetz 12y ago
    Agree but then of course who will put up the bonus money? Could be a great philanthropic outreach opportunity for Google or other big company.
- Touche 12y ago
  Maybe I'm weird but writing a test, seeing it fail, fixing the bug, and then seeing it pass is quite delightful. Having a set of 100+ test, doing a large refactoring and seeing them all pass, is a completely joy. Then again, I also like writing documentation.
  - Alupis 12y ago
    A diamond in the rough you are ;-P
sp332 12y ago
OpenSSL was such a painful codebase to work on that people quickly gave up trying to test it. https://twitter.com/OpenSSLFact https://twitter.com/OpenSSLFact Also their memory-management wrappers made it impossible for valgrind etc. to even see which memory was "used" and "freed" inside the program. http://www.openbsd.org/papers/bsdcan14-libressl/mgp00007.html http://www.openbsd.org/papers/bsdcan14-libressl/mgp00007.htm...
- Alupis 12y ago
  Well, this is sort of fudd being spread by the libressl project. Most old and large codebases will have a mix of coding styles, etc... even at big companies (take a gander at the recently posted Windows 2000 source code for example, or the leaked Half Life 2 codebase, etc). Does it make testing more difficult with preferred tool X? Maybe... but perhaps tool X isn't the best fit for the codebase? Or perhaps minor changes could be made to make it work well. In any event... part of being a developer is being able to read and understand a variety of coding styles... since, coding styles have zero effect on the code's purpose/execution. Not trying to detract... but we should remember the people who complained about the coding styles and difficultly in reading/understanding the codebase were new to the codebase, and the codebase was not in their expected format... which is normal when you sit down to any new codebase. Also... this is a crypto codebase... so there will be an element that is always difficult for an outsider to read and understand.
  - sp332 12y ago
    It's not a complaint about coding styles. It's a complaint about the code's "purpose/execution". Heartbleed, for example!
  - kevingadd 12y ago
    The OpenSSL developers intentionally breaking malloc/free and making other ridiculous engineering decisions is not 'mix of coding styles', it's literally incompetence.
    - Alupis 12y ago

Alupis 12y ago

Even at big shops... testing is not fun and not done as much as you'd [want to] think.

jacquesm 12y ago
Proposal: When something is completed according to the devs a bonus is posted. For every vulnerability discovered by all the other devs in the company a portion of the bonus gets paid out. If an 'x' period has passed and no vulnerabilities have been discovered the remainder is paid out to the original devs.
- contingencies 12y ago
  Your proposal is interesting in an office dynamic sense, however I feel that the problem is really time pressure, not a lack of quality pressure. Additional carrot (financial reward) for programmers is always good, but perhaps a more meaningful range of attacks would only be made by experienced external consultants... and the size of the fund must then be large enough to interest them. For fun, I just duckduckwent 'make a game of it' and got some amusing responses in the context of this suggestion. Kids can make a game out of almost anything! See more about gross motor activities, outdoor play and pool noodles. Whatever you'd like to improve about your life, try making it into a game. Challenge yourself. Make up your own rules. Then play it to win. When the family gets so big that you no longer can buy holiday gifts for everyone, you have to find an alternative. For many, that turns the gift exchange into a rousing game. "We used to buy for everyone and I enjoyed that, but in the last couple of years, we had to face the fact ... I decided to put together my own Iron Rations and make a game of it. You are welcome to play. I hope you have some helpful ideas to add to the stash. The rules are: Everything must be shelf safe with an expiration date or best by date of at least two years. Principal Skinner: Oh, licking envelopes can be fun! All you have to do is make a game of it. Bart: What kind of game? Principal Skinner: Well, for example, you could see how many you could lick in an hour, then try to break that record.
  - bayonetz 12y ago
    Perhaps but there are so many more actually fun "games" than testing your code that you are again fighting natural incentives - people will pick the most fun game when given the choice of many, yes?
    - contingencies 12y ago
      With reference to the parent's suggestion, 'winning' the game means getting paid. Hence game theory :)
- bayonetz 12y ago
  Agree but then of course who will put up the bonus money? Could be a great philanthropic outreach opportunity for Google or other big company.
Touche 12y ago
Maybe I'm weird but writing a test, seeing it fail, fixing the bug, and then seeing it pass is quite delightful. Having a set of 100+ test, doing a large refactoring and seeing them all pass, is a completely joy. Then again, I also like writing documentation.
- Alupis 12y ago
  A diamond in the rough you are ;-P

sp332 12y ago

OpenSSL was such a painful codebase to work on that people quickly gave up trying to test it. https://twitter.com/OpenSSLFact https://twitter.com/OpenSSLFact Also their memory-management wrappers made it impossible for valgrind etc. to even see which memory was "used" and "freed" inside the program. http://www.openbsd.org/papers/bsdcan14-libressl/mgp00007.html http://www.openbsd.org/papers/bsdcan14-libressl/mgp00007.htm...

Alupis 12y ago
Well, this is sort of fudd being spread by the libressl project. Most old and large codebases will have a mix of coding styles, etc... even at big companies (take a gander at the recently posted Windows 2000 source code for example, or the leaked Half Life 2 codebase, etc). Does it make testing more difficult with preferred tool X? Maybe... but perhaps tool X isn't the best fit for the codebase? Or perhaps minor changes could be made to make it work well. In any event... part of being a developer is being able to read and understand a variety of coding styles... since, coding styles have zero effect on the code's purpose/execution. Not trying to detract... but we should remember the people who complained about the coding styles and difficultly in reading/understanding the codebase were new to the codebase, and the codebase was not in their expected format... which is normal when you sit down to any new codebase. Also... this is a crypto codebase... so there will be an element that is always difficult for an outsider to read and understand.
- sp332 12y ago
  It's not a complaint about coding styles. It's a complaint about the code's "purpose/execution". Heartbleed, for example!
- kevingadd 12y ago
  The OpenSSL developers intentionally breaking malloc/free and making other ridiculous engineering decisions is not 'mix of coding styles', it's literally incompetence.
  - Alupis 12y ago