4 ms·

I'm about to type something ... interesting * Bugs → Bad * Less bugs → Good * Less bugs → Code reviewed * Less bugs → Valuable * Code review → Valuab

by fallat 5y ago

I'm about to type something ... interesting * Bugs → Bad * Less bugs → Good * Less bugs → Code reviewed * Less bugs → Valuable * Code review → Valuable * GPG → Digital signatures * GPG → Digital identity with database (key servers) * Signed code review → Proof code was reviewed by an entity * Signed code reviews → Valuable * Signed code reviews → Identity verifiable, proof verifiable * Signed code reviews → Can calculate reputation across GitHub or similar * Signed code reviews which can have their ownership transfered via a new proof from the original author can "sell" these reviews to people. Yes, I'm talking NFTs. As I was writing this, you can also replace "code review" with just general "claim". Claim you wrote a book, or wrote some special software, or thought of a new mathematical idea before anyone else. From here, people can make a living making and selling code reviews / claims. "Proof of able to do useful work"

vlovich123 5y ago
How do you solve the ghost writer problem? The capitalist version to cheating on your homework that we’re taught to believe is bad in academia but totally ok in the real world.
- charcircuit 5y ago
  What's wrong with having a group of people do code review?
  - pas 5y ago
    Paying 100 people to write "lgtm!" mindlessly for your profit is ... while code review, its false positive rate is problematic :) Obviously the solution is a meta-reputation system with staking (ie. reverse bounties and when it turns out someone missed a bug they lose their stake, etc), and ... and a web of trust too! Oh and I want a pet dragon too!
    - dane-pgp 5y ago
      > reverse bounties and when it turns out someone missed a bug they lose their stake, etc That's a really good insight into the sort of implementation that's needed. I'm wondering how we would design the incentive structure, and maybe gamify things, to encourage positive participation. For example, would we need people whose job it is to deliberately inject underhanded code to try catching out / testing the reviewers? These pen-testers would need a way to pre-commit to the fact that they were trying this (so an attacker couldn't turn around and say "just testing!" when they got caught), and a way to guarantee the underhanded code doesn't reach production even if the reviewers don't catch it. I still think the distributed reputation system is the hardest part (and would be one of the most useful things to happen to the web since TLS), but there definitely seems to be other hard problems ahead after that.
      - pas 5y ago
        I think first we should simply try the good old altruistic build it and they'll come approach. Simply build a transparent, simple, easy to use system and get an agreeable group of people to start using it. I mean authors/contributors of top 1000 of npm packages. We already very explicitly trust them. (Sure not exactly for this kind of thing, but if there were a yarn/npm plugin that shows relevant GitHub code review comments, people might start writing those code review comments somewhere.) cargo-crev is very barebones, a bit hard to use, but the concept seems solid. https://web.crev.dev/rust-reviews/ https://web.crev.dev/rust-reviews/ > For example, would we need people whose job it is to deliberately inject underhanded code to try catching out / testing the reviewers? That seems ethically iffy. (Reminds me of the university study (from last year?) when they tried to push bugs into the Linux kernel on purpose to test this.) I guess life creates enough natural experiments for us anyway, so there's enough variance that allows for fishing for a signal. But of course with consent of the involved, sure, why not. It'd be a nice high-high-high level addition to the Simian Army. > doesn't reach production even if the reviewers don't catch it I think that's the hard part. If they commit to introducing a vulnerability, it has to be a vulnerability for the test to make sense, and ... at that point there's going to be a time window where it's present in the codebase. Of course it might make sense for projects that have a known release cadence and/or some kind of security policy which allows for notifying the maintainers and potentially all downstream affected, etc. > I still think the distributed reputation system is the hardest part (and would be one of the most useful things to happen to the web since TLS), but there definitely seems to be other hard problems ahead after that. Agreed :)
divs1210 5y ago
I have been thinking about this a lot lately.
okl 5y ago
> Code review → Valuable The activity in itself is not valuable, and code does not become better just because somebody looks at it. Identifying and subsequently fixing flaws -- that's what makes a code review worthwhile. Your concept more closely fits the idea of "certifying" code and in that case, as the review sponsor, you will interested in being able to hold someone accountable. And why would you want that liability to be transferable?
Communitivity 5y ago
There are some good people working on this within the Verifiable Credentials working group within the W3C, and the Decentralized Identity Foundation. - https://www.w3.org/2017/vc/WG/ https://www.w3.org/2017/vc/WG/ - https://identity.foundation/ https://identity.foundation/

by fallat 5y ago

vlovich123 5y ago

How do you solve the ghost writer problem? The capitalist version to cheating on your homework that we’re taught to believe is bad in academia but totally ok in the real world.

charcircuit 5y ago
What's wrong with having a group of people do code review?
- pas 5y ago
  Paying 100 people to write "lgtm!" mindlessly for your profit is ... while code review, its false positive rate is problematic :) Obviously the solution is a meta-reputation system with staking (ie. reverse bounties and when it turns out someone missed a bug they lose their stake, etc), and ... and a web of trust too! Oh and I want a pet dragon too!
  - dane-pgp 5y ago
    > reverse bounties and when it turns out someone missed a bug they lose their stake, etc That's a really good insight into the sort of implementation that's needed. I'm wondering how we would design the incentive structure, and maybe gamify things, to encourage positive participation. For example, would we need people whose job it is to deliberately inject underhanded code to try catching out / testing the reviewers? These pen-testers would need a way to pre-commit to the fact that they were trying this (so an attacker couldn't turn around and say "just testing!" when they got caught), and a way to guarantee the underhanded code doesn't reach production even if the reviewers don't catch it. I still think the distributed reputation system is the hardest part (and would be one of the most useful things to happen to the web since TLS), but there definitely seems to be other hard problems ahead after that.
    - pas 5y ago
      I think first we should simply try the good old altruistic build it and they'll come approach. Simply build a transparent, simple, easy to use system and get an agreeable group of people to start using it. I mean authors/contributors of top 1000 of npm packages. We already very explicitly trust them. (Sure not exactly for this kind of thing, but if there were a yarn/npm plugin that shows relevant GitHub code review comments, people might start writing those code review comments somewhere.) cargo-crev is very barebones, a bit hard to use, but the concept seems solid. https://web.crev.dev/rust-reviews/ https://web.crev.dev/rust-reviews/ > For example, would we need people whose job it is to deliberately inject underhanded code to try catching out / testing the reviewers? That seems ethically iffy. (Reminds me of the university study (from last year?) when they tried to push bugs into the Linux kernel on purpose to test this.) I guess life creates enough natural experiments for us anyway, so there's enough variance that allows for fishing for a signal. But of course with consent of the involved, sure, why not. It'd be a nice high-high-high level addition to the Simian Army. > doesn't reach production even if the reviewers don't catch it I think that's the hard part. If they commit to introducing a vulnerability, it has to be a vulnerability for the test to make sense, and ... at that point there's going to be a time window where it's present in the codebase. Of course it might make sense for projects that have a known release cadence and/or some kind of security policy which allows for notifying the maintainers and potentially all downstream affected, etc. > I still think the distributed reputation system is the hardest part (and would be one of the most useful things to happen to the web since TLS), but there definitely seems to be other hard problems ahead after that. Agreed :)

divs1210 5y ago

I have been thinking about this a lot lately.

okl 5y ago

> Code review → Valuable The activity in itself is not valuable, and code does not become better just because somebody looks at it. Identifying and subsequently fixing flaws -- that's what makes a code review worthwhile. Your concept more closely fits the idea of "certifying" code and in that case, as the review sponsor, you will interested in being able to hold someone accountable. And why would you want that liability to be transferable?

Communitivity 5y ago

There are some good people working on this within the Verifiable Credentials working group within the W3C, and the Decentralized Identity Foundation. - https://www.w3.org/2017/vc/WG/ https://www.w3.org/2017/vc/WG/ - https://identity.foundation/ https://identity.foundation/