12 ms·
Crev – Socially scalable Code REView and recommendation system
- hn_throwaway_99 5y agoI think this is really cool. On the recent NPM thread I stated that I want a feature where I can get the latest semver compatible version that is at least N days old. The idea being that I want to stay current with my dependencies, but I don't want to be on the bleeding edge. This is even better: I only want to have a version that has a minimum level of a verified "reputation score". I could perhaps even treat it similar to a Blockchain new block validation: I want to wait until at least X validations from people with a reputation score of at least Y vouch for the package.
- yknx4 5y agoYou should try renovate, it has a self hosted option for is open source. And allows you to manage your dependencies with certain constraints, one of those is "days it's been stable/released". So you can set it up to automatically create a pull request with the updated dependencies that matches your criteria
- fallat 5y agoI'm about to type something ... interesting * Bugs → Bad * Less bugs → Good * Less bugs → Code reviewed * Less bugs → Valuable * Code review → Valuable * GPG → Digital signatures * GPG → Digital identity with database (key servers) * Signed code review → Proof code was reviewed by an entity * Signed code reviews → Valuable * Signed code reviews → Identity verifiable, proof verifiable * Signed code reviews → Can calculate reputation across GitHub or similar * Signed code reviews which can have their ownership transfered via a new proof from the original author can "sell" these reviews to people. Yes, I'm talking NFTs. As I was writing this, you can also replace "code review" with just general "claim". Claim you wrote a book, or wrote some special software, or thought of a new mathematical idea before anyone else. From here, people can make a living making and selling code reviews / claims. "Proof of able to do useful work"
- vlovich123 5y agoHow do you solve the ghost writer problem? The capitalist version to cheating on your homework that we’re taught to believe is bad in academia but totally ok in the real world.
- charcircuit 5y agoWhat's wrong with having a group of people do code review?
- pas 5y agoPaying 100 people to write "lgtm!" mindlessly for your profit is ... while code review, its false positive rate is problematic :) Obviously the solution is a meta-reputation system with staking (ie. reverse bounties and when it turns out someone missed a bug they lose their stake, etc), and ... and a web of trust too! Oh and I want a pet dragon too!
- dane-pgp 5y ago> reverse bounties and when it turns out someone missed a bug they lose their stake, etc That's a really good insight into the sort of implementation that's needed. I'm wondering how we would design the incentive structure, and maybe gamify things, to encourage positive participation. For example, would we need people whose job it is to deliberately inject underhanded code to try catching out / testing the reviewers? These pen-testers would need a way to pre-commit to the fact that they were trying this (so an attacker couldn't turn around and say "just testing!" when they got caught), and a way to guarantee the underhanded code doesn't reach production even if the reviewers don't catch it. I still think the distributed reputation system is the hardest part (and would be one of the most useful things to happen to the web since TLS), but there definitely seems to be other hard problems ahead after that.
- pas 5y agoI think first we should simply try the good old altruistic build it and they'll come approach. Simply build a transparent, simple, easy to use system and get an agreeable group of people to start using it. I mean authors/contributors of top 1000 of npm packages. We already very explicitly trust them. (Sure not exactly for this kind of thing, but if there were a yarn/npm plugin that shows relevant GitHub code review comments, people might start writing those code review comments somewhere.) cargo-crev is very barebones, a bit hard to use, but the concept seems solid. https://web.crev.dev/rust-reviews/ https://web.crev.dev/rust-reviews/ > For example, would we need people whose job it is to deliberately inject underhanded code to try catching out / testing the reviewers? That seems ethically iffy. (Reminds me of the university study (from last year?) when they tried to push bugs into the Linux kernel on purpose to test this.) I guess life creates enough natural experiments for us anyway, so there's enough variance that allows for fishing for a signal. But of course with consent of the involved, sure, why not. It'd be a nice high-high-high level addition to the Simian Army. > doesn't reach production even if the reviewers don't catch it I think that's the hard part. If they commit to introducing a vulnerability, it has to be a vulnerability for the test to make sense, and ... at that point there's going to be a time window where it's present in the codebase. Of course it might make sense for projects that have a known release cadence and/or some kind of security policy which allows for notifying the maintainers and potentially all downstream affected, etc. > I still think the distributed reputation system is the hardest part (and would be one of the most useful things to happen to the web since TLS), but there definitely seems to be other hard problems ahead after that. Agreed :)
- zokier 5y agoOne idea that I got with all these posts popping around is that could we recycle the "distro" paradigm to work with these language ecosystems? Basically that there would be dedicated group of people doing curation, maintenance, and integration work, being essentially a bridge between users and upstreams. I think Anaconda is pretty much exactly this idea, so it's not entirely novel concept. If you squint a bit, you could also see traditional linux distros being this sort of thing for the C ecosystem and then gotten extended to everything. Big question is how could we encourage more of those? Because package maintenance I imagine is pretty unglamorous job, getting volunteers could be tricky. But maybe these problems with npm etc are becoming so aggravated that enterprises could be willing to chip in?
- sanxiyn 5y agoThis is basically Haskell's Stackage. It works! Re getting volunteers for packaging: Debian exists, so it must be possible. In fact, not only Debian exists, but Debian has >1000 members. It's a matter of organization.
- pornel 5y agoSo cargo-crev is effectively such a distro, but also tackles the next questions: how do you find maintainers? How do you find maintainers you can trust? The answers are: anybody can review and approve packages, but you select a trustworthy subset through a web of trust. And you can adjust trust requirements to be anything between "better than nothing" and "full paranoid". A traditional distro is a central authority, with fixed policies, and they don't have to agree with you which packages are worth including and what level of scrutiny they get. If you want something that's not a part of the distro, you're on your own. With the crev system you can organize or sponsor reviews of any packages you want. There can be multiple sets of approved packages, and you can decide yourself whether e.g. packages approved by an NSA employee are checked for good cryptography, or an evil govt backdoor.
- sulam 5y agoThis is the better link IMO: https://github.com/crev-dev/crev https://github.com/crev-dev/crev