8 ms·

Show HN: Crocodile - Better code review for GitHub

144 points by jameslao 4y ago

Hi HN! I've been working on a code review app for GitHub called Crocodile for about a year. I used to work at Microsoft where we used a tool called CodeFlow for reviewing code and I missed it after I left. I know many other ex-Microsoft engineers feel the same. Here are some of the distinguishing features of Crocodile that are inspired by CodeFlow: * Comments float above the code instead of being inline. Long discussions that are displayed inline make it really hard to review the code. * Comment on any text selection in the file, even a single character. * Comments don't get lost when code changes. I hate it when comments become "outdated" because I rebase or the line is edited. I also implemented lots of features that I wish CodeFlow had which you can read more about on the blog. [1] For those curious about the tech stack: it's mostly written in Go with Alpine.js, HTMX, and Tailwind CSS for the frontend. For storage I use PostgreSQL, S3 compatible object storage, and Redis for caching. I use Pulumi for infrastructure provisioning and Kubernetes deployments. Everything is hosted on DigitalOcean. Feedback is welcome! [1] https://www.crocodile.dev/blog/why-crocodile https://www.crocodile.dev/blog/why-crocodile

perpil 4y ago
Nit: Couple typos on landing page: coversation, shorcut. Worth taking a pass with a spell checker.
- jameslao 4y ago
  Thanks for pointing those out! Fixed.
silverlake 4y ago
“You can comment anywhere in the file and on any text selection, even if it's just one character.” I’ve been looking for this feature forever. A changed line might impact code that wasn’t changed. There wasn’t a convenient way to comment on those. Looking forward to trying this.
http-teapot 4y ago
I am a bit at odd with the pricing. My team has 6 engineers and base price for Github is $24. Your product would increase our bill by $48, an increase of 200%. $48 is nothing compared to the salaries of 6 engineers but I am not convinced the feature set would make my team more productive. If you told me that your solution help my team ship faster and saves an hour per engineer per week then that's easy math: your product pays for itself. Suggestions: - Make all base features free (the ones on your site currently) - Add analytics to your product, collect data and put it behind a paywall (entirely or partially by truncating historical data) - Iterate on premium features that improves critical metrics - Offer analytics with a trial of 2 to 3 months, enough time for graphs to speak for themselves - Make sure the gains are seen by the manager or business owner or whoever is the person in charge Pricing can be based on the average of hours saved.
- paulgb 4y ago
  I’m going to disagree, I think the model they have is both familiar to me as a potential user and priced such that, if it is even slightly better than GitHub, it’s a no-brainer. I wouldn’t pay extra for analytics, though, it just sounds like another thing to check. IMHO if the one-month free trial isn’t enough to know whether a SaaS solution is worth an $8/head price, it’s probably not worth that price.
  - tr1ll10nb1ll 4y ago
    For me, the most worthy SaaS purchases usually have been the ones that I knew were worth it from day 0. The trial has always just been a cherry on top. I haven’t ever used the entire trial for the sake of “trying” and then choosing to not use the product further.
  - OJFord 4y ago
    Adding Y to Xy is already a tough sell, but when the upgraded Y costs more than all of Xy? I sympathise with GP. GitHub can do it from an economy of scale and probably 'should' be more expensive, but it can manage not to be (maybe cheaper is actually optimal, given how many more will pay for it, point is it's in some sense 'worth' more) which puts OP in a tough spot. Imagine trying to sell a better WYSIWYG editor for WordPress for example. You can be better, save time, etc. - it's still always going to be a tough sell to charge more than a fraction of WP itself which offers a WYSIWYG editor and everything else.
burlesona 4y ago
This looks great! Food for thought on pricing: Because you offer free for open source, there’s minimal need for a free trial for private repos. I would consider requiring a credit card and making it a 14 day trial. My experience selling SaaS in the past is this will net you fewer but much more serious evals, so whenever someone signs up you know they are legit and worth your time to contact and support. That leads to higher conversion, better retention, etc. Best of luck!
- gingerlime 4y ago
  generally I agree, but a no-card trial removes friction. I have a few open source projects, but they don’t use much code review or at all compared to the private repo at my company. Yeah I can add a card, but it’s easier to start without it, see if there’s a fit and then decide.
paxys 4y ago
This looks fantastic! I'd love to try this out, but I don't have admin access to the repo I work in and IT will definitely not approve a random new app. Any chance I can set this up with a personal access token instead?
- jameslao 4y ago
  Thanks for taking a look! Yeah, not having admin access is definitely a problem. Using a PAT is an interesting idea that I'll have to think about that from a security and liability perspective.
  - igetspam 4y ago
    Just today I had to go through the work of auditing a GitHub app, for compliance reasons. Having the source available made that possible. Since your source isn't available, the barrier to entry would be a SOC2 report or you filling out a vendor survey. Compliance is a PITA but it's a real thing for companies. How are you planning on clearing that hurdle? I wouldn't want to put you through the vendor survey gauntlet and SOC2 is a lot to ask of a solo dev but the aforementioned IT team could likely be persuaded if you did Type 1 at least.
decebalus1 4y ago
Oh wow, this looks great. Congratulations!
searchableguy 4y ago
This provides better UX but adds one more redirection when you are using github. Do you support features like github suggestions which can be committed easily by author? Given your history with Microsoft and their internal tool which this is inspired by, how long until github copies some of the ideas? How many active users does this have? Are there potential problems you see outside of current product? I remember a startup providing paid code reviews as a service launched on HN a while ago. That could be a pivot for example in providing more value.
- jameslao 4y ago
  > Do you support features like github suggestions which can be committed easily by author? Not right now. The suggestion feature is something I want to add in the future if there's interest though! > Given your history with Microsoft and their internal tool which this is inspired by, how long until github copies some of the ideas? No idea! But CodeFlow had been around for many years before they acquired GitHub so they've had plenty of time to do it! I think this is an existential fear that anyone building code review tools has. For Crocodile, I think the long term strategy will be to decouple this from GitHub so that it can work with any source control provider. I architected it this way from the beginning but decided to target GitHub first because they have mindshare and because it made it really easy to dogfood since I host my code on GitHub. > Are there potential problems you see outside of current product? There's the one you already mentioned, which is GitHub improving their PR system, but there's not much I can do about that. Current problems I'm looking at are roadmap related. I have a few ideas of features I'd like to add, but no real indication on what to do next. Hence, posting on HN to get feedback. :) Thanks for taking a look and the feedback!
  - gingerlime 4y ago
    +1 for suggestions. It’s so much more helpful to see a diff and even be able to commit it.
  - jhogervorst 4y ago
    Another +1 for suggestions (with 'commit now' button). It's the thing I love most about GitHub PRs, and I wouldn't consider leaving it for a solution without it :-)
imron 4y ago
> * Comments float above the code > * Comment on any text selection in the file > * Comments don't get lost when code changes This addresses my main pet peeves with GitHub/Bitbucket reviews! 2 questions: With regards to comments not getting lost, how well does this work across rebases and force pushes? Are you considering supporting other products like Bitbucket and Gitlab?
- jameslao 4y ago
  Comments should get transferred just fine across rebases and force pushes. If they don't, you can let me know using the feedback form at the top right. Supporting other VCS providers, even non-Git ones, is on my list. The underlying data model is agnostic to GitHub so it's just a matter of having demand for it and doing the work!
catchmeifyoucan 4y ago
Congrats on the launch! Here's a demo link in case folks might have missed it: https://app.crocodile.dev/reviews/rwsfSKbgZoSt?change=README.md&iter=2 https://app.crocodile.dev/reviews/rwsfSKbgZoSt?change=README...
mtlynch 4y ago
This is cool! I'm happy to see more options in this space. The best code review tool I've ever used was a tool at Google called Critique.[0] They've open-sourced it as Gerrit[1], but there are sadly no hosted versions available for under $15k/yr, and it's complicated to self-host. I've been using Reviewable, and my experience has been good not great. Github's native code review has caught up a bit, but Github's review tool falls apart if your review lasts more than one round. Here are my gripes with Reviewable: Steep learning curve - Every new developer who joins the team spends their first few reviews being confused and frustrated by Reviewable. Performance - Reviewable has awful performance. It takes about 10 seconds to load a code review. It seems like it's doing some odd websockets stuff where sometimes my "connection" to Reviewable will drop and I can't add comments. I've never experienced this with any other web app. It's gotten better over the last few years, but it's still annoyingly frequent. Complicated configuration - I just want the reviewer to be able to hit an "LGTM" button to mark their approval. Reviewable's decision about when a PR is approved is based on this complicated function combining whether the reviewer typed the text ":lgtm", how many people looked at the review, whether they also hit the approve button. Each repo has its own configuration, and I can't make org-level changes without changing every repo one at a time. Excessive permissions - This might be a Github thing, but you can't grant Reviewable permissions to a particular private repo - you have to grant it permissions to all of your private repos. Several developers who join my team need to create a dedicated Github account to avoid exposing their other private repos to Reviewable. Thread state is unclear - The options are "discussing", "satisfied," "blocking," or "working," and it's not obvious who's supposed to move the thread to what state at what point. No development - I've been a paying customer of Reviewable for about 7 years, and I can remember only 1-2 minor features that have been added during that time. They haven't updated their blog[2] in 6 years, and they've never communicated with me as a paying customer to tell me anything they're doing. I checked out Crocodile, and it looks like it has potential. I'm not sure I'd pitch it to my team to switch yet. Here are some of my thoughts: * When do the reviewer's comments become visible to the author? One of the must-have features for me is that both author and reviwer(s) can prepare a set of notes, but they're not visible to anyone else until they hit "publish" to share them with the team. Sometimes I make comments in one spot, and then as I read more of the code, I revise a previous comment. If all my comments publish immediately, I can't revise comments like that. Github, Reviewable, and Gerrit all support a flow of preparing comments and then committing them in a separate step. * Crocodile touts the floating thread thing, and I've never used a tool that has it, but it doesn't seem better to me. Inline comments do break the flow, but floating comments actually cover up the code and prevent me from reading it. I see I can close threads, but I can't figure out how to get them back. * Being able to comment on character-level granularity is cool! * I think your thread state is better than Reviewable's, but I'd prefer an even simpler model where threads are either "open" or "resolved." When an author responds to a comment, the default action is to resolve it, but the author can override the default and leave it "open" if their comment is asking for clarification rather than declaring a fix. The reviewer can reopen a thread if they feel that the author has misunderstood the note. 95% of the time in my reviews, the reviewer makes a note and the author resolves it, so having a whole extra confirmation phase for that last 5% feels unnecessary when the reviewer can just reopen it instead. * Ditto for review state. The only two states I've ever needed for a code review are "pending approval" and "approved." I've never wanted to mark a PR as "rejected" unless it's just a spam submission from a stranger on an open-source repo, and even then, I'd close it from Github rather than my code review tool. The worst I'll do to a teammate is withhold approval until they address my notes, but I'd never mark it as "rejected." I don't need an explicit state for "pending review" or "waiting for author" because if the author is the last commenter, it's implicitly pending review. * I like that there's a view of all the comments at once. I like to review all my comments before pushing them to the author. * I'd like a way to mark a comment as "no action required" when I just want to say something nice[2] about the code that doesn't require action from the author. * I couldn't understand the "iterations" UI control. It's not obvious to me what the different circles represent.[4] Once I compared two diffs, I couldn't figure out how to compare to the the full PR to the base branch (i.e., all commits aggregated). I think it's replicating a control that Reviewable actually does pretty well, so I recommend giving it a look for inspiration. * It looks like I'm only allowed to make code-level comments, but I'd like to make review-level comments as well for high-level notes about the review as a whole. Hope that's useful. I'm very interested in code reviews, so if you want to do user interviews, feel free to reach out. You can find my contact info through my HN profile. [0] https://abseil.io/resources/swe-book/html/ch19.html https://abseil.io/resources/swe-book/html/ch19.html [1] https://www.gerritcodereview.com/ https://www.gerritcodereview.com/ [2] http://blog.reviewable.io/ http://blog.reviewable.io/ [3] https://mtlynch.io/human-code-reviews-2/#offer-sincere-praise https://mtlynch.io/human-code-reviews-2/#offer-sincere-prais... [4] https://i.imgur.com/3ZhDAR1.png https://i.imgur.com/3ZhDAR1.png
gingerlime 4y ago
looks very interesting. a quick question about pricing: do we need to pay for all people in our github organization or can pick a few? we have some bot users, translator access, but only need the review tool for developers.
- jameslao 4y ago
  You can add just the users you want. See: https://www.crocodile.dev/docs#adding-team-members https://www.crocodile.dev/docs#adding-team-members Crocodile has its own organization and member list. Billing is based on the max number of users in the Crocodile org for the month.
bilalq 4y ago
My biggest issue with Github code reviews is how broken the "Changes requested" state is. If I request changes, there's no easy way to see that changes have been addressed and are clear for me to look at again. I end up using open unresolved comments as the basis for changes requested now. It's hard to filter for those though.
- metadaemon 4y ago
  As long as someone re-requests reviews (clicks on the little refresh icon) it should show up again in your pull requests to review.
  - bilalq 4y ago
    That's still a pretty broken experience though. If I update a PR via the CLI, why do I have to fumble around on github.com's UI again? The next revision is already up!
- jameslao 4y ago
  I hear you. Crocodile has a couple features that try to address this. 1. More comment states. Comments start as "open". The workflow I had in mind was that once the author addresses it, they would mark it as "resolved". The reviewer can go to the comments view (in the meta section on the left nav) where they can filter to resolved comments. If it looks good, they can mark it as "closed" or "open" again if they want more changes. I have this as a suggested workflow in the docs. [1] 2. The iterations section in the left nav lets you compare any two iterations. You can use this to see what has changed since your last review. Crocodile even indicates which iteration your last review was on to make this easy. [1] https://www.crocodile.dev/docs#commenting https://www.crocodile.dev/docs#commenting
wingspan 4y ago
Thanks for calling out that this was inspired by CodeFlow, I kept thinking that as I was reading. Still one of my favorite code review tools, used ~2012. One of my favorite features was a panel with every comment on the PR, sorted and organized by status. Because all files in the PR were preloaded clicking a comment instantly took you to the specific code and version. So good!
- jameslao 4y ago
  If you click the "Comments ->" link in the meta section on the left nav of the review, it will take you to my version of the comments panel. A couple improvements I made: * You can see the whole discussion. The last time I used CodeFlow it only showed one line of the comment which often wasn't enough for me to remember what it was so in practice I always had to click on every single one to see the whole thing. Not a big deal with CodeFlow since it was so snappy as you mentioned. * It shows a preview of the code that was commented on to provide context. Thanks for taking a look! Great to see other CodeFlow fans out there.
jhogervorst 4y ago
Nice idea! I'd be interested in giving it a try for our project in the future. It does seem that the demo review is broken in Safari, I get a JS error and the UI doesn't seem to work completely: > [Error] TypeError: e.connect is not a function. (In 'e.connect(l,s)', 'e.connect' is undefined) Also, is there any way to keep up-to-date on the project? I don't see a newsletter or Twitter link on your homepage.
- jameslao 4y ago
  Ah yeah, it only supports Chrome (and Chrome based browsers) and Firefox for now. I've been meaning to add an error message if you use a different browser. No newsletter yet, but you can follow me on Twitter for now @jameslao.
heeton 4y ago
I like the site + product idea, but the demo fails to show me anything interesting except a floating comment. So I can't yet see the value of the tool. The files you've chosen to use don't appear to show any difference between iteration 1 and 2, so one of your major features doesn't do anything. Is that a bug, or accidental? (I'm using Firefox 101 on MacOS 11.4). My personal dumb suggestion: give a few demos, showing off the very worst points of code review hell, and how crocodile fixes each one. Make it a game. E.g. "You're halfway through a large code review, and Sally has just added 2 new commits, ugh. Challenge: find the button to see the new additions, then decide if you want to include them in your current review, or review this iteration first and the new additions separately." "Simon has just added a merge commit that fucks everything up, all the files look weird. Challenge: there's a way you can trivially see that the PR before this extra commit was looking great." "You can spot a new method that was introduced, and you'd like to see the places where it's being called, but that's a lot of scrolling back and forth. Challenge: find how to show code hints on a selected piece of text"
bluehatbrit 4y ago
This looks super interesting, it looks like it'd solve a few of my gripes with GitHub reviews. Congratulations on the launch! One thing I've always wondered is why all these review tools use centralised databases. Git itself is a distributed model and reviews tend to boil down to code comments on set of lines or characters. I'm always surprised no one has created a review tool that ships around reviews like patch files. Even if there was a server as an option, like github, I could then work offline and build little tools to help make my review process more efficient. I suppose it's not quite as easy to monitise as it's decentralised, but I'd love to see one crop up some day. Then my review process can match up with my coding process. Either way this looks like a big improvement in some areas over the GitHub tools so I'll definitely be checking it out.
- tionis 4y ago
  Well there is git-appraise[1] used by Google which stores it's data in the git repo itself. [1]: https://github.com/google/git-appraise https://github.com/google/git-appraise
  - bluehatbrit 4y ago
    Oh wow I totally hadn't come across this one, thanks
lallysingh 4y ago
It's worth trying. There's one quibble: How does Crocodile access my source code? Crocodile stores the source code files that are part of reviews to provide a fast user experience. Every file is encrypted with per file data encryption keys. The data encryption keys are then encrypted with a master encryption key. All cryptographic operations are performed using Google Tink, which is a cryptographic library created by cryptographers at Google that is designed to be misuse resistant. Files are encrypted using Stream AEAD using AES128_GCM_HKDF_4KB key type as recommended by Google. The data encryption keys above are encrypted using AEAD with a master AES128 key. So, um, what's the story with the master encryption key? Are the master keys in their own file? E.g., if Crocodile gets hacked, can the hackers pull up everyone's reviews (and sources)? Or does all this encryption keep it encrypted at rest and require something from the user (e.g., their password) to derive the master key?
- jameslao 4y ago
  The master key is stored as a secret in Kubernetes so if someone compromised it, they would be able to decrypt the source code in the reviews.
bstpierre 4y ago
Not a knock against crocodile, which looks like a nice set of improvements over gh, but something I’d like to see done better in any code review system is significantly better code navigation. For any PR that is nontrivial I will pull it locally so that I can more easily navigate to functions/data types that are used by or changed in the PR. It would be nice if the review ui provided a way to click through to the definition of a symbol that appears in the code. (I think gh does this when browsing code for some languages.) A related helpful feature would be the ability to see “what calls this”. Currently I have to do this kind of review with ‘git grep’, after pulling locally. I’d also love to be able to toggle into ‘git blame’ for a given bit of code, in order to better understand why the code is in its current state
- darkerside 4y ago
  You're basically asking for an IDE to be part of your code review tool, which I don't think is a bad idea
  - user3939382 4y ago
    ctags allows this and isn’t necessarily part of an IDE. Granted, that may be where most people encounter this feature.
  - jlkuester7 4y ago
    Yeah, IDEA already basically supports this concept: https://blog.jetbrains.com/idea/2018/10/intellij-idea-2018-3-eap-github-pull-requests-and-more/ https://blog.jetbrains.com/idea/2018/10/intellij-idea-2018-3...
- dsmmcken 4y ago
  Hitting the keyboard shortcut "." on a PR in GitHub to open it in a vs code web view is pretty great, and I find it a substantial improvement to the normal GitHub PR workflow, especially with large PRs. You can also install the GitHub PR extension in vs code to get the same experience locally, along with gitlens to get a nice inline blame.
wnolens 4y ago
I also miss CodeFlow. At Amazon now and their review tool sucks.
alephnan 4y ago
Are there plans for Gitlab?
difflens 4y ago
Congratulations on the launch. I find reviewing code on GitHub to be a pain too, so we came up with DiffLens (https://github.com/marketplace/difflens https://github.com/marketplace/difflens). DiffLens is only concerned with showing better diffs though, and doesn't handle comments on GitHub at all. Maybe there's room here for us to combine our approaches :) Our email is support@difflens.com if you want to get in touch.

144 points by jameslao 4y ago

perpil 4y ago

Nit: Couple typos on landing page: coversation, shorcut. Worth taking a pass with a spell checker.

jameslao 4y ago
Thanks for pointing those out! Fixed.

silverlake 4y ago

“You can comment anywhere in the file and on any text selection, even if it's just one character.” I’ve been looking for this feature forever. A changed line might impact code that wasn’t changed. There wasn’t a convenient way to comment on those. Looking forward to trying this.

http-teapot 4y ago

I am a bit at odd with the pricing. My team has 6 engineers and base price for Github is $24. Your product would increase our bill by $48, an increase of 200%. $48 is nothing compared to the salaries of 6 engineers but I am not convinced the feature set would make my team more productive. If you told me that your solution help my team ship faster and saves an hour per engineer per week then that's easy math: your product pays for itself. Suggestions: - Make all base features free (the ones on your site currently) - Add analytics to your product, collect data and put it behind a paywall (entirely or partially by truncating historical data) - Iterate on premium features that improves critical metrics - Offer analytics with a trial of 2 to 3 months, enough time for graphs to speak for themselves - Make sure the gains are seen by the manager or business owner or whoever is the person in charge Pricing can be based on the average of hours saved.

paulgb 4y ago
I’m going to disagree, I think the model they have is both familiar to me as a potential user and priced such that, if it is even slightly better than GitHub, it’s a no-brainer. I wouldn’t pay extra for analytics, though, it just sounds like another thing to check. IMHO if the one-month free trial isn’t enough to know whether a SaaS solution is worth an $8/head price, it’s probably not worth that price.
- tr1ll10nb1ll 4y ago
  For me, the most worthy SaaS purchases usually have been the ones that I knew were worth it from day 0. The trial has always just been a cherry on top. I haven’t ever used the entire trial for the sake of “trying” and then choosing to not use the product further.
- OJFord 4y ago
  Adding Y to Xy is already a tough sell, but when the upgraded Y costs more than all of Xy? I sympathise with GP. GitHub can do it from an economy of scale and probably 'should' be more expensive, but it can manage not to be (maybe cheaper is actually optimal, given how many more will pay for it, point is it's in some sense 'worth' more) which puts OP in a tough spot. Imagine trying to sell a better WYSIWYG editor for WordPress for example. You can be better, save time, etc. - it's still always going to be a tough sell to charge more than a fraction of WP itself which offers a WYSIWYG editor and everything else.

burlesona 4y ago

This looks great! Food for thought on pricing: Because you offer free for open source, there’s minimal need for a free trial for private repos. I would consider requiring a credit card and making it a 14 day trial. My experience selling SaaS in the past is this will net you fewer but much more serious evals, so whenever someone signs up you know they are legit and worth your time to contact and support. That leads to higher conversion, better retention, etc. Best of luck!

gingerlime 4y ago
generally I agree, but a no-card trial removes friction. I have a few open source projects, but they don’t use much code review or at all compared to the private repo at my company. Yeah I can add a card, but it’s easier to start without it, see if there’s a fit and then decide.

paxys 4y ago

This looks fantastic! I'd love to try this out, but I don't have admin access to the repo I work in and IT will definitely not approve a random new app. Any chance I can set this up with a personal access token instead?

jameslao 4y ago
Thanks for taking a look! Yeah, not having admin access is definitely a problem. Using a PAT is an interesting idea that I'll have to think about that from a security and liability perspective.
- igetspam 4y ago
  Just today I had to go through the work of auditing a GitHub app, for compliance reasons. Having the source available made that possible. Since your source isn't available, the barrier to entry would be a SOC2 report or you filling out a vendor survey. Compliance is a PITA but it's a real thing for companies. How are you planning on clearing that hurdle? I wouldn't want to put you through the vendor survey gauntlet and SOC2 is a lot to ask of a solo dev but the aforementioned IT team could likely be persuaded if you did Type 1 at least.

decebalus1 4y ago

Oh wow, this looks great. Congratulations!

searchableguy 4y ago

This provides better UX but adds one more redirection when you are using github. Do you support features like github suggestions which can be committed easily by author? Given your history with Microsoft and their internal tool which this is inspired by, how long until github copies some of the ideas? How many active users does this have? Are there potential problems you see outside of current product? I remember a startup providing paid code reviews as a service launched on HN a while ago. That could be a pivot for example in providing more value.

jameslao 4y ago
> Do you support features like github suggestions which can be committed easily by author? Not right now. The suggestion feature is something I want to add in the future if there's interest though! > Given your history with Microsoft and their internal tool which this is inspired by, how long until github copies some of the ideas? No idea! But CodeFlow had been around for many years before they acquired GitHub so they've had plenty of time to do it! I think this is an existential fear that anyone building code review tools has. For Crocodile, I think the long term strategy will be to decouple this from GitHub so that it can work with any source control provider. I architected it this way from the beginning but decided to target GitHub first because they have mindshare and because it made it really easy to dogfood since I host my code on GitHub. > Are there potential problems you see outside of current product? There's the one you already mentioned, which is GitHub improving their PR system, but there's not much I can do about that. Current problems I'm looking at are roadmap related. I have a few ideas of features I'd like to add, but no real indication on what to do next. Hence, posting on HN to get feedback. :) Thanks for taking a look and the feedback!
- gingerlime 4y ago
  +1 for suggestions. It’s so much more helpful to see a diff and even be able to commit it.
- jhogervorst 4y ago
  Another +1 for suggestions (with 'commit now' button). It's the thing I love most about GitHub PRs, and I wouldn't consider leaving it for a solution without it :-)

imron 4y ago

> * Comments float above the code > * Comment on any text selection in the file > * Comments don't get lost when code changes This addresses my main pet peeves with GitHub/Bitbucket reviews! 2 questions: With regards to comments not getting lost, how well does this work across rebases and force pushes? Are you considering supporting other products like Bitbucket and Gitlab?

jameslao 4y ago
Comments should get transferred just fine across rebases and force pushes. If they don't, you can let me know using the feedback form at the top right. Supporting other VCS providers, even non-Git ones, is on my list. The underlying data model is agnostic to GitHub so it's just a matter of having demand for it and doing the work!

catchmeifyoucan 4y ago

Congrats on the launch! Here's a demo link in case folks might have missed it: https://app.crocodile.dev/reviews/rwsfSKbgZoSt?change=README.md&iter=2 https://app.crocodile.dev/reviews/rwsfSKbgZoSt?change=README...

mtlynch 4y ago

This is cool! I'm happy to see more options in this space. The best code review tool I've ever used was a tool at Google called Critique.[0] They've open-sourced it as Gerrit[1], but there are sadly no hosted versions available for under $15k/yr, and it's complicated to self-host. I've been using Reviewable, and my experience has been good not great. Github's native code review has caught up a bit, but Github's review tool falls apart if your review lasts more than one round. Here are my gripes with Reviewable: Steep learning curve - Every new developer who joins the team spends their first few reviews being confused and frustrated by Reviewable. Performance - Reviewable has awful performance. It takes about 10 seconds to load a code review. It seems like it's doing some odd websockets stuff where sometimes my "connection" to Reviewable will drop and I can't add comments. I've never experienced this with any other web app. It's gotten better over the last few years, but it's still annoyingly frequent. Complicated configuration - I just want the reviewer to be able to hit an "LGTM" button to mark their approval. Reviewable's decision about when a PR is approved is based on this complicated function combining whether the reviewer typed the text ":lgtm", how many people looked at the review, whether they also hit the approve button. Each repo has its own configuration, and I can't make org-level changes without changing every repo one at a time. Excessive permissions - This might be a Github thing, but you can't grant Reviewable permissions to a particular private repo - you have to grant it permissions to all of your private repos. Several developers who join my team need to create a dedicated Github account to avoid exposing their other private repos to Reviewable. Thread state is unclear - The options are "discussing", "satisfied," "blocking," or "working," and it's not obvious who's supposed to move the thread to what state at what point. No development - I've been a paying customer of Reviewable for about 7 years, and I can remember only 1-2 minor features that have been added during that time. They haven't updated their blog[2] in 6 years, and they've never communicated with me as a paying customer to tell me anything they're doing. I checked out Crocodile, and it looks like it has potential. I'm not sure I'd pitch it to my team to switch yet. Here are some of my thoughts: * When do the reviewer's comments become visible to the author? One of the must-have features for me is that both author and reviwer(s) can prepare a set of notes, but they're not visible to anyone else until they hit "publish" to share them with the team. Sometimes I make comments in one spot, and then as I read more of the code, I revise a previous comment. If all my comments publish immediately, I can't revise comments like that. Github, Reviewable, and Gerrit all support a flow of preparing comments and then committing them in a separate step. * Crocodile touts the floating thread thing, and I've never used a tool that has it, but it doesn't seem better to me. Inline comments do break the flow, but floating comments actually cover up the code and prevent me from reading it. I see I can close threads, but I can't figure out how to get them back. * Being able to comment on character-level granularity is cool! * I think your thread state is better than Reviewable's, but I'd prefer an even simpler model where threads are either "open" or "resolved." When an author responds to a comment, the default action is to resolve it, but the author can override the default and leave it "open" if their comment is asking for clarification rather than declaring a fix. The reviewer can reopen a thread if they feel that the author has misunderstood the note. 95% of the time in my reviews, the reviewer makes a note and the author resolves it, so having a whole extra confirmation phase for that last 5% feels unnecessary when the reviewer can just reopen it instead. * Ditto for review state. The only two states I've ever needed for a code review are "pending approval" and "approved." I've never wanted to mark a PR as "rejected" unless it's just a spam submission from a stranger on an open-source repo, and even then, I'd close it from Github rather than my code review tool. The worst I'll do to a teammate is withhold approval until they address my notes, but I'd never mark it as "rejected." I don't need an explicit state for "pending review" or "waiting for author" because if the author is the last commenter, it's implicitly pending review. * I like that there's a view of all the comments at once. I like to review all my comments before pushing them to the author. * I'd like a way to mark a comment as "no action required" when I just want to say something nice[2] about the code that doesn't require action from the author. * I couldn't understand the "iterations" UI control. It's not obvious to me what the different circles represent.[4] Once I compared two diffs, I couldn't figure out how to compare to the the full PR to the base branch (i.e., all commits aggregated). I think it's replicating a control that Reviewable actually does pretty well, so I recommend giving it a look for inspiration. * It looks like I'm only allowed to make code-level comments, but I'd like to make review-level comments as well for high-level notes about the review as a whole. Hope that's useful. I'm very interested in code reviews, so if you want to do user interviews, feel free to reach out. You can find my contact info through my HN profile. [0] https://abseil.io/resources/swe-book/html/ch19.html https://abseil.io/resources/swe-book/html/ch19.html [1] https://www.gerritcodereview.com/ https://www.gerritcodereview.com/ [2] http://blog.reviewable.io/ http://blog.reviewable.io/ [3] https://mtlynch.io/human-code-reviews-2/#offer-sincere-praise https://mtlynch.io/human-code-reviews-2/#offer-sincere-prais... [4] https://i.imgur.com/3ZhDAR1.png https://i.imgur.com/3ZhDAR1.png

gingerlime 4y ago

looks very interesting. a quick question about pricing: do we need to pay for all people in our github organization or can pick a few? we have some bot users, translator access, but only need the review tool for developers.

jameslao 4y ago
You can add just the users you want. See: https://www.crocodile.dev/docs#adding-team-members https://www.crocodile.dev/docs#adding-team-members Crocodile has its own organization and member list. Billing is based on the max number of users in the Crocodile org for the month.

bilalq 4y ago

My biggest issue with Github code reviews is how broken the "Changes requested" state is. If I request changes, there's no easy way to see that changes have been addressed and are clear for me to look at again. I end up using open unresolved comments as the basis for changes requested now. It's hard to filter for those though.

metadaemon 4y ago
As long as someone re-requests reviews (clicks on the little refresh icon) it should show up again in your pull requests to review.
- bilalq 4y ago
  That's still a pretty broken experience though. If I update a PR via the CLI, why do I have to fumble around on github.com's UI again? The next revision is already up!
jameslao 4y ago
I hear you. Crocodile has a couple features that try to address this. 1. More comment states. Comments start as "open". The workflow I had in mind was that once the author addresses it, they would mark it as "resolved". The reviewer can go to the comments view (in the meta section on the left nav) where they can filter to resolved comments. If it looks good, they can mark it as "closed" or "open" again if they want more changes. I have this as a suggested workflow in the docs. [1] 2. The iterations section in the left nav lets you compare any two iterations. You can use this to see what has changed since your last review. Crocodile even indicates which iteration your last review was on to make this easy. [1] https://www.crocodile.dev/docs#commenting https://www.crocodile.dev/docs#commenting

wingspan 4y ago

Thanks for calling out that this was inspired by CodeFlow, I kept thinking that as I was reading. Still one of my favorite code review tools, used ~2012. One of my favorite features was a panel with every comment on the PR, sorted and organized by status. Because all files in the PR were preloaded clicking a comment instantly took you to the specific code and version. So good!

jameslao 4y ago
If you click the "Comments ->" link in the meta section on the left nav of the review, it will take you to my version of the comments panel. A couple improvements I made: * You can see the whole discussion. The last time I used CodeFlow it only showed one line of the comment which often wasn't enough for me to remember what it was so in practice I always had to click on every single one to see the whole thing. Not a big deal with CodeFlow since it was so snappy as you mentioned. * It shows a preview of the code that was commented on to provide context. Thanks for taking a look! Great to see other CodeFlow fans out there.

jhogervorst 4y ago

Nice idea! I'd be interested in giving it a try for our project in the future. It does seem that the demo review is broken in Safari, I get a JS error and the UI doesn't seem to work completely: > [Error] TypeError: e.connect is not a function. (In 'e.connect(l,s)', 'e.connect' is undefined) Also, is there any way to keep up-to-date on the project? I don't see a newsletter or Twitter link on your homepage.

jameslao 4y ago
Ah yeah, it only supports Chrome (and Chrome based browsers) and Firefox for now. I've been meaning to add an error message if you use a different browser. No newsletter yet, but you can follow me on Twitter for now @jameslao.

heeton 4y ago

I like the site + product idea, but the demo fails to show me anything interesting except a floating comment. So I can't yet see the value of the tool. The files you've chosen to use don't appear to show any difference between iteration 1 and 2, so one of your major features doesn't do anything. Is that a bug, or accidental? (I'm using Firefox 101 on MacOS 11.4). My personal dumb suggestion: give a few demos, showing off the very worst points of code review hell, and how crocodile fixes each one. Make it a game. E.g. "You're halfway through a large code review, and Sally has just added 2 new commits, ugh. Challenge: find the button to see the new additions, then decide if you want to include them in your current review, or review this iteration first and the new additions separately." "Simon has just added a merge commit that fucks everything up, all the files look weird. Challenge: there's a way you can trivially see that the PR before this extra commit was looking great." "You can spot a new method that was introduced, and you'd like to see the places where it's being called, but that's a lot of scrolling back and forth. Challenge: find how to show code hints on a selected piece of text"

bluehatbrit 4y ago

This looks super interesting, it looks like it'd solve a few of my gripes with GitHub reviews. Congratulations on the launch! One thing I've always wondered is why all these review tools use centralised databases. Git itself is a distributed model and reviews tend to boil down to code comments on set of lines or characters. I'm always surprised no one has created a review tool that ships around reviews like patch files. Even if there was a server as an option, like github, I could then work offline and build little tools to help make my review process more efficient. I suppose it's not quite as easy to monitise as it's decentralised, but I'd love to see one crop up some day. Then my review process can match up with my coding process. Either way this looks like a big improvement in some areas over the GitHub tools so I'll definitely be checking it out.

tionis 4y ago
Well there is git-appraise[1] used by Google which stores it's data in the git repo itself. [1]: https://github.com/google/git-appraise https://github.com/google/git-appraise
- bluehatbrit 4y ago
  Oh wow I totally hadn't come across this one, thanks

lallysingh 4y ago

It's worth trying. There's one quibble: How does Crocodile access my source code? Crocodile stores the source code files that are part of reviews to provide a fast user experience. Every file is encrypted with per file data encryption keys. The data encryption keys are then encrypted with a master encryption key. All cryptographic operations are performed using Google Tink, which is a cryptographic library created by cryptographers at Google that is designed to be misuse resistant. Files are encrypted using Stream AEAD using AES128_GCM_HKDF_4KB key type as recommended by Google. The data encryption keys above are encrypted using AEAD with a master AES128 key. So, um, what's the story with the master encryption key? Are the master keys in their own file? E.g., if Crocodile gets hacked, can the hackers pull up everyone's reviews (and sources)? Or does all this encryption keep it encrypted at rest and require something from the user (e.g., their password) to derive the master key?

jameslao 4y ago
The master key is stored as a secret in Kubernetes so if someone compromised it, they would be able to decrypt the source code in the reviews.

bstpierre 4y ago

Not a knock against crocodile, which looks like a nice set of improvements over gh, but something I’d like to see done better in any code review system is significantly better code navigation. For any PR that is nontrivial I will pull it locally so that I can more easily navigate to functions/data types that are used by or changed in the PR. It would be nice if the review ui provided a way to click through to the definition of a symbol that appears in the code. (I think gh does this when browsing code for some languages.) A related helpful feature would be the ability to see “what calls this”. Currently I have to do this kind of review with ‘git grep’, after pulling locally. I’d also love to be able to toggle into ‘git blame’ for a given bit of code, in order to better understand why the code is in its current state

darkerside 4y ago
You're basically asking for an IDE to be part of your code review tool, which I don't think is a bad idea
- user3939382 4y ago
  ctags allows this and isn’t necessarily part of an IDE. Granted, that may be where most people encounter this feature.
- jlkuester7 4y ago
  Yeah, IDEA already basically supports this concept: https://blog.jetbrains.com/idea/2018/10/intellij-idea-2018-3-eap-github-pull-requests-and-more/ https://blog.jetbrains.com/idea/2018/10/intellij-idea-2018-3...
dsmmcken 4y ago
Hitting the keyboard shortcut "." on a PR in GitHub to open it in a vs code web view is pretty great, and I find it a substantial improvement to the normal GitHub PR workflow, especially with large PRs. You can also install the GitHub PR extension in vs code to get the same experience locally, along with gitlens to get a nice inline blame.

wnolens 4y ago

I also miss CodeFlow. At Amazon now and their review tool sucks.

alephnan 4y ago

Are there plans for Gitlab?

difflens 4y ago

Congratulations on the launch. I find reviewing code on GitHub to be a pain too, so we came up with DiffLens (https://github.com/marketplace/difflens https://github.com/marketplace/difflens). DiffLens is only concerned with showing better diffs though, and doesn't handle comments on GitHub at all. Maybe there's room here for us to combine our approaches :) Our email is support@difflens.com if you want to get in touch.