5 ms·
> Another common critique is that the pull requests encourage reviewers to only look at the code. They never even pull down the changes they are reviewing! You
by yjftsjthsd-h 5y ago
> Another common critique is that the pull requests encourage reviewers to only look at the code. They never even pull down the changes they are reviewing!
You should run it somehow, but if you've got CI running tests, which I think you should have anyways, then does it matter?
- juanca 5y agoAt least with UI code, the tests / code don't really express the experience for the user. (But it's getting better with time!) In that practice, it's better to run it locally! For each UI file / function, try to get it to execute from a user's POV.
- watwut 5y agoThat is what testers are for. Testing is great thing.
- bern4444 5y ago/s/testers/users Just kidding and in reality UI tests exists and aren't super difficult to set up. The front end has plenty of tools like Cypress and its pretty simple to automate running a bunch of tests that screenshot and diff compare your site. I never really pulled down UI code to test them out. There has to be a certain level of trust between engineers. Sometimes I'd ask or post screenshots of changes but that was rare.
- shinryuu 5y agoStill, getting a second set of fresh eyes using the code that you wrote is pretty invaluable. This can then be fed back as improvements.
- akiselev 5y ago> Just kidding and in reality UI tests exists and aren't super difficult to set up. The front end has plenty of tools like Cypress and its pretty simple to automate running a bunch of tests that screenshot and diff compare your site. In a well oiled machine, that's what the QA department does. They have close working relationships with the engineering and customer support departments so they develop testing processes and systems tailored to the "quirks" of engineers and users. They spend extra time on the places they know the engineers have blind spots and work on automated regression tests based on customer support workloads. Whether they go as low level as unit tests depends on the organization but one place I worked at, the domain was complicated enough that "I see no red squiggles in my IDE, pinky swear" was the threshold to go from engineering to QA. It was an ongoing joke that whenever there was a major dependency update, 10-25% of the work sent to QA that day contained obvious syntax errors and wouldn't compile because the IDE everyone used often took an hour or more to reindex the project without any clear indication that it was in a useless intermediate state. Running the compiler would slow it and the indexer to a crawl, so the engineers just YOLOd it so they could move on to other work, and the QA department set up a static analyzer with an empty rule set on all PRs to catch dumb errors caused by immutable process/purchasing decisions. IME these kinds of QA departments are the #1 springboard into software engineering roles because they work so closely with engineering and are exposed to code in the form of automated tests.
- sfvisser 5y agoOf course it matters. Otherwise, how do you know if your app works as intended? And don’t tell me your tests tell you, because I guarantee you they don’t!
- the_af 5y agoIsn't that what QA/testers are for? Testing that the feature/release works as intended?
- loloquwowndueo 5y agoA reviewer in my team is fond of changing some of the code in the PR and running the tests. The expectation is that if the code is changed randomly a test should always fail. This helps catch untested code paths. It’s somewhat similar to the practice of automated code fuzzing, only manual and scoped to changes in a particular PR.
- travisjungroth 5y agoThat's mutation testing[1]. For most languages there are tools that can scope the mutations to a diff. [1]https://en.wikipedia.org/wiki/Mutation_testing https://en.wikipedia.org/wiki/Mutation_testing
- elwell 5y agoIn practice, I end up finding at least one bug nearly every time I pull the code down and test it locally.
- kmtrowbr 5y agoYou should attack it. You should pick it apart and try to break it in as many ways as possible. Of course you should meter your effort depending on how critical the changes are. It amazes me the lackadaisical attitude towards code review. "It will probably be okay" is the worst attitude to have with computer code. It is the most brittle, brutal environment imaginable.
- all2 5y agoCan you expand on this?
- kmtrowbr 5y agoYou should pull the PR you're reviewing down into your IDE and you should spend like, an hour reviewing it carefully and trying to break it. Have a mental checklist: say they renamed a function ... is the file named the same as the function? Did they remember to rename the file? Did they delete parts of the code? Did they remember to delete everything related to that? Often the most challenging parts are negative: it's harder to remove properly than it is to add new things. Say they changed the validation on a model: is this going to work well with the production database? Do we need to write a migration to catch the database up to the new reality of the code? Automated tests are just automated warning flags that give you confidence that the codebase is basically hanging together. You still need to be very careful & thoughtful as you're making changes. I like to actually reset the branch so I can see all the raw changes in my IDE, this helps me to think it through, in git you can do this via: git pull feature-branch; git checkout develop; git checkout -b my_initials/feature-branch; git merge feature-branch --squash --no-commit
- efficax 5y agoAn hour is way too much time. If you need that long the pr is too big and needs to be broken up into smaller parts
- QuercusMax 5y agoHow big are these changes? That sounds horrendously slow. If you don't trust your tests, write better tests. If you don't trust your migrations, write better tests.
- all2 5y agoJust because there are tests doesn't mean the tests are any good. I've only started my quest in TDD and I've discovered it is rather trivial to write tests that don't really mean anything. I find myself asking "what is this supposed to do?" and "does this actually do what it is supposed to do?". I tend to drift into a design mindset during test writing (more-so than in code review) that highlights data flow and the basic bits of logic. Even then, I find that I can miss test cases rather easily.