4 ms·

TravisBuddy: Adds comments to failed pull requests, tells author what went wrong

39 points by bluzi 9y ago

bigblind 9y ago
I really like this idea. I think more tools should integrate into GitHub this way. The status API is nice, but you always have to click through to get the details. I think you could do something similar with code analysis tools, so they make review comments in the code where issues occur.
- bluzi 9y ago
  Thanks, that's an awesome idea. :)
- ddm379 9y ago
  I know this is nit-picky, but I'm not a fan of how the "Details" link opens in the same tab.
  - bluzi 9y ago
    Not sure what do you mean
TomK32 9y ago
Cute little list on the example screenshot. looks at own code with some 3300 tests
- bluzi 9y ago
  We have methods to handle big build logs, and we have a backlog full of ideas on how to improve the way we handle big logs.
- nulagrithom 9y ago
  mocha -R min
- bluntfang 9y ago
  I'm not a travis user, but with Jenkins there is an xunit plugin that opens up an api with test results for the specific build. since xunit formatting is standard/generic, we've implemented a parser that just gets the # of pass/failed tests and links to the build for further investigation. We had a similar problem where posting all of the results was just more noise than useful, this is at least a good starting point for handling build result feedback.
diggan 9y ago
This will get messy quickly if you have a PR that you are adding commits to and have a failure for each commit. This has the same problem as some code coverage solutions, they are spamming the PR with comments that are only important for a few moments, then they are just noise... Speaking about finding test failures quickly though, Jenkins and many others (except ALL the cloud/SaaS solutions?) actually parse test results via junit or similar to show you failures easily. I'm wondering why more SaaS don't do this? See an example here (for our js-ipfs project): https://ci.ipfs.team/blue/organizations/jenkins/IPFS%2Fjs-ipfs/detail/fix%2Fremove-unused-var/2/tests https://ci.ipfs.team/blue/organizations/jenkins/IPFS%2Fjs-ip... It's so much easier to find test failures now and doesn't involve leaving a comment on the PR. It's simply two clicks to find this view from a PR (click the status message, click on "tests")
- bluzi 9y ago
  Thanks for the feedback, we're working on methods to prevent TravisBuddy from commenting if the PR is being rapidly updated or if its comments are not relevant for some reason. One idea is that if the PR has a special label TravisBuddy won't comment on it. Another one is to be able to communicate with TravisBuddy using comments, so when you're sick of it you could just comment: "@TravisBuddy stop", and it'll stop immediately. (Maybe even "@TravisBuddy clear" to remove the previous comments) Yeah, Jenkins' tests view was one of the inspirations for TravisBuddy :)
  - neebz 9y ago
    You can do what codecov does; They keep updating the same comment with the latest result.
    - bluzi 9y ago
      That's an awesome idea, will be a feature for sure.
- bostonaholic 9y ago
  CircleCI will parse and collect JUnit test metadata. https://circleci.com/docs/2.0/collect-test-data/ https://circleci.com/docs/2.0/collect-test-data/
dotsh 9y ago
Idea is great but this logo looks like evil douchebag kind of guy with this tagline "Not the bot you deserve, but the one you need" doesn't look great. You definitely need to work on your brand. :)
- neuronexmachina 9y ago
  It's a Batman (The Dark Knight) reference: "He's the hero Gotham deserves, but not the one it needs right now"
  - kemayo 9y ago
    I suspect the grandparent knew this, and was making the point that the reference isn't sending a good message.
- bluzi 9y ago
  Thanks for the feedback, the website and any part of the brand is open-source, so if you feel like contributing, you may start a discussion on GitHub or make a PR :)
  - masukomi 9y ago
    i'd suggest that he looks like he's just waiting for his chance to fuck up your stuff.
- koolba 9y ago
  Reminds me of vault boy[1] from Fallout but with a sneaky hat on. https://www.google.com/search?q=vault+boy&tbm=isch https://www.google.com/search?q=vault+boy&tbm=isch
  - kemayo 9y ago
    Yeah, but Vault Boy looks cheerful, this guy looks angry.
    - 9y ago
matchai 9y ago
For those not using Travis, ci-reporter supports CircleCI too, and is extensible to support any other CI platform. https://github.com/JasonEtco/ci-reporter https://github.com/JasonEtco/ci-reporter
dandigangi 9y ago
Really cool! I'm going to give it a try. I've been looking to build our Travis CI more in-depth.

39 points by bluzi 9y ago

I really like this idea. I think more tools should integrate into GitHub this way. The status API is nice, but you always have to click through to get the details. I think you could do something similar with code analysis tools, so they make review comments in the code where issues occur.

bluzi 9y ago
Thanks, that's an awesome idea. :)
ddm379 9y ago
I know this is nit-picky, but I'm not a fan of how the "Details" link opens in the same tab.
- bluzi 9y ago
  Not sure what do you mean

TomK32 9y ago

Cute little list on the example screenshot. looks at own code with some 3300 tests

bluzi 9y ago
We have methods to handle big build logs, and we have a backlog full of ideas on how to improve the way we handle big logs.
nulagrithom 9y ago
mocha -R min
bluntfang 9y ago
I'm not a travis user, but with Jenkins there is an xunit plugin that opens up an api with test results for the specific build. since xunit formatting is standard/generic, we've implemented a parser that just gets the # of pass/failed tests and links to the build for further investigation. We had a similar problem where posting all of the results was just more noise than useful, this is at least a good starting point for handling build result feedback.

diggan 9y ago

This will get messy quickly if you have a PR that you are adding commits to and have a failure for each commit. This has the same problem as some code coverage solutions, they are spamming the PR with comments that are only important for a few moments, then they are just noise... Speaking about finding test failures quickly though, Jenkins and many others (except ALL the cloud/SaaS solutions?) actually parse test results via junit or similar to show you failures easily. I'm wondering why more SaaS don't do this? See an example here (for our js-ipfs project): https://ci.ipfs.team/blue/organizations/jenkins/IPFS%2Fjs-ipfs/detail/fix%2Fremove-unused-var/2/tests https://ci.ipfs.team/blue/organizations/jenkins/IPFS%2Fjs-ip... It's so much easier to find test failures now and doesn't involve leaving a comment on the PR. It's simply two clicks to find this view from a PR (click the status message, click on "tests")

bluzi 9y ago
Thanks for the feedback, we're working on methods to prevent TravisBuddy from commenting if the PR is being rapidly updated or if its comments are not relevant for some reason. One idea is that if the PR has a special label TravisBuddy won't comment on it. Another one is to be able to communicate with TravisBuddy using comments, so when you're sick of it you could just comment: "@TravisBuddy stop", and it'll stop immediately. (Maybe even "@TravisBuddy clear" to remove the previous comments) Yeah, Jenkins' tests view was one of the inspirations for TravisBuddy :)
- neebz 9y ago
  You can do what codecov does; They keep updating the same comment with the latest result.
  - bluzi 9y ago
    That's an awesome idea, will be a feature for sure.
bostonaholic 9y ago
CircleCI will parse and collect JUnit test metadata. https://circleci.com/docs/2.0/collect-test-data/ https://circleci.com/docs/2.0/collect-test-data/

dotsh 9y ago

Idea is great but this logo looks like evil douchebag kind of guy with this tagline "Not the bot you deserve, but the one you need" doesn't look great. You definitely need to work on your brand. :)

neuronexmachina 9y ago
It's a Batman (The Dark Knight) reference: "He's the hero Gotham deserves, but not the one it needs right now"
- kemayo 9y ago
  I suspect the grandparent knew this, and was making the point that the reference isn't sending a good message.
bluzi 9y ago
Thanks for the feedback, the website and any part of the brand is open-source, so if you feel like contributing, you may start a discussion on GitHub or make a PR :)
- masukomi 9y ago
  i'd suggest that he looks like he's just waiting for his chance to fuck up your stuff.
koolba 9y ago
Reminds me of vault boy[1] from Fallout but with a sneaky hat on. https://www.google.com/search?q=vault+boy&tbm=isch https://www.google.com/search?q=vault+boy&tbm=isch
- kemayo 9y ago
  Yeah, but Vault Boy looks cheerful, this guy looks angry.
  - 9y ago

matchai 9y ago

For those not using Travis, ci-reporter supports CircleCI too, and is extensible to support any other CI platform. https://github.com/JasonEtco/ci-reporter https://github.com/JasonEtco/ci-reporter

dandigangi 9y ago

Really cool! I'm going to give it a try. I've been looking to build our Travis CI more in-depth.