3 ms·

Go has code coverage analysis in its testing facilities... why not use those?

kjksf 8y ago
I do use builtin coverage. It's just it's nice to be able to track code coverage over time, which is what codecov.io provides. Also, running all tests (to get full code coverage) takes 20 minutes and makes fan on my laptop unhappy. The way it works is that on every checkin the CI job runs all the tests with code coverage enabled and uploads the results to codecov. Codecov can then plot coverage over time. My gripe is with inaccurate accounting of empty lines (like comments or struct definitions) by codecov. Go's tool to visualize this count them properly. I don't know if it's codecov or maybe I'm not sending the data properly.
- jimsmart 8y ago
  In the above linked article you say: > Codecov is barely adequate. For Go, they count non-code lines (comments etc.) as not executed. It's impossible to get 100% code coverage as reported by the tool. I have an open source Go project that has some comments in methods, but still achieves 100% coverage using Codecov.io — I'm not sure what I do differently to yourself? (Perhaps I'm not using any inline struct definitions?) Here's a link, in case there's anything useful to you in my .travis.yml ? https://github.com/jimsmart/store4 https://github.com/jimsmart/store4 HTH
  - kjksf 8y ago
    It looks like we're both doing the same thing. I don't know, maybe I'm not counting things right but for example https://codecov.io/gh/ravendb/ravendb-go-client/src/master/delete_attachment_command_data.go https://codecov.io/gh/ravendb/ravendb-go-client/src/master/d... shows less then 80% coverage and there are only 2 lines not exected out of at least 18, which should be at most 10% counted as not covered.
    - jimsmart 8y ago
      In the example you link, there are 18 coloured lines, 4 of which are not green: 4/18 = ~0.22 = ~22%. This tallies-up as expected with the 77.78% coverage shown at the top of the file. Codecov* doesn't count an 'if' statement as having full coverage unless one tests both outcomes: so the yellow lines here have been executed, but do not count towards your coverage score. Granted, one could argue that that's not very generous! But on the other hand: those yellow lines have not been fully tested, despite being executed, so I can understand their decision. In the linked code, just implement a couple of simple tests to test for the expected error conditions: it's easy (here at least) and ensures the code behaves as expected. (Obviously not all partial/no coverage lines will be so easy to hit with tests, it might not always be possible to easily get 100% coverage, but hey: start with the low hanging fruit!) * I say Codecov here, but I highly suspect that they may simply be using Go's coverage reports under the hood?

I do use builtin coverage. It's just it's nice to be able to track code coverage over time, which is what codecov.io provides. Also, running all tests (to get full code coverage) takes 20 minutes and makes fan on my laptop unhappy. The way it works is that on every checkin the CI job runs all the tests with code coverage enabled and uploads the results to codecov. Codecov can then plot coverage over time. My gripe is with inaccurate accounting of empty lines (like comments or struct definitions) by codecov. Go's tool to visualize this count them properly. I don't know if it's codecov or maybe I'm not sending the data properly.

jimsmart 8y ago
In the above linked article you say: > Codecov is barely adequate. For Go, they count non-code lines (comments etc.) as not executed. It's impossible to get 100% code coverage as reported by the tool. I have an open source Go project that has some comments in methods, but still achieves 100% coverage using Codecov.io — I'm not sure what I do differently to yourself? (Perhaps I'm not using any inline struct definitions?) Here's a link, in case there's anything useful to you in my .travis.yml ? https://github.com/jimsmart/store4 https://github.com/jimsmart/store4 HTH
- kjksf 8y ago
  It looks like we're both doing the same thing. I don't know, maybe I'm not counting things right but for example https://codecov.io/gh/ravendb/ravendb-go-client/src/master/delete_attachment_command_data.go https://codecov.io/gh/ravendb/ravendb-go-client/src/master/d... shows less then 80% coverage and there are only 2 lines not exected out of at least 18, which should be at most 10% counted as not covered.
  - jimsmart 8y ago
    In the example you link, there are 18 coloured lines, 4 of which are not green: 4/18 = ~0.22 = ~22%. This tallies-up as expected with the 77.78% coverage shown at the top of the file. Codecov* doesn't count an 'if' statement as having full coverage unless one tests both outcomes: so the yellow lines here have been executed, but do not count towards your coverage score. Granted, one could argue that that's not very generous! But on the other hand: those yellow lines have not been fully tested, despite being executed, so I can understand their decision. In the linked code, just implement a couple of simple tests to test for the expected error conditions: it's easy (here at least) and ensures the code behaves as expected. (Obviously not all partial/no coverage lines will be so easy to hit with tests, it might not always be possible to easily get 100% coverage, but hey: start with the low hanging fruit!) * I say Codecov here, but I highly suspect that they may simply be using Go's coverage reports under the hood?