4 ms·

One thing that really annoys me about Github/Gitlab et al is that they don't provide a nice UI to show CI results in a structured way. There are semi-standard

One thing that really annoys me about Github/Gitlab et al is that they don't provide a nice UI to show CI results in a structured way. There are semi-standard XML formats that can be used to provide file position, severity and message, which could easily be produced from CI actions and would give devs and reviewers a great view of failing tests, compiler and linter warnings, ... With links to the files etc. Instead we are still stuck with scanning text logs to figure out what failed. I always assumed this is not implemented to eventually upsell something automated. This is still a great feature though, which will probably prevent a large amount of bugs/vulnerabilities, assuming they can minimize false positives. To give credit where it is due, I'd also note that most of Githubs new features since the acquisition were already present in Gitlab [1]. Github will be able to commit way more resources to make it polished, though. [1] https://about.gitlab.com/stages-devops-lifecycle/secure/ https://about.gitlab.com/stages-devops-lifecycle/secure/ Edit: apparently both Gitlab and Github have at least a limited version of this now, although Gitlabs implementation seems much nicer. See below.

Rockslide 7y ago
But GitHub actions do exactly support that in the form of code annotations, see e.g. here https://github.com/ember-template-lint/ember-template-lint/pull/966 https://github.com/ember-template-lint/ember-template-lint/p... - no need to scan logs if you do that, you get the comments right on the code.
- the_duke 7y ago
  Do you have a docs link for this functionality? I can't find anything. The way of reporting seems somewhat odd, and the UI seems a bit limited, but it's a start! Edit: https://help.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-error-message https://help.github.com/en/actions/reference/workflow-comman...
  - judge2020 7y ago
    Integration and usage seem to be a limiting factor but there's some gap bridging happening with GH actions. This Action works pretty great for python's flake8 - https://github.com/marketplace/actions/run-flake8-on-your-pr-with-annotations https://github.com/marketplace/actions/run-flake8-on-your-pr...
kuschku 7y ago
Of course gitlab supports this: https://git.kuschku.de/justJanne/QuasselDroid-ng/pipelines/563/test_report https://git.kuschku.de/justJanne/QuasselDroid-ng/pipelines/5... It can automatically parse most XML formats, JUnit for example.
- sytse 7y ago
  Thanks! Docs for the GitLab feature to show CI results in a structured way are on https://docs.gitlab.com/ee/ci/junit_test_reports.html https://docs.gitlab.com/ee/ci/junit_test_reports.html GitHub Codescanning functionality is best compared to what GitLab has in GitLab SAST https://docs.gitlab.com/ee/user/application_security/sast/ https://docs.gitlab.com/ee/user/application_security/sast/ and Secret Detection https://docs.gitlab.com/ee/user/application_security/sast/#secret-detection https://docs.gitlab.com/ee/user/application_security/sast/#s...
  - deleted 7y ago
    [deleted]
- the_duke 7y ago
  I stand corrected, and gladly so! This looks perfect.
shoyer 7y ago
Surprisingly Azure pipelines, which I understand is used underneath GitHub actions, does support report CI results in this nice way on GitHub, e.g., see these results from a recent PR on one of my projects: https://github.com/pydata/xarray/pull/4017/checks?check_run_id=633135948 https://github.com/pydata/xarray/pull/4017/checks?check_run_... For now, at least, it seems like this is one reason not to switch to GitHub actions yet.
- 616c 7y ago
  Heard this suggested a few times but what is the source?
pabs3 7y ago
There is a standard for this: Static Analysis Results Interchange Format (SARIF): https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v2.0-csprd01.html https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v...
- tandr 7y ago
  Do you know who supports it?
  - Harvesterify 7y ago
    Given that it's a pretty young standard, nobody for now, but there is an open ticket to support it on GitLab : https://gitlab.com/gitlab-org/gitlab/-/issues/118496 https://gitlab.com/gitlab-org/gitlab/-/issues/118496

Rockslide 7y ago

But GitHub actions do exactly support that in the form of code annotations, see e.g. here https://github.com/ember-template-lint/ember-template-lint/pull/966 https://github.com/ember-template-lint/ember-template-lint/p... - no need to scan logs if you do that, you get the comments right on the code.

the_duke 7y ago
Do you have a docs link for this functionality? I can't find anything. The way of reporting seems somewhat odd, and the UI seems a bit limited, but it's a start! Edit: https://help.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-error-message https://help.github.com/en/actions/reference/workflow-comman...
- judge2020 7y ago
  Integration and usage seem to be a limiting factor but there's some gap bridging happening with GH actions. This Action works pretty great for python's flake8 - https://github.com/marketplace/actions/run-flake8-on-your-pr-with-annotations https://github.com/marketplace/actions/run-flake8-on-your-pr...

kuschku 7y ago

Of course gitlab supports this: https://git.kuschku.de/justJanne/QuasselDroid-ng/pipelines/563/test_report https://git.kuschku.de/justJanne/QuasselDroid-ng/pipelines/5... It can automatically parse most XML formats, JUnit for example.

sytse 7y ago
Thanks! Docs for the GitLab feature to show CI results in a structured way are on https://docs.gitlab.com/ee/ci/junit_test_reports.html https://docs.gitlab.com/ee/ci/junit_test_reports.html GitHub Codescanning functionality is best compared to what GitLab has in GitLab SAST https://docs.gitlab.com/ee/user/application_security/sast/ https://docs.gitlab.com/ee/user/application_security/sast/ and Secret Detection https://docs.gitlab.com/ee/user/application_security/sast/#secret-detection https://docs.gitlab.com/ee/user/application_security/sast/#s...
- deleted 7y ago
  [deleted]
the_duke 7y ago
I stand corrected, and gladly so! This looks perfect.

shoyer 7y ago

Surprisingly Azure pipelines, which I understand is used underneath GitHub actions, does support report CI results in this nice way on GitHub, e.g., see these results from a recent PR on one of my projects: https://github.com/pydata/xarray/pull/4017/checks?check_run_id=633135948 https://github.com/pydata/xarray/pull/4017/checks?check_run_... For now, at least, it seems like this is one reason not to switch to GitHub actions yet.

616c 7y ago
Heard this suggested a few times but what is the source?

pabs3 7y ago

There is a standard for this: Static Analysis Results Interchange Format (SARIF): https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v2.0-csprd01.html https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v...

tandr 7y ago
Do you know who supports it?
- Harvesterify 7y ago
  Given that it's a pretty young standard, nobody for now, but there is an open ticket to support it on GitLab : https://gitlab.com/gitlab-org/gitlab/-/issues/118496 https://gitlab.com/gitlab-org/gitlab/-/issues/118496