4 ms·

Ugh. One of my pet peeves is the generation of release notes from commit messages. Commit messages and PR descriptions have a different audience (i.e. contribut

by Vinnl 5y ago

Ugh. One of my pet peeves is the generation of release notes from commit messages. Commit messages and PR descriptions have a different audience (i.e. contributors) from release notes (i.e. users). For example, take a look at ESLint's autogenerated changelog [1]: 67c0074 Update: Suggest missing rule in flat config (fixes #14027) (#15074) (Nicholas C. Zakas) cf34e5c Update: space-before-blocks ignore after switch colons (fixes #15082) (#15093) (Milos Djermanovic) c9efb5f Fix: preserve formatting when rules are removed from disable directives (#15081) (Milos Djermanovic) 14a4739 Update: no-new-func rule catching eval case of MemberExpression (#14860) (Mojtaba Samimi) 7f2346b Docs: Update release blog post template (#15094) (Nicholas C. Zakas) fabdf8a Chore: Remove target.all from Makefile.js (#15088) (Hirotaka Tagawa / wafuwafu13) e3cd141 Sponsors: Sync README with website (ESLint Jenkins) 05d7140 Chore: document target global in Makefile.js (#15084) (Hirotaka Tagawa / wafuwafu13) 0a1a850 Update: include ruleId in error logs (fixes #15037) (#15053) (Ari Perkkiö) 47be800 Chore: test Property > .key with { a = 1 } pattern (fixes #14799) (#15072) (Milos Djermanovic) a744dfa Docs: Update CLA info (#15058) (Brian Warner) 9fb0f70 Chore: fix bug report template (#15061) (Milos Djermanovic) f87e199 Chore: Cleanup issue templates (#15039) (Nicholas C. Zakas) I'm reading release notes to get a feel for how the new release might impact me. This takes so much time to scan, because there's so much useless cruft (to me, as a user) I have to ignore. What's worked very well for me is to simply have an "I updated the changelog, if applicable" entry in my PR template checklist. Then when I cut a new release, I simply add the release date above the release notes currently listed under "Unreleased", and they'll list all relevant changes, reviewed during the pull request to verify that it is relevant to users. [1] https://github.com/eslint/eslint/blob/master/CHANGELOG.md https://github.com/eslint/eslint/blob/master/CHANGELOG.md

madeofpalk 5y ago
I agree - all commits to release notes doesn't seem very helpful. We have a workflow that will only add to the release notes PRs that have the "Add to changelog" label which works okay, but sometimes you do have to decide who you want to address your PR title to.
steerablesafe 5y ago
It feels like a missed opportunity of more automation, TBH. Notice that each commit mentions one or more bugs, probably enforced by commit hooks. What about dropping the ones that don't fix bugs (assuming they are part of ongoing work), replace the ones that fixes the bug with the description of the bug (and a working link!), and categorize them based on the tags of the bug (defect, feature request, etc...) so you would automatically get a list like: New Features: "title of feature request 1" "title of feature request 2" Bugs fixed: "bug1" "bug2"
- Vinnl 5y ago
  Even then, in a commit message/PR description I (as a contributor) expect to read the origin/big picture of the bug, how it was discovered/narrowed down, references to where it was introduced, what other solutions were tried — those kinds of things. Those aren't relevant to users though.
- mylesborins 5y ago
  The Release Notes are organized by PR and you can use configuration to sort into sections based on labels on those PRs. You can also ignore labels / particular users. So in theory everything you are outlining here is possible https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes#example-configuration https://docs.github.com/en/repositories/releasing-projects-o...
  - Vinnl 5y ago
    > in theory everything you are outlining here is possible I think part of the problem is also that in theory, it's already possible to write a good changelog as well — but of course, what's relevant is what happens in practice :)
OJFord 5y ago
That's just an extension of my (I believe common) pet peeve of commit subjects/PR titles that describe 'what' rather than 'why' isn't it?
- Vinnl 5y ago
  I guess so - I suppose contributors should be interested in the 'why', whereas users are more interested in the 'what' and, more importantly, 'how will this affect me?'
  - OJFord 5y ago
    Oh I meant the opposite - a 'what' description (bad) being just an English re-run of the code in the commit ('add missing indentation') vs. 'why' that explains what problem it fixed/feature it added ('fix syntax error on navigating to page blah'). I meant that a good 'why' commit subject that gets propagated to release notes is more likely helpful to an end user - ooh nice yes I have had that problem, cool it's fixed, I don't care exactly what fixed it.
    - toomanybeersies 5y ago
      My expectations these days are so low, I'll settle for commit messages a full sentence, of any kind. I don't even get hung up about messages not being written in the imperative mood any more. At least 'add missing indentation' is a few steps better than 'changes', or 'fixed', or dozens of commits with 'WIP' as the commit message.
    - pwdisswordfish0 5y ago
      The "commit subject" should be a re-run of the code in the commit—condensed into single, 50-chars or less line. It's the rest of the commit message that should contain the detailed context, which includes the why, but most GitHub users are awful at both the commit summaries and writing the rest of the message. The "collaboration" that takes place on GitHub is bottom-of-the-barrel, and you shouldn't expect anything other than dregs there.
maxekman 5y ago
Are you sure they list the individual commits? It looks like they summarize PRs when watching the demo screen capture. Will be interesting to see how useful it is for an open source project with diverse commit/PR message levels.
- Vinnl 5y ago
  That's still a different target audience. Often my PR description will just be my commit message, though every now and then a PR consists of several commits and the PR description guides the reviewer through them and outlines a suggested way to review it, for example. Not at all what you'd want in release notes.
e3bc54b2 5y ago
This is why I believe Nate Graham does a fantastic job with weekly KDE updates[0]. He actually goes through the pain to condense every bugfix/feature addition to readable and grokkable single line, and it makes for great changelog read. Of course, not every project can manage to write such a thoughtful post for every release, but if Mr. Graham can do it for something as sprawling as KDE, then can sure at least try. [0] https://pointieststick.com/ https://pointieststick.com/
epage 5y ago
Its better than a project having no notes and trying to identify the range of commits manually, especially when there are breaking changes and trying to figure out how to update.
mylesborins 5y ago
I guess it is a good thing we made automated release notes on GitHub use PRs instead of commits then ; )
- veverkap 5y ago
  The OP shouldn't be the highest voted comment.
  - deleted 5y ago
    [deleted]
- Vinnl 5y ago
  I did explicitly call out that both commits and PRs have a different target audience (i.e. contributors) than release notes, because that is the root of the problem in my view. PS. I do appreciate the effort btw! Better release notes in general is definitely something to strive for, and I hope you'll keep a tight pulse on how this affects what release notes on average look like and whether they get better, and of course how they can get even better :)
- majora2007 5y ago
  This is what got me so excited because I ensure my PRs have a user-friendly markdown Added/Fixed/Changed style. Can't wait to test this out for my next release.
tlamponi 5y ago
> I'm reading release notes to get a feel for how the new release might impact me. This takes so much time to scan, because there's so much useless cruft (to me, as a user) I have to ignore. so true, I even wrote a small script to drop the obvious cruft: https://git.proxmox.com/?p=pve-eslint.git;a=blob;f=debian/scripts/prepare-changes.sh;h=b6f1df5a0808b8953c567d59df43b80f7d0a37cb;hb=HEAD https://git.proxmox.com/?p=pve-eslint.git;a=blob;f=debian/sc... (I know I could make that shorter, but I explicitly split some sed statements to make it easier to read, it's a few lines only anyway)

by Vinnl 5y ago

madeofpalk 5y ago

I agree - all commits to release notes doesn't seem very helpful. We have a workflow that will only add to the release notes PRs that have the "Add to changelog" label which works okay, but sometimes you do have to decide who you want to address your PR title to.

steerablesafe 5y ago

It feels like a missed opportunity of more automation, TBH. Notice that each commit mentions one or more bugs, probably enforced by commit hooks. What about dropping the ones that don't fix bugs (assuming they are part of ongoing work), replace the ones that fixes the bug with the description of the bug (and a working link!), and categorize them based on the tags of the bug (defect, feature request, etc...) so you would automatically get a list like: New Features: "title of feature request 1" "title of feature request 2" Bugs fixed: "bug1" "bug2"

Vinnl 5y ago
Even then, in a commit message/PR description I (as a contributor) expect to read the origin/big picture of the bug, how it was discovered/narrowed down, references to where it was introduced, what other solutions were tried — those kinds of things. Those aren't relevant to users though.
mylesborins 5y ago
The Release Notes are organized by PR and you can use configuration to sort into sections based on labels on those PRs. You can also ignore labels / particular users. So in theory everything you are outlining here is possible https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes#example-configuration https://docs.github.com/en/repositories/releasing-projects-o...
- Vinnl 5y ago
  > in theory everything you are outlining here is possible I think part of the problem is also that in theory, it's already possible to write a good changelog as well — but of course, what's relevant is what happens in practice :)

OJFord 5y ago

That's just an extension of my (I believe common) pet peeve of commit subjects/PR titles that describe 'what' rather than 'why' isn't it?

Vinnl 5y ago
I guess so - I suppose contributors should be interested in the 'why', whereas users are more interested in the 'what' and, more importantly, 'how will this affect me?'
- OJFord 5y ago
  Oh I meant the opposite - a 'what' description (bad) being just an English re-run of the code in the commit ('add missing indentation') vs. 'why' that explains what problem it fixed/feature it added ('fix syntax error on navigating to page blah'). I meant that a good 'why' commit subject that gets propagated to release notes is more likely helpful to an end user - ooh nice yes I have had that problem, cool it's fixed, I don't care exactly what fixed it.
  - toomanybeersies 5y ago
    My expectations these days are so low, I'll settle for commit messages a full sentence, of any kind. I don't even get hung up about messages not being written in the imperative mood any more. At least 'add missing indentation' is a few steps better than 'changes', or 'fixed', or dozens of commits with 'WIP' as the commit message.
  - pwdisswordfish0 5y ago
    The "commit subject" should be a re-run of the code in the commit—condensed into single, 50-chars or less line. It's the rest of the commit message that should contain the detailed context, which includes the why, but most GitHub users are awful at both the commit summaries and writing the rest of the message. The "collaboration" that takes place on GitHub is bottom-of-the-barrel, and you shouldn't expect anything other than dregs there.

maxekman 5y ago

Are you sure they list the individual commits? It looks like they summarize PRs when watching the demo screen capture. Will be interesting to see how useful it is for an open source project with diverse commit/PR message levels.

Vinnl 5y ago
That's still a different target audience. Often my PR description will just be my commit message, though every now and then a PR consists of several commits and the PR description guides the reviewer through them and outlines a suggested way to review it, for example. Not at all what you'd want in release notes.

e3bc54b2 5y ago

This is why I believe Nate Graham does a fantastic job with weekly KDE updates[0]. He actually goes through the pain to condense every bugfix/feature addition to readable and grokkable single line, and it makes for great changelog read. Of course, not every project can manage to write such a thoughtful post for every release, but if Mr. Graham can do it for something as sprawling as KDE, then can sure at least try. [0] https://pointieststick.com/ https://pointieststick.com/

epage 5y ago

Its better than a project having no notes and trying to identify the range of commits manually, especially when there are breaking changes and trying to figure out how to update.

mylesborins 5y ago

I guess it is a good thing we made automated release notes on GitHub use PRs instead of commits then ; )

veverkap 5y ago
The OP shouldn't be the highest voted comment.
- deleted 5y ago
  [deleted]
Vinnl 5y ago
I did explicitly call out that both commits and PRs have a different target audience (i.e. contributors) than release notes, because that is the root of the problem in my view. PS. I do appreciate the effort btw! Better release notes in general is definitely something to strive for, and I hope you'll keep a tight pulse on how this affects what release notes on average look like and whether they get better, and of course how they can get even better :)
majora2007 5y ago
This is what got me so excited because I ensure my PRs have a user-friendly markdown Added/Fixed/Changed style. Can't wait to test this out for my next release.

tlamponi 5y ago

> I'm reading release notes to get a feel for how the new release might impact me. This takes so much time to scan, because there's so much useless cruft (to me, as a user) I have to ignore. so true, I even wrote a small script to drop the obvious cruft: https://git.proxmox.com/?p=pve-eslint.git;a=blob;f=debian/scripts/prepare-changes.sh;h=b6f1df5a0808b8953c567d59df43b80f7d0a37cb;hb=HEAD https://git.proxmox.com/?p=pve-eslint.git;a=blob;f=debian/sc... (I know I could make that shorter, but I explicitly split some sed statements to make it easier to read, it's a few lines only anyway)