10 ms·

GPT based tool that writes the commit message for you

153 points by guytv 4y ago

amelius 4y ago
Wait, if the tool can derive the commit message from the delta, then why does it need to be part of the commit? You could improve the algorithm behind the commit message later.
- killthebuddha 4y ago
  This is where my thoughts went as well: it's like a data structure that includes 2 numbers and their sum. Then I thought that this particular operation is not just a sum and having the memoized value could make sense: using GPT to generate the commit message for a few thousand commits would be pretty expensive.
  - Zvez 4y ago
    Yep, this should go to commit metadata or at the very least as an additional part of commit message with proper marking.
baq 4y ago
Examples! Please!
- adamkl 4y ago
  https://medium.com/@knaan.harpaz/leverage-openais-language-model-for-automated-commit-summaries-8181cef30375 https://medium.com/@knaan.harpaz/leverage-openais-language-m...
tablespoon 4y ago
> Are you a software engineer looking for ways to make your workflow more efficient, or tired of manually sifting through long commit logs to understand pull requests? Introducing the GPT summarizer GitHub action: a powerful tool that leverages OpenAI’s latest and greatest large language model to generate concise and informative summaries of the entire pull request, as well as descriptions of the changes to individual files and individual commits. This kind of stupid. 1. At best, all this can do is look at what you actually committed, but that's not going to tell you the most useful stuff that should be included in a commit message, like why you're making the change in the first place and what your intent was (which may deviate from what you actually did). 2. If you wrote the damn commits, you should be able to describe them yourself. If you can't, something's wrong. The only case where I can see a tool like this being useful, instead of a bad habit, is summarizing old commits, where the committer didn't put anything useful in the commit message.
- maximilianroos 4y ago
  "Let GPT write the description, so you can focus on the explanation" would be better
- baq 4y ago
  It isn’t stupid if it works. If this allows an engineer to spend 15 mins less time while generating good enough PR descriptions, that’s a huge win, especially if the developer isn’t a native English speaker. IME people after getting whatever they’re working on to pass tests don’t feel like writing prose about their work. This would significantly reduce the burden of work which is perceived ‘low value’ and ‘boring’ by most (yes I do review code and yes I will reject a PR without a summary).
  - tablespoon 4y ago
    > It isn’t stupid if it works. If this allows an engineer to spend 15 mins less time while generating good enough PR descriptions, that’s a huge win... It can't work, unless it's hooked up to a brain scanner and can read your thoughts. It seems more like dumb thing that people who don't know what they're doing will think is smart.
    - baq 4y ago
      You see a tool which doesn’t do all you want it to do and say it’s useless where it clearly can make the act of actually doing what you want simpler and faster. I don’t understand why you think it’ll do 100% if 80% is already good enough and very helpful to get the last 20% done.
      - tablespoon 4y ago
        It's a path to laziness and skipping the 20% that's actually helpful in favor of a useless 80%. I see a useful use case for using a tool like this after the commit messages are written (e.g. auto-generate these summaries on the fly when browsing history), but using it to write commit messages themselves is a terrible idea.
jefftk 4y ago
Information that can be automatically extracted from the delta is the least important part of a good commit message. I can figure that part out from looking at the commit later. Instead, what I really want to know is why you made the change. What is the context? What problem were you solving? Why this way and not other ways you might have solved the problem? A summary of what the change does is still useful, and I'm glad this can now be automated, but it's not the main benefit of a well written commit message.
- nunobrito 4y ago
  Was looking at the examples (link posted at another comment) and this addon seems like a good starting point for an accurate commit message. Starting point. After that it should up to the committer to make modifications and upstream the message. For those cases, seems good enough for me.
- Karunamon 4y ago
  What I have always been told for writing good commit messages is that the message should tell you what the commit does when applied to the code. The place for these what/why messages would appear to be in a ticketing system of some kind, referenced by ID in the commit message. What you're asking for would result in extremely bloated messages.
  - singpolyma3 4y ago
    There's no meaningful limit on message length for a reason. Don't bury the context in some external system that may change or go offline, let it live in the commit where it is safe and easily accessible.
  - baq 4y ago
    On the contrary, git commit messages and checked in docs are the only places that are guaranteed not to be lost to history. Ticketing systems change, get migrated to with data loss or are access limited. Git is forever. The commit message is a much better place for explaining what was done why than some obscure Jira instance.
  - 4y ago
greatpostman 4y ago
The crazy thing is chatgpt is still primitive. A decade more of working with these models will be stunning
dub 4y ago
I'd be more excited to use GPT to draft a summary of release notes by scanning all the new PRs in a release, summarizing what they are, and dividing them up into categories (bug fix, feature, breaking changes, etc.)
- eddiequinn 4y ago
  I've been using conventional commits for that, if there was equivalent of this that conformed to the CC standard then I would give it a try
deegles 4y ago
Love the concept, but don't run this on company code without approval!
fragmede 4y ago
While we're making tools for developers, make it so that I can yell at my computer, have it run voice recognition on it, and have ChatGPT interact with Jira for me.
- deleted 4y ago
  [deleted]
rdxm 4y ago
civopsec 4y ago
I wouldn’t be at all surprised if this improved the level of commit messages overall.
soofgo 4y ago
I made an `npx` command to add this awesome tool to codebases: `npx add-gpt-summarizer@latest` The CLI will walk you through all the setup steps. source code: https://github.com/soof-golan/add-gpt-summarizer https://github.com/soof-golan/add-gpt-summarizer
eCa 4y ago
In addition to the other issues (eg. it can only explain the what) I'm low-key disappointed that the repository isn't using itself. It would undoubtedly have lead to an increase in commit message quality here.
odo1242 4y ago
One problem with this I can see is that it's quite easy to write a delta that's larger than 4096 tokens minus prompt length (the maximum amount of context gpt-3 can process), and if the delta's so short it's less than that then it's not particularly necessary to use the AI to generate the commit anyway.
moyix 4y ago
Is there a name for the phenomenon where new automation is disparaged by comparing it to the best human outputs, rather than considering what the average human output looks like and asking if it's a net improvement on that? Like, in this case, I would guess that the vast majority of commit messages are something like "foo", "fixed", "argh", "stupid CI thing try 10", etc. Pretty sure GPT can improve on those.
- Kuinox 4y ago
  I see your point and I agree, this would be a great improvement compared to all my 'wip' commits.
furyofantares 4y ago
I'll just start writing bad commits now and assume that robots will fix it up later. In fact I might as well start many years ago.
xorcist 4y ago
I can absolutely see how a tool like this could provide a useful hook, to reject any commit messages that do not provide information above what GPT3 can do.
- choward 4y ago
  And how would you determine that's it's worse? More AI?
gkfasdfasdf 4y ago
See also: https://github.com/RomanHotsiy/commitgpt https://github.com/RomanHotsiy/commitgpt
- romanhotsiy 4y ago
  Thanks for mentioning! I posted it to "Show HN" last week but it didn't get lots of attention back then.
- freedomben 4y ago
  Nice! Thanks for sharing. This feels far more useful to me than OP. The idea of having to push your code to github to have it generate the commit message makes me immensely sad (even for repos that are already on github, but especially when they are on gitlab).
tonnydourado 4y ago
I get the appeal of offloading toil to AI. But gotta be careful to not mistake thinking for toil. Sometimes thinking is hard and not fun but that doesn't make it toil. Writing good commit messages is thinking, not toil. 90% of the time is even easy, one line description, good to go. The other 10% is when you gotta do the thinking, though. Extracting the relevant changes from the incidental ones, summarizing the whys, the pros and cons, the weird interactions that lead to the specific choices made in the code. This is not toil, it's thinking. It's not even a matter of whether AI can do it well (for this 10%, I'm *very* skeptical it can), the issue is that this is the kind of thinking you should *want* to do, that you should want to get better at. TL;DR: this feels like asking the computer to jog for you and expect to loose weight.
sequoia 4y ago
These things are going to drive me crazy. I think stuff generated by an AI like this should have to be tagged. It’s infuriating to waste one’s time trying to read an interpret a message (or some code), trying to understand the author’s intent, only to realize the author merely pressed a “generate a bunch of code/text” button and I’m spending time trying to interpret shit the author didn’t even spend time writing. My experience with this hitherto relates to terrible scaffolded/generated code. I’m scratching my head wondering “why did s/he do it this way?” only to realize the nonsense code was lazily churned out by some generator. I wish this stuff was labeled clearly.
romanhotsiy 4y ago
The title is a little bit misleading. It doesn't write commit messages, it generates PR summary.
choward 4y ago
It seems to me that a lot AI is used as either tool to solve made up problems or as a bad solution to real problems. This seems like the former. Who is the target audience? I'm baffled.
Niksko 4y ago
Please add functionality so that you can show some sort of badge on your GitHub profile based on your use of this. It would be a great tool to sort out who actually understands the purpose of a commit message.

153 points by guytv 4y ago

Wait, if the tool can derive the commit message from the delta, then why does it need to be part of the commit? You could improve the algorithm behind the commit message later.

killthebuddha 4y ago
This is where my thoughts went as well: it's like a data structure that includes 2 numbers and their sum. Then I thought that this particular operation is not just a sum and having the memoized value could make sense: using GPT to generate the commit message for a few thousand commits would be pretty expensive.
- Zvez 4y ago
  Yep, this should go to commit metadata or at the very least as an additional part of commit message with proper marking.

baq 4y ago

Examples! Please!

adamkl 4y ago
https://medium.com/@knaan.harpaz/leverage-openais-language-model-for-automated-commit-summaries-8181cef30375 https://medium.com/@knaan.harpaz/leverage-openais-language-m...

tablespoon 4y ago

> Are you a software engineer looking for ways to make your workflow more efficient, or tired of manually sifting through long commit logs to understand pull requests? Introducing the GPT summarizer GitHub action: a powerful tool that leverages OpenAI’s latest and greatest large language model to generate concise and informative summaries of the entire pull request, as well as descriptions of the changes to individual files and individual commits. This kind of stupid. 1. At best, all this can do is look at what you actually committed, but that's not going to tell you the most useful stuff that should be included in a commit message, like why you're making the change in the first place and what your intent was (which may deviate from what you actually did). 2. If you wrote the damn commits, you should be able to describe them yourself. If you can't, something's wrong. The only case where I can see a tool like this being useful, instead of a bad habit, is summarizing old commits, where the committer didn't put anything useful in the commit message.

maximilianroos 4y ago
"Let GPT write the description, so you can focus on the explanation" would be better
baq 4y ago
It isn’t stupid if it works. If this allows an engineer to spend 15 mins less time while generating good enough PR descriptions, that’s a huge win, especially if the developer isn’t a native English speaker. IME people after getting whatever they’re working on to pass tests don’t feel like writing prose about their work. This would significantly reduce the burden of work which is perceived ‘low value’ and ‘boring’ by most (yes I do review code and yes I will reject a PR without a summary).
- tablespoon 4y ago
  > It isn’t stupid if it works. If this allows an engineer to spend 15 mins less time while generating good enough PR descriptions, that’s a huge win... It can't work, unless it's hooked up to a brain scanner and can read your thoughts. It seems more like dumb thing that people who don't know what they're doing will think is smart.
  - baq 4y ago
    You see a tool which doesn’t do all you want it to do and say it’s useless where it clearly can make the act of actually doing what you want simpler and faster. I don’t understand why you think it’ll do 100% if 80% is already good enough and very helpful to get the last 20% done.
    - tablespoon 4y ago
      It's a path to laziness and skipping the 20% that's actually helpful in favor of a useless 80%. I see a useful use case for using a tool like this after the commit messages are written (e.g. auto-generate these summaries on the fly when browsing history), but using it to write commit messages themselves is a terrible idea.

jefftk 4y ago

Information that can be automatically extracted from the delta is the least important part of a good commit message. I can figure that part out from looking at the commit later. Instead, what I really want to know is why you made the change. What is the context? What problem were you solving? Why this way and not other ways you might have solved the problem? A summary of what the change does is still useful, and I'm glad this can now be automated, but it's not the main benefit of a well written commit message.

nunobrito 4y ago
Was looking at the examples (link posted at another comment) and this addon seems like a good starting point for an accurate commit message. Starting point. After that it should up to the committer to make modifications and upstream the message. For those cases, seems good enough for me.
Karunamon 4y ago
What I have always been told for writing good commit messages is that the message should tell you what the commit does when applied to the code. The place for these what/why messages would appear to be in a ticketing system of some kind, referenced by ID in the commit message. What you're asking for would result in extremely bloated messages.
- singpolyma3 4y ago
  There's no meaningful limit on message length for a reason. Don't bury the context in some external system that may change or go offline, let it live in the commit where it is safe and easily accessible.
- baq 4y ago
  On the contrary, git commit messages and checked in docs are the only places that are guaranteed not to be lost to history. Ticketing systems change, get migrated to with data loss or are access limited. Git is forever. The commit message is a much better place for explaining what was done why than some obscure Jira instance.
- 4y ago

greatpostman 4y ago

The crazy thing is chatgpt is still primitive. A decade more of working with these models will be stunning

dub 4y ago

I'd be more excited to use GPT to draft a summary of release notes by scanning all the new PRs in a release, summarizing what they are, and dividing them up into categories (bug fix, feature, breaking changes, etc.)

eddiequinn 4y ago
I've been using conventional commits for that, if there was equivalent of this that conformed to the CC standard then I would give it a try

deegles 4y ago

Love the concept, but don't run this on company code without approval!

fragmede 4y ago

While we're making tools for developers, make it so that I can yell at my computer, have it run voice recognition on it, and have ChatGPT interact with Jira for me.

deleted 4y ago
[deleted]

rdxm 4y ago

civopsec 4y ago

I wouldn’t be at all surprised if this improved the level of commit messages overall.

soofgo 4y ago

I made an `npx` command to add this awesome tool to codebases: `npx add-gpt-summarizer@latest` The CLI will walk you through all the setup steps. source code: https://github.com/soof-golan/add-gpt-summarizer https://github.com/soof-golan/add-gpt-summarizer

eCa 4y ago

In addition to the other issues (eg. it can only explain the what) I'm low-key disappointed that the repository isn't using itself. It would undoubtedly have lead to an increase in commit message quality here.

odo1242 4y ago

One problem with this I can see is that it's quite easy to write a delta that's larger than 4096 tokens minus prompt length (the maximum amount of context gpt-3 can process), and if the delta's so short it's less than that then it's not particularly necessary to use the AI to generate the commit anyway.

moyix 4y ago

Is there a name for the phenomenon where new automation is disparaged by comparing it to the best human outputs, rather than considering what the average human output looks like and asking if it's a net improvement on that? Like, in this case, I would guess that the vast majority of commit messages are something like "foo", "fixed", "argh", "stupid CI thing try 10", etc. Pretty sure GPT can improve on those.

Kuinox 4y ago
I see your point and I agree, this would be a great improvement compared to all my 'wip' commits.

furyofantares 4y ago

I'll just start writing bad commits now and assume that robots will fix it up later. In fact I might as well start many years ago.

xorcist 4y ago

I can absolutely see how a tool like this could provide a useful hook, to reject any commit messages that do not provide information above what GPT3 can do.

choward 4y ago
And how would you determine that's it's worse? More AI?

gkfasdfasdf 4y ago

See also: https://github.com/RomanHotsiy/commitgpt https://github.com/RomanHotsiy/commitgpt

romanhotsiy 4y ago
Thanks for mentioning! I posted it to "Show HN" last week but it didn't get lots of attention back then.
freedomben 4y ago
Nice! Thanks for sharing. This feels far more useful to me than OP. The idea of having to push your code to github to have it generate the commit message makes me immensely sad (even for repos that are already on github, but especially when they are on gitlab).

tonnydourado 4y ago

I get the appeal of offloading toil to AI. But gotta be careful to not mistake thinking for toil. Sometimes thinking is hard and not fun but that doesn't make it toil. Writing good commit messages is thinking, not toil. 90% of the time is even easy, one line description, good to go. The other 10% is when you gotta do the thinking, though. Extracting the relevant changes from the incidental ones, summarizing the whys, the pros and cons, the weird interactions that lead to the specific choices made in the code. This is not toil, it's thinking. It's not even a matter of whether AI can do it well (for this 10%, I'm *very* skeptical it can), the issue is that this is the kind of thinking you should *want* to do, that you should want to get better at. TL;DR: this feels like asking the computer to jog for you and expect to loose weight.

sequoia 4y ago

These things are going to drive me crazy. I think stuff generated by an AI like this should have to be tagged. It’s infuriating to waste one’s time trying to read an interpret a message (or some code), trying to understand the author’s intent, only to realize the author merely pressed a “generate a bunch of code/text” button and I’m spending time trying to interpret shit the author didn’t even spend time writing. My experience with this hitherto relates to terrible scaffolded/generated code. I’m scratching my head wondering “why did s/he do it this way?” only to realize the nonsense code was lazily churned out by some generator. I wish this stuff was labeled clearly.

romanhotsiy 4y ago

The title is a little bit misleading. It doesn't write commit messages, it generates PR summary.

choward 4y ago

It seems to me that a lot AI is used as either tool to solve made up problems or as a bad solution to real problems. This seems like the former. Who is the target audience? I'm baffled.

Niksko 4y ago

Please add functionality so that you can show some sort of badge on your GitHub profile based on your use of this. It would be a great tool to sort out who actually understands the purpose of a commit message.