4 ms·

sure hope so, especially with copilot labs brushes.. imagine highlighting some code and dumping the contents of a jira ticket into a text prompt.. these days a

sure hope so, especially with copilot labs brushes.. imagine highlighting some code and dumping the contents of a jira ticket into a text prompt.. these days are coming.

steve_adams_86 4y ago
I’m currently slapping together something like this, though the highlighting of code is automated – in a sense – when an error is piped into the program and the error stack is used as a prompt to find a solution to the error. The goal is to splice a solution into a file on a new branch, write a test to ensure it resolves the issue (if necessary), then create a pull request describing the change set and how the test addresses the issue. I’ve so far worked through it manually and it works okay, but for fairly limited scopes. Like, why didn’t this environmental variable load properly? Why couldn’t we parse this input? Why did this time out? No big picture stuff. I have a feeling GitHub will be adding this directly to repositories so initial solutions and ideas (perhaps even fixes) can be generated by an AI trained on your repo and its issues/PRs/dependencies. I’m not doing this with aspirations of building a successful tool so much as learning to think with AI as a tool for improving systems. It’s a very tractable thing at the moment and I think future iterations of GPT will make it truly useful.
- jerkstate 4y ago
  I agree, it seems inevitable that github would include this sort of auto-resolve functionality. I have had good luck with GPT outputting unified diff and LSP format. As the neighbor comment mentions, context window will be a limitation especially with large codebases - I'm thinking for that, it will be necessary to either be able to fine-tune or to summarize the rest of the code base and take multiple requests, and store the results of these intermediate summaries. I can construct this sort of thing by hand on larger codebases already, surely it can be automated to a large degree.
kashnote 4y ago
I think the problem will be context and token limit size. Right now, GitHub Copilot uses the current file “and related files” as context. But if you have several thousand files, even GPT-4 won’t be able to handle that. Either you need to cleverly know exactly which files to feed in for context or have a very very high token limit size.

sure hope so, especially with copilot labs brushes.. imagine highlighting some code and dumping the contents of a jira ticket into a text prompt.. these days are coming.

steve_adams_86 4y ago

I’m currently slapping together something like this, though the highlighting of code is automated – in a sense – when an error is piped into the program and the error stack is used as a prompt to find a solution to the error. The goal is to splice a solution into a file on a new branch, write a test to ensure it resolves the issue (if necessary), then create a pull request describing the change set and how the test addresses the issue. I’ve so far worked through it manually and it works okay, but for fairly limited scopes. Like, why didn’t this environmental variable load properly? Why couldn’t we parse this input? Why did this time out? No big picture stuff. I have a feeling GitHub will be adding this directly to repositories so initial solutions and ideas (perhaps even fixes) can be generated by an AI trained on your repo and its issues/PRs/dependencies. I’m not doing this with aspirations of building a successful tool so much as learning to think with AI as a tool for improving systems. It’s a very tractable thing at the moment and I think future iterations of GPT will make it truly useful.

jerkstate 4y ago
I agree, it seems inevitable that github would include this sort of auto-resolve functionality. I have had good luck with GPT outputting unified diff and LSP format. As the neighbor comment mentions, context window will be a limitation especially with large codebases - I'm thinking for that, it will be necessary to either be able to fine-tune or to summarize the rest of the code base and take multiple requests, and store the results of these intermediate summaries. I can construct this sort of thing by hand on larger codebases already, surely it can be automated to a large degree.

kashnote 4y ago

I think the problem will be context and token limit size. Right now, GitHub Copilot uses the current file “and related files” as context. But if you have several thousand files, even GPT-4 won’t be able to handle that. Either you need to cleverly know exactly which files to feed in for context or have a very very high token limit size.