4 ms·

Oscar, an open-source contributor agent architecture

175 points by theptip 2y ago

lmeyerov 2y ago
It sounds like the indexer is go-specific, while aigen and others are going multilingual via treesitter (I believe) I'm curious if there are any projects folks like for the indexing/embedding step, like Git repo -> vector index / graph index of code + comments + docs? (I am not as interested in the RAG, LLM, UX, etc after)
- codelion 2y ago
  We have a patchflow that does that in patchwork - https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork it is called ResolveIssue - https://github.com/patched-codes/patchwork/tree/main/patchwork/patchflows/ResolveIssue https://github.com/patched-codes/patchwork/tree/main/patchwo...
  - lmeyerov 2y ago
    afaict patchwork also sits on top of treesitter for indexing ? https://github.com/patched-codes/patchwork/blob/b24a3ee07040af2e7f657262892262ca906560ea/patchwork/common/context_strategy/protocol.py https://github.com/patched-codes/patchwork/blob/b24a3ee07040...
    - codelion 2y ago
      Yes, we also use treesitter.
LeSaucy 2y ago
Very interesting to see if they are successful, and/or if this type of maintainer load reducing could be adapted to something like ubuntu/debian where maintaining packages is quite time consuming.
- codelion 2y ago
  We have a few large open source projects already using patchwork to help manage issues and PRs - https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork
- remram 2y ago
  The value of Debian/Ubuntu is that there are people vetting and packaging and testing software. If you remove that and have AI do it, there isn't much left.
  - derefr 2y ago
    This agent isn't claiming to displace that work; more like act as the contribution equivalent of Tier 1 Support in a customer-service setting (where the actual project maintainers would then be Tier 2 Support). Such agents would mostly provide guidance to jumping through the hoops necessary to get your PR in a state where it's mergeable according to the project's contribution guidelines — removing the need for humans to be that guide. (In other words, they'd act as a "compiler for your PR" that you could iterate on, with good English-language error messages. IMHO something that should already exist locally — but git has no local reified concept of PRs, so this is hard.) But I would expect that in almost every case, the project's human maintainers would still eventually step in to review the PR, after the bot seems happy with it. --- Mind you, in theory, for a particular set of limited-scope (but frequently occurring) problems, Tier 1 Support agents are usually empowered to solve those problems directly for a customer. And likewise, there could potentially be a set of limited-scope contribution types that a Tier 1 Project Contribution Auditor would be able to directly approve. Things like, say, fixing typos in doc comments (gated by the bot determining that the diff increases semantic validity of the text by some weird LLM "parseability" metric.)
    - remram 2y ago
      I am not sure what you are talking about, doesn't sound like Debian/Ubuntu package maintenance.
deleted 2y ago
[deleted]
seveibar 2y ago
I think they should expand the scope to direct contribution for easy issues (right now Oscar seems mostly for surfacing project information to contributors), I've had a lot of luck using aider + sonnet for direct contribution, and I'm pretty sure you could do it at scale for "getting-started" issues. [1] Github bot that does direct contributions https://github.com/tscircuit/bunaider https://github.com/tscircuit/bunaider [2] Example contribution https://github.com/tscircuit/checks/pull/8 https://github.com/tscircuit/checks/pull/8
- jclulow 2y ago
  Dear god I want off Mr Bone's Wild Ride. I can't wait for even more pull request spam, now fully automated.
  - matheusmoreira 2y ago
    The ride never ends.
- jamilton 2y ago
  I think it's good that they're focusing on all the not-code parts of contributing, as you say there are other projects for code contributions.
- codelion 2y ago
  We also tried to implement a GH Issue to PR workflow in patchwork - https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork It is a bit hard to get it to work reliably except for small changes.
- keybored 2y ago
  > > Oscar differs from many development-focused uses of LLMs by not trying to augment or displace the code writing process at all. After all, writing code is the fun part of writing software.
John374McCray 2y ago
[dead]
deleted 2y ago
[deleted]
anotherpaulg 2y ago
I feel this pain as a solo maintainer of a somewhat popular open source project. More users brings more questions in GitHub issues and Discord. That’s one reason I added interactive help [0] to aider. Users can type `/help <question>` in the app to get AI help, based on all of aider’s docs. I’ve been considering turning it into a bot for GitHub issues and discord. Dosu [1] is another app in this “triage issues” space that looks really good when I encounter it in the wild. [0] https://aider.chat/docs/troubleshooting/support.html https://aider.chat/docs/troubleshooting/support.html [1] https://dosu.dev/ https://dosu.dev/
- muixoozie 2y ago
  Have you tried to apply for GitHub Sponsors? Kinda surprised to see aider.chat is solo maintained. I keep seeing it as one of the best at what it does. Though I haven't tried it out much. I'm waiting for NixOS packaging of it [0]. Note someone wants aider bad enough they put a bounty on it getting packaged in nixpkgs. - [0] https://github.com/NixOS/nixpkgs/issues/330726 https://github.com/NixOS/nixpkgs/issues/330726
- bogwog 2y ago
  I think it would be much more effective if "/help <question>" just redirected users to a search engine. That won't find stuff on Discord of course, which is a good reason to not use Discord at all. No need to use some elaborate AI solution for such a simple (and long-time solved) problem.
  - anotherpaulg 2y ago
    The interactive help has the full context of the user's current aider session: what they're working on in the coding chat, active config settings, their OS & shell, etc. Aider can tailor the help response to exactly their question and situation. It also always links to relevant docs in these help responses. I've pasted doc links in reply to user questions in discord, and I've pasted /help output. The latter is far more helpful, and includes the links as a bonus.
  - eddd-ddde 2y ago

175 points by theptip 2y ago

It sounds like the indexer is go-specific, while aigen and others are going multilingual via treesitter (I believe) I'm curious if there are any projects folks like for the indexing/embedding step, like Git repo -> vector index / graph index of code + comments + docs? (I am not as interested in the RAG, LLM, UX, etc after)

codelion 2y ago
We have a patchflow that does that in patchwork - https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork it is called ResolveIssue - https://github.com/patched-codes/patchwork/tree/main/patchwork/patchflows/ResolveIssue https://github.com/patched-codes/patchwork/tree/main/patchwo...
- lmeyerov 2y ago
  afaict patchwork also sits on top of treesitter for indexing ? https://github.com/patched-codes/patchwork/blob/b24a3ee07040af2e7f657262892262ca906560ea/patchwork/common/context_strategy/protocol.py https://github.com/patched-codes/patchwork/blob/b24a3ee07040...
  - codelion 2y ago
    Yes, we also use treesitter.

LeSaucy 2y ago

Very interesting to see if they are successful, and/or if this type of maintainer load reducing could be adapted to something like ubuntu/debian where maintaining packages is quite time consuming.

codelion 2y ago
We have a few large open source projects already using patchwork to help manage issues and PRs - https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork
remram 2y ago
The value of Debian/Ubuntu is that there are people vetting and packaging and testing software. If you remove that and have AI do it, there isn't much left.
- derefr 2y ago
  This agent isn't claiming to displace that work; more like act as the contribution equivalent of Tier 1 Support in a customer-service setting (where the actual project maintainers would then be Tier 2 Support). Such agents would mostly provide guidance to jumping through the hoops necessary to get your PR in a state where it's mergeable according to the project's contribution guidelines — removing the need for humans to be that guide. (In other words, they'd act as a "compiler for your PR" that you could iterate on, with good English-language error messages. IMHO something that should already exist locally — but git has no local reified concept of PRs, so this is hard.) But I would expect that in almost every case, the project's human maintainers would still eventually step in to review the PR, after the bot seems happy with it. --- Mind you, in theory, for a particular set of limited-scope (but frequently occurring) problems, Tier 1 Support agents are usually empowered to solve those problems directly for a customer. And likewise, there could potentially be a set of limited-scope contribution types that a Tier 1 Project Contribution Auditor would be able to directly approve. Things like, say, fixing typos in doc comments (gated by the bot determining that the diff increases semantic validity of the text by some weird LLM "parseability" metric.)
  - remram 2y ago
    I am not sure what you are talking about, doesn't sound like Debian/Ubuntu package maintenance.

deleted 2y ago

[deleted]

seveibar 2y ago

I think they should expand the scope to direct contribution for easy issues (right now Oscar seems mostly for surfacing project information to contributors), I've had a lot of luck using aider + sonnet for direct contribution, and I'm pretty sure you could do it at scale for "getting-started" issues. [1] Github bot that does direct contributions https://github.com/tscircuit/bunaider https://github.com/tscircuit/bunaider [2] Example contribution https://github.com/tscircuit/checks/pull/8 https://github.com/tscircuit/checks/pull/8

jclulow 2y ago
Dear god I want off Mr Bone's Wild Ride. I can't wait for even more pull request spam, now fully automated.
- matheusmoreira 2y ago
  The ride never ends.
jamilton 2y ago
I think it's good that they're focusing on all the not-code parts of contributing, as you say there are other projects for code contributions.
codelion 2y ago
We also tried to implement a GH Issue to PR workflow in patchwork - https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork It is a bit hard to get it to work reliably except for small changes.
keybored 2y ago
> > Oscar differs from many development-focused uses of LLMs by not trying to augment or displace the code writing process at all. After all, writing code is the fun part of writing software.

John374McCray 2y ago

[dead]

deleted 2y ago

[deleted]

anotherpaulg 2y ago

I feel this pain as a solo maintainer of a somewhat popular open source project. More users brings more questions in GitHub issues and Discord. That’s one reason I added interactive help [0] to aider. Users can type `/help <question>` in the app to get AI help, based on all of aider’s docs. I’ve been considering turning it into a bot for GitHub issues and discord. Dosu [1] is another app in this “triage issues” space that looks really good when I encounter it in the wild. [0] https://aider.chat/docs/troubleshooting/support.html https://aider.chat/docs/troubleshooting/support.html [1] https://dosu.dev/ https://dosu.dev/

muixoozie 2y ago
Have you tried to apply for GitHub Sponsors? Kinda surprised to see aider.chat is solo maintained. I keep seeing it as one of the best at what it does. Though I haven't tried it out much. I'm waiting for NixOS packaging of it [0]. Note someone wants aider bad enough they put a bounty on it getting packaged in nixpkgs. - [0] https://github.com/NixOS/nixpkgs/issues/330726 https://github.com/NixOS/nixpkgs/issues/330726
bogwog 2y ago
I think it would be much more effective if "/help <question>" just redirected users to a search engine. That won't find stuff on Discord of course, which is a good reason to not use Discord at all. No need to use some elaborate AI solution for such a simple (and long-time solved) problem.
- anotherpaulg 2y ago
  The interactive help has the full context of the user's current aider session: what they're working on in the coding chat, active config settings, their OS & shell, etc. Aider can tailor the help response to exactly their question and situation. It also always links to relevant docs in these help responses. I've pasted doc links in reply to user questions in discord, and I've pasted /help output. The latter is far more helpful, and includes the links as a bonus.
- eddd-ddde 2y ago