3 ms·

Interesting project for sure but I'm trying to find the reasons for shifting the AI to the PR stage, wouldn't this be more efficient at development time? I.e co

Interesting project for sure but I'm trying to find the reasons for shifting the AI to the PR stage, wouldn't this be more efficient at development time? I.e copilot/openAI tool chain?

hunterbrooks 2y ago
Teams should use both Copilot (synchronous code generation) and Ellipsis (async code gen). Sure, Copilot speeds up human dev productivity, but our take is that humans should only be spending their time on the highest value code changes and use products like Ellipsis to handle the rest. The downside of async code gen is that Ellipsis workflows take a few minutes to run because Ellipsis is actually building the project, running the tests, fixing it's mistakes, etc. The upside is that a developer can have multiple workflows running at once, and each workflow delivers higher quality code because it's guaranteed to be working + tested. I'm super bullish on async code gen. I think there's a whole category of tedious development tasks with unambiguous solutions that can be automated to the point where a human just needs to give it a LGTM.
- mirsadm 2y ago
  Do you use your own product in that way?
  - hunterbrooks 2y ago
    Yeah, I use it for lots of boilerplate work like adding new API endpoints, new Celery jobs, and building react components.
danenania 2y ago
I think we're going to see different AI tools optimized for different stages of the development workflow, with developers assembling and using a collection of them, just like they currently assemble a collection of tools for different tasks within the stack (backend language, frontend language, database, cache, infrastructure, etc.). It's unlikely that there's ever going to be one AI coding tool to rule them all. As someone building (and frequently using) an AI coding tool that is very much focused on development time[1], I still also use GH Copilot and ChatGPT plus heavily as well. In a team setting, I could definitely see using my tool in conjunction with Ellipsis too. A feature built partly (or entirely) with AI still needs PR review. And an agent focused specifically on that job is likely going to be better at it than an agent designed for building new features from scratch. 1 - https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex
- hunterbrooks 2y ago
  Definitely agree with this. One analogy I see today is the typical testing pipeline. Unit tests get run locally, then maybe a CI job runs unit tests + integration tests, then maybe there's a deployment job which does a blue/green release. At every stage the system is being "tested", but because the tests validate different capabilities, it's like concentric circles that grow confidence for the change. A software dev lifecycle that uses AI dev tools is similar. Agents will review/contribute at the various stages of the SDLC, sometimes with overlap, but mostly additive and building on the output of one another.

Interesting project for sure but I'm trying to find the reasons for shifting the AI to the PR stage, wouldn't this be more efficient at development time? I.e copilot/openAI tool chain?

hunterbrooks 2y ago

Teams should use both Copilot (synchronous code generation) and Ellipsis (async code gen). Sure, Copilot speeds up human dev productivity, but our take is that humans should only be spending their time on the highest value code changes and use products like Ellipsis to handle the rest. The downside of async code gen is that Ellipsis workflows take a few minutes to run because Ellipsis is actually building the project, running the tests, fixing it's mistakes, etc. The upside is that a developer can have multiple workflows running at once, and each workflow delivers higher quality code because it's guaranteed to be working + tested. I'm super bullish on async code gen. I think there's a whole category of tedious development tasks with unambiguous solutions that can be automated to the point where a human just needs to give it a LGTM.

mirsadm 2y ago
Do you use your own product in that way?
- hunterbrooks 2y ago
  Yeah, I use it for lots of boilerplate work like adding new API endpoints, new Celery jobs, and building react components.

danenania 2y ago

I think we're going to see different AI tools optimized for different stages of the development workflow, with developers assembling and using a collection of them, just like they currently assemble a collection of tools for different tasks within the stack (backend language, frontend language, database, cache, infrastructure, etc.). It's unlikely that there's ever going to be one AI coding tool to rule them all. As someone building (and frequently using) an AI coding tool that is very much focused on development time[1], I still also use GH Copilot and ChatGPT plus heavily as well. In a team setting, I could definitely see using my tool in conjunction with Ellipsis too. A feature built partly (or entirely) with AI still needs PR review. And an agent focused specifically on that job is likely going to be better at it than an agent designed for building new features from scratch. 1 - https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex

hunterbrooks 2y ago
Definitely agree with this. One analogy I see today is the typical testing pipeline. Unit tests get run locally, then maybe a CI job runs unit tests + integration tests, then maybe there's a deployment job which does a blue/green release. At every stage the system is being "tested", but because the tests validate different capabilities, it's like concentric circles that grow confidence for the change. A software dev lifecycle that uses AI dev tools is similar. Agents will review/contribute at the various stages of the SDLC, sometimes with overlap, but mostly additive and building on the output of one another.