4 ms·

Show HN: A local merge queue for parallel Claude Code agents

42 points by funador 2mo ago

I have been pushing up to 90 commits a day on a MacBook Air via 4-5 parallel agents. As you can imagine when all the agents try to build, test and run dev servers on an 8GB machine it is the fast lane to a force quit and restart. I also did not want to pay the CI minutes on 90 pushes a day. So I designed a local merge queue to have all commits land one at a time and fully tested. Hopefully this helps other folks with more modest machines. Appreciate any feedback.

orsorna 2mo ago
I have CI set up so that I can just let that handle builds and exclusive locks for me, but there are particular limitations (network connectivity is mandatory). I did this because I wanted to avoid something like you built, which theoretically can't scale as well as a dev box (much easier to extend deploy environments versus my local machine). I can see that you are working within resource constraints though which is admirable :)
- funador 2mo ago
  Exactly, have to shave a few corners when your laptop doesn't even have a fan :)
alikhater30000 2mo ago
[flagged]
barrkel 2mo ago
A decent idea, however it seems to me a good chunk of it is because of git limitations. Have you looked jj? I'm having a much better time with jj and a workspace per subagent than I was with git and worktrees. It's still useful to have the CI gating on updating the master branch pointer, but you largely stop working with branches once you switch to jj.
- funador 2mo ago
  Giving up branches is the dream. This feels like the nudge I needed to give jj a proper go. Thank you.
- kazinator 2mo ago
  git worktrees are a horrible misfeature; it's better to just clone multiple times (which you can do locally---space is saved by use of hard links!) So that is to say, we clone some remote upstream down to /path/A. Then we can clone file:///path/A to /path/B locally. Both A and B are fully fleded git repos; no monkeying around with worktrees. The objects are shared between them with hard links. You can edit B/.gitconfig to point it to have the same upstream as A for pushing. Unlike worktrees, you can blow away a repo with "rm -rf", and not worry that you are pulling the rug from under something which depends on it. If I have several trees, A, B, C, one of which is the parent, while the other are worktrees, removing any one of them randomly is Russian roulette: if we do "rm -rf B" and that happens to be the parent repo, worktrees A and C are no longer functional and need to be recovered. If A, B, and C are independent clones (even with objects hardlinked among themselves), this isn't a problem.
  - yiyingzhang 2mo ago
    What you described sounds like this: https://github.com/GenseeAI/gensee-crate https://github.com/GenseeAI/gensee-crate. It offers whole workspace fork and merge. But the merge is still somewhat cumbersome and require manual overseeing.
  - ithkuil 2mo ago
    You're right about everything you said, but there is a flip side: with work trees you can: 1. find/list all related "clones" of the repo 2. Ensure that the same branch is not being worked on in different checkouts (which is useful if you do rebases as part of your workflow) Worktrees force you distinguish the "main" repo from the swarm of checkouts, and if you locate or name directories with a rule you can always tell which are which. For my own use, I built a little helper to create, list and destroy worktrees that fit in my workflow with Claude code: https://github.com/mkmik/ccwt https://github.com/mkmik/ccwt
damlab 2mo ago
[flagged]
ElijahLynn 2mo ago
Definitely going to look into this! I just built something pretty much in the same vein of this with Claude over the past 2 months. It's a custom deployment system that uses work trees and each work tree has to to run end-to-end tests (Playright, Supabase, Astro) and other tests and create a stamp based on the contents of the entire tree. And then if another work tree has the same digest that matches the stamp then it can bypass the tests. So nothing can land on main without tests passing. Which is all in in enforced with a pre-push hook that checks the digest. Then once it's on main it is deployed to Dev and then from there it can be promoted to prod. I like the idea of just having one branch and the work trees funnel into it. But definitely going to have my setup analyze this and see what can be improved with it or if I should just throw out mine and use that one.
- funador 2mo ago
  Sweet, let me know how it goes. Really appreciate any feedback
throwaw12 2mo ago
idea looks nice, but can you share what's your workflow to push 90 commits/day? I understand each commit might have different size, but I am guessing your commits == Pull Request (because you need this constant merge queue to rebase, merge, test) and each are around 40-80 lines of code (additions and removals), and you are probably not reviewing the code because at this rate even if one commit takes 2 minutes to review we are talking about 3 hours of non stop code review. if you are not reviewing each code, probably your commits are small items and 10-20 of them are a part of single PR, then how do you sustain working across multiple projects?
- funador 2mo ago
  Definitely not reviewing the code when 90 commits go in a day :) I have local test suite of unit / arch / e2e / build / lint / tsc when it merges to dev. Then the same suite runs on a merge to main which pushes to prod. I have post main-push e2e that checks for regressions and automatically rolls back. Each suite run takes 3-4 minutes so I can usually push through 15 or so commits an hour fully tested and cleanly. I do not use PRs with this workflow, and if something breaks I add tests to prevent it happening again rather than altering the setup. Most of the commits are for a side project (https://hola.career/ https://hola.career/) so I afford to a bit more loose than I would in professional capacity.
sqemo 2mo ago
[flagged]
wudmaing00 2mo ago
[flagged]
dpc_01234 2mo ago
SelfCI: https://radicle.network/nodes/radicle.dpc.pw/rad:z2tDzYbAXxTQEKTGFVwiJPajkbeDU https://radicle.network/nodes/radicle.dpc.pw/rad:z2tDzYbAXxT... , supports git & jj
- funador 2mo ago
  That looks like a much slicker version, will check that out
vasanthrb 2mo ago
One thing I'm wondering: have you tried this on a team with multiple humans as well, or is it mainly optimized for a single developer coordinating a bunch of Claude Code agents?
- funador 2mo ago
  There are a ton of places this package would not be suitable :) I have not used this on a team, and what you would end up with is likely a giant PR with all those local commits that no one wants to review. Eventually someone might hold their nose and press the approve button. But that would probably be the end of the package at that workplace. This is optimized for one engineer that does not have the machine capacity to run multiple lanes of their check test suites locally at once and also does want to pay for GHA minutes on a free plan to run those suites in the cloud.
otekengineering 2mo ago
set up a channel for agents to communicate and you shouldn't have issues with messy merges. your force quitting is probably more from your build pipeline than anything agents are doing. i use an m2 8gb air with no performance issues at up to 10 parallel claude code interactive sessions (ghostty), each with deep/transient subagent heirarchies that mix claude and codex. very rarely need to force quit. only seen a couple restart-level failures, and those were on me for being cowboy with subagent delegation logic, resulting in 100s of claude moles popping up faster than i could whack them with force close.
- 4b11b4 2mo ago
  In codex threads can inspect each other, rename themselves , send each other messages. communication is default primitive
- funador 2mo ago
  I do have a pretty heavy pipe line of local tests and running more than one of those at once is what spurred me to write this package. Especially the build step.
deleted 2mo ago
[deleted]
4b11b4 2mo ago
Lately I just use an isolated worktree per conversation and I have all these threads sitting which need to be merged then I just let codex orchestrate that itself. I tell each thread to name itself with a prefix and it's picked up by a merge queue thread which instructs them merge one at a time. Each thread has to look at the main ref as it's changed since they did their work and determine what kind of merge needs to happen. To safely merge a a variety of merge types I have a commit skill which details the process for forward reconstruction, for example. As of yesterday I have a single voice orchestrator thread which then can poke other threads as well as the merge queue thread. Sometimes I just do stuff in working copy when I know exactly where in the history it's going to land (I rewrite history extensively to know cleanly where that it) and I'll git absorb / jj absorb it manually.
- ThomasSchijf 2mo ago
  [dead]
felixlu2026 2mo ago
[dead]

42 points by funador 2mo ago

orsorna 2mo ago

I have CI set up so that I can just let that handle builds and exclusive locks for me, but there are particular limitations (network connectivity is mandatory). I did this because I wanted to avoid something like you built, which theoretically can't scale as well as a dev box (much easier to extend deploy environments versus my local machine). I can see that you are working within resource constraints though which is admirable :)

funador 2mo ago
Exactly, have to shave a few corners when your laptop doesn't even have a fan :)

alikhater30000 2mo ago

[flagged]

barrkel 2mo ago

A decent idea, however it seems to me a good chunk of it is because of git limitations. Have you looked jj? I'm having a much better time with jj and a workspace per subagent than I was with git and worktrees. It's still useful to have the CI gating on updating the master branch pointer, but you largely stop working with branches once you switch to jj.

funador 2mo ago
Giving up branches is the dream. This feels like the nudge I needed to give jj a proper go. Thank you.
kazinator 2mo ago
git worktrees are a horrible misfeature; it's better to just clone multiple times (which you can do locally---space is saved by use of hard links!) So that is to say, we clone some remote upstream down to /path/A. Then we can clone file:///path/A to /path/B locally. Both A and B are fully fleded git repos; no monkeying around with worktrees. The objects are shared between them with hard links. You can edit B/.gitconfig to point it to have the same upstream as A for pushing. Unlike worktrees, you can blow away a repo with "rm -rf", and not worry that you are pulling the rug from under something which depends on it. If I have several trees, A, B, C, one of which is the parent, while the other are worktrees, removing any one of them randomly is Russian roulette: if we do "rm -rf B" and that happens to be the parent repo, worktrees A and C are no longer functional and need to be recovered. If A, B, and C are independent clones (even with objects hardlinked among themselves), this isn't a problem.
- yiyingzhang 2mo ago
  What you described sounds like this: https://github.com/GenseeAI/gensee-crate https://github.com/GenseeAI/gensee-crate. It offers whole workspace fork and merge. But the merge is still somewhat cumbersome and require manual overseeing.
- ithkuil 2mo ago
  You're right about everything you said, but there is a flip side: with work trees you can: 1. find/list all related "clones" of the repo 2. Ensure that the same branch is not being worked on in different checkouts (which is useful if you do rebases as part of your workflow) Worktrees force you distinguish the "main" repo from the swarm of checkouts, and if you locate or name directories with a rule you can always tell which are which. For my own use, I built a little helper to create, list and destroy worktrees that fit in my workflow with Claude code: https://github.com/mkmik/ccwt https://github.com/mkmik/ccwt

damlab 2mo ago

[flagged]

ElijahLynn 2mo ago

Definitely going to look into this! I just built something pretty much in the same vein of this with Claude over the past 2 months. It's a custom deployment system that uses work trees and each work tree has to to run end-to-end tests (Playright, Supabase, Astro) and other tests and create a stamp based on the contents of the entire tree. And then if another work tree has the same digest that matches the stamp then it can bypass the tests. So nothing can land on main without tests passing. Which is all in in enforced with a pre-push hook that checks the digest. Then once it's on main it is deployed to Dev and then from there it can be promoted to prod. I like the idea of just having one branch and the work trees funnel into it. But definitely going to have my setup analyze this and see what can be improved with it or if I should just throw out mine and use that one.

funador 2mo ago
Sweet, let me know how it goes. Really appreciate any feedback

throwaw12 2mo ago

idea looks nice, but can you share what's your workflow to push 90 commits/day? I understand each commit might have different size, but I am guessing your commits == Pull Request (because you need this constant merge queue to rebase, merge, test) and each are around 40-80 lines of code (additions and removals), and you are probably not reviewing the code because at this rate even if one commit takes 2 minutes to review we are talking about 3 hours of non stop code review. if you are not reviewing each code, probably your commits are small items and 10-20 of them are a part of single PR, then how do you sustain working across multiple projects?

funador 2mo ago
Definitely not reviewing the code when 90 commits go in a day :) I have local test suite of unit / arch / e2e / build / lint / tsc when it merges to dev. Then the same suite runs on a merge to main which pushes to prod. I have post main-push e2e that checks for regressions and automatically rolls back. Each suite run takes 3-4 minutes so I can usually push through 15 or so commits an hour fully tested and cleanly. I do not use PRs with this workflow, and if something breaks I add tests to prevent it happening again rather than altering the setup. Most of the commits are for a side project (https://hola.career/ https://hola.career/) so I afford to a bit more loose than I would in professional capacity.

sqemo 2mo ago

[flagged]

wudmaing00 2mo ago

[flagged]

dpc_01234 2mo ago

SelfCI: https://radicle.network/nodes/radicle.dpc.pw/rad:z2tDzYbAXxTQEKTGFVwiJPajkbeDU https://radicle.network/nodes/radicle.dpc.pw/rad:z2tDzYbAXxT... , supports git & jj

funador 2mo ago
That looks like a much slicker version, will check that out

vasanthrb 2mo ago

One thing I'm wondering: have you tried this on a team with multiple humans as well, or is it mainly optimized for a single developer coordinating a bunch of Claude Code agents?

funador 2mo ago
There are a ton of places this package would not be suitable :) I have not used this on a team, and what you would end up with is likely a giant PR with all those local commits that no one wants to review. Eventually someone might hold their nose and press the approve button. But that would probably be the end of the package at that workplace. This is optimized for one engineer that does not have the machine capacity to run multiple lanes of their check test suites locally at once and also does want to pay for GHA minutes on a free plan to run those suites in the cloud.

otekengineering 2mo ago

set up a channel for agents to communicate and you shouldn't have issues with messy merges. your force quitting is probably more from your build pipeline than anything agents are doing. i use an m2 8gb air with no performance issues at up to 10 parallel claude code interactive sessions (ghostty), each with deep/transient subagent heirarchies that mix claude and codex. very rarely need to force quit. only seen a couple restart-level failures, and those were on me for being cowboy with subagent delegation logic, resulting in 100s of claude moles popping up faster than i could whack them with force close.

4b11b4 2mo ago
In codex threads can inspect each other, rename themselves , send each other messages. communication is default primitive
funador 2mo ago
I do have a pretty heavy pipe line of local tests and running more than one of those at once is what spurred me to write this package. Especially the build step.

deleted 2mo ago

[deleted]

4b11b4 2mo ago

Lately I just use an isolated worktree per conversation and I have all these threads sitting which need to be merged then I just let codex orchestrate that itself. I tell each thread to name itself with a prefix and it's picked up by a merge queue thread which instructs them merge one at a time. Each thread has to look at the main ref as it's changed since they did their work and determine what kind of merge needs to happen. To safely merge a a variety of merge types I have a commit skill which details the process for forward reconstruction, for example. As of yesterday I have a single voice orchestrator thread which then can poke other threads as well as the merge queue thread. Sometimes I just do stuff in working copy when I know exactly where in the history it's going to land (I rewrite history extensively to know cleanly where that it) and I'll git absorb / jj absorb it manually.

ThomasSchijf 2mo ago
[dead]

felixlu2026 2mo ago

[dead]