6 ms·

Towards Self-Driving Codebases

115 points by wilhelmklopp 2mo ago

rrook 2mo ago
My bet is that we'll see a second layer of harness emerge, as self-driving codebases become the target. There will be an application facing harness, orthogonal to the agent facing harness. The app harness will represent the software factory that is emergent for the specific application being developed. Anyway, here's mine, still wip: https://hale-lang.org/docs/dna/ https://hale-lang.org/docs/dna/ https://github.com/hale-lang/hale/issues/690 https://github.com/hale-lang/hale/issues/690
- cyanydeez 2mo ago
  I suspect it won't be a harness, but just a more specific LLM trained in the universe of user-selected context of vetted resources. Why? Because LLMs are always going to be dumb when they're trained at scale. Their ability to speak software diverges from their friendly user input layer. A harness won't overcome that, but an LLM saddle ontop of a larger model would provide the type of feedback loops you'd want to look into. I don't think you'll find two deterministic systems will produce much.
  - verdverm 2mo ago
    fine-tuning may be a more scalable approach to LLM personalization than sending all the same context to two LLMs I'm working towards both in my homelab to see which works better with little qwen
  - rrook 2mo ago
    I think that dissolves the "self driving" distinction, though? The mechanisms for driving the codebase must be present in the codebase itself. Otherwise its just a regular out-of-band development process.
  - cindyllm 2mo ago
    [dead]
- tdg5 2mo ago
  I’ve definitely had a hunch about a software factory harness kind of thing. Your thoughts on the matter seem clearer than mine. I’m kind of relieved to see so many stages; that’s something I’ve grappled with conceptually. The volume of (sometimes optional) stages has made it hard to resist the temptation to hack something together with a GHA workflow. I salute your choice to support distributed; that’s a requirement in my book. I look forward to trying it out.
  - rrook 2mo ago
    Expressing complex emergent workflows is fairly straightforward given how the workflow engine recurses: - a workflow is a series of steps - a step is one or more tasks that can be done in parallel - a step is complete when all of its tasks are complete - a task can be a unit of work, or a pointer to another workflow
sroerick 2mo ago
This article hints at it, but I think the blocker is environment. How can you self drive an app on windows? There's no clear UI framework, design pattern, nothing which can bullet proof your app. I bet you'd have better luck on Plan 9. Remove the choice. Make frameworks which have limited options which are usable by default.
- TimVasil 2mo ago
  Spot on about environment. I recently built a calculator to try to quantify the gains an org could expect as they become more "AI-native." I tried to account for a bunch of things including env, which I call "AI roadway," but I'm definitely missing codebase readiness (thank you, author), and also bottleneck analysis. Even without those considerations, though, gains are usually modest. It's hard to get to 2x. Tool is here if useful: https://timvasil.com/ai-native https://timvasil.com/ai-native
CharlieDigital 2mo ago
> It’s actually fine if agents make a lot of boneheaded mistakes. What’s not ok is if they keep making the same mistakes. I worked in life sciences for a bit. There is a process in clinical trials called corrective and preventative actions (CAPA). You'll also find this in other areas where failure tolerance is low (e.g. aircraft). It's simple: when a mistake happens, you run your CAPA process (Google CAPA form and see examples to extrapolate what that process might look like) and determine the root cause and the correction to the process that allowed the mistake to happen in the first place. (At least as a SaaS vendor in life sciences, when we had a CAPA (e.g. after a SEV0 failure), it would be folded into our SOPs and then we would be required to retrain on the SOP. Auditors would want to see our evidence of CAPAs, the versions of our SOPs, the records of training. All to extreme for most shops, but I add this for context/color) This is something most eng shops do not have the discipline for since it requires some diligence. Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered.
- drob 2mo ago
  Agreed that domain experts should be making these kinds of decisions, and the right way for an agent mistake to get caught is going to depend a lot on the cost and probability of the mistake. My only point here is that the agent side of the system needs to learn. The software production machine needs to improve over time.
- skinfaxi 2mo ago
  > Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered. I think a point we are grappling with is, what necessitates human intervention, like philosophically. Is it accountability? I was thinking about this in terms of code review and it's not like we would fire someone if they broke prod, so at what point do we need someone accountable. Maybe it's for approvals for certain classes of risk (like those associated with actual harm to life). It's also not lost on me that many human systems lack accountability.
  - jaggederest 2mo ago
drob 2mo ago
Author here, hello! Happy to expand on how we're thinking about this if any of it is unclear. We wrote this post as part of a launch, which you can check out here: https://x.com/danlovesproofs/status/2095182189499711759 https://x.com/danlovesproofs/status/2095182189499711759
writeslowly 2mo ago
I suspect that if you're not careful with agent memory it creates a danger of agent-driven cargo-cult behavior. I've watched this in my own ad-hoc agent loops where it starts with something basic, like the first agent tried to run some gigantic dependency inspection command and OOMed the local JVM and eventually recorded a workaround (to enable it to run gigantic dependency inspection commands...), and by time I get a few more agents into the loop, agents have written entire paragraphs about testing and validating local dev environment memory configurations that are mostly irrelevant to whatever is being worked on. In general I've seen other issues like this where small errors and irrelevant comments in the codebase spin out into larger problems that consume annoying amounts of time/tokens. Maybe Anthropic and OpenAI don't notice this because they're in an "infinite monkeys with typewriters" scenario, but it's noticeable to me when the agent in my CLI has been spinning for 15 minutes contemplating irrelevant details
- memjay 2mo ago
  This happens way too often in my experience and I tried a lot of things. If the agent has access to an existing codebase it will always try to stay as close to it as possible. It doesn’t diverge from the initial direction the project/code went into, even if it’s already clear that this direction goes nowhere. Somehow agents aren’t able to detach themselves from existing code to come up with completely new solutions. Sometimes I feel like it would be best to only give the agent access to clear API boundaries (say public interfaces to certain modules) and let it work out a new system from scratch given the expected inputs and outputs. Then plug this independent solution into the system. Of course it can still overfit these interfaces but it’s less than having access to the entire codebase. Thinking about it, maybe it’s possible to let one agent extract those interfaces, then use them as grounding for a new session. One less nice way of achieving the same is to tell the agent once in a while that it should think completely from scratch (from first principles). But this relies heavily on instruction following in the reasoning part, which sometimes works and sometimes fails.
wrs 2mo ago
This would be more convincing if there were a simple stat on the home page of the dispositions of the reported bugs (fixed/no-repro/won't-fix). As a random sample of one, I looked at one of the bugs this reported on Tailscale (first thing on the homepage) [0], and the pull request ends with "Apologies for the lack of due diligence here. I'll go ahead and close this out." [0] https://github.com/tailscale/tailscale/pull/17843 https://github.com/tailscale/tailscale/pull/17843
- drob 2mo ago
  We should publish some stats. The fix rates on bugs are around 45% baseline, increasing over time as we learn from behavior within a given codebase. The merge rates for the codebase health work we do (e.g. deleting dead code) are very high, at least 90%.
aerodexis 2mo ago
Makes sense to me - I see it going down two separate routes in parallel: One is to take the human out the OODA loop for cyber defense. Servlet libs (for example) are going to become fluid, self-modifying things w/ contracts that operate much differenlty from how they do today. The engineering practice around these things will need to change. The second is that UX will be self-modifying. Just like how pi can modify itself, I can see this being a general practice for user-facing applications. Perhaps a text-box in which users can describe tweaks in can request changes to how the application functions. Engineering an application will focus on modeling the non-negotiables of how an application works, and providing the correct primitives for user-driven LLMs to modify software on the fly, as well as track and rollback changes. There'll also be funsies around how to ensure that ads get delivered regardless of the user trying to get rid of them.
- smallmancontrov 2mo ago
  > fluid, self-modifying things w/ contracts This smells like worm food. Those contracts had better be airtight.
  - deleted 2mo ago
    [deleted]
  - aerodexis 2mo ago
    What would you prefer: 99% of user requests to succeed but w/ 5% risk of getting O-day'd, or have 90% of user requests succeed but w/ 0.1% risk of getting 0-day'd.
    - aerodexis 2mo ago
      I could imagine it going so far as to have an LLM re-impl core libs every time a request is processed. Lightning never strikes the same place twice.
    - bigstrat2003 2mo ago
      The first and it isn't even close. Normal operation of the system is more important than preventing attacks, and trading it away at a 2:1 ratio is a bad trade.
zraiq 2mo ago
The missing piece in most versions of this is the feedback loop, not the harness. A codebase can only drive itself as far as it can tell whether it's still on the road: tests that fail for the right reason, error budgets, a way to notice that a change made something slower rather than broken. Without that, "self-driving" degrades into applying diffs that compile. With it, the interesting question becomes what the system is allowed to do when the signal goes red - revert, retry, or stop and ask - and that's a policy decision nobody has good defaults for yet.
- Tatendaz 2mo ago
  [flagged]
- sfraxo 2mo ago
  [flagged]
geooff_ 2mo ago
If you have a logo in the top left corner of your company blog the logo should route to the home page so I can see what product you're actually building.
ody2 2mo ago
I have integrated AI into my application. The source code is embedded into the binary. When someone questions the numbers in my app it can debug itself. If it truly finds a bug in itself it will tell the user and open a ticket. Soon we are going to let the application fix itself.
- jorl17 2mo ago
  How do you protect intellectual property? Or is this a case of the value being somewhere else, such as in your backend? If so, how does the agent debug frontend and backend? I presume it stops at frontend
jaksa 2mo ago
Not sure about you, but I've been writing real world software using agents for a while now. https://github.com/jaksa76/microfactory https://github.com/jaksa76/microfactory
- owebmaster 2mo ago
  Is this the "real world software" or a tool that creates "real world software"?
ModernMech 2mo ago
It's a losing proposition. Self-driving cars go from point A to point B, and it's really easy to specify point B. My experience so far using agentic AI 24/7 for almost a year has been the AI can drive almost anywhere, so the whole problem is specifying where to drive. This doesn't get solved with faster, better AI, because all that will do is allow it to autonomously drive your codebase to random places you didn't want it to go. Then you're still left with the problem of specifying the problem. But then there's the cases where the AI can't actually drive. What if you tell the agent to invent AGI? Or time travel? Where does it drive to? Does it tell you that it can't? How does it know it can't just drive here? How you you know? At the end of the day these are not wish granting machines, so someone connected to reality is still going to have to make decisions. And that person is going to be the one held liable for whatever the AI does so would they want a self-driving codebase in the first place?
- Sivart13 2mo ago
  Well put. I've found that agents are great at adding features to areas of the code that I already understand, where I can phrase my goals coherently and keep them on the rails. Any attempt to describe software by "specification" leaves enough ambiguity that the agent will do deranged stuff like add a ton of code to satisfy error cases that can't happen. Or, like you imply, it will confidently create a bunch of nonsense to "solve" a problem in a way that is not really possible.
vb-8448 2mo ago
> Global memory I don't get why we need global memory for code? Aren't code comments (even if invented for humans) the ideal place where to put "memories"?
- rrr_oh_man 2mo ago
  LLMs also do not have anything like memory, by design. Everything bolted on that smells like memory or is being called memory is a crutch, at best. Not being pedantic, I just think this aspect is lost on a lot of people.
  - Terr_ 2mo ago
    Also, when we humans are tricked into perceiving a remote mind, we automatically assume it has a memory like us, or at least like large animals. In other words, there's a story-document between a SherlockHolmesBot and UserPlaceholder that's growing like a crystal formation. Some software sees "Sherlock Homes Says" and then "performs" the dialogue at us, and we then assume Sherlock Holmes exists with a mind and memory, rather than being a facet of a text-story.
  - alansaber 2mo ago
    Exactly, this is why we see such minimalistic memory systems being used. Information retrieval is bad, but improving information availability by pre-computing information retrieval is not the obvious fix people think it is.
- drob 2mo ago
  Sometimes, but I think there are a few problems with this: 1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it. 2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look. 3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.
icase 2mo ago
“how can we make ourselves even MORE replaceable?”
- lioeters 2mo ago
  Reminds me of that Richard Scarry children's book illustration of a fox busily sawing off a tree branch he's sitting on.
tonymet 2mo ago
I'm hoping the "blogging about doing" format will fade away, now that doing is cheaper than blogging. It used to be that the doing part was expensive, and risky. Now you can just do 5 things and throw away 4. So I'd rather see the finished product than talk about it.
AIiscoming 2mo ago
I'm playing around with ecosystem driven development (hear me out ;)): So i want claude to build a small game for me so of course i create one big file in which i write everything then i tell claude to analyse it and grill me (grill me skill) to clarify all smaller details. This alone might lead to really good small sfotware but I still have to push it sometimes. Now instead of doing this, i tell claude to build a small tool which generates a dashboard and memory and which can save specs and ask me choicses (do you prefer this color over the other, what do you think about problem A? What solution would you prefer? A, b, c or something else). This does a few things: 1. claude doesn't has to save/store everything in a context 2. claude can now talk to that tool to ask it stuff 3. claude can now use the tool as a todo list 4. claude now can more easily spin up more agents in parallel 5. i have a nice interface and i can solve issues while claude works on unblocked tasks 6. I actually can follow the progress a lot easier The only problem with this is: with the next update, you have to reinvestigate how claude was finetuned and adjust. A few month ago /goal was really good, now you need it a lot less because claude will do something for an hour without /goal And the spec file only started to work after November/Opus moment but it got so good, that i can pack A LOT of stuff in a half structured markdown file and let it code what I need. The progress is still too fast for the whole ecosystem
calldacopsidc 2mo ago
Sorry to get hung up on what is probably just flavor text, but the first sentence.. "Agents can oneshot games that are actually fun", uhhh such as? Would love to be proven wrong here, feels like I've just seen the same basic one-level kart racer in like 3 model announcements. Not even sure if they were one-shotted, I certainly would not describe them as "fun"
- ludamn 2mo ago
  Agreed, I read this and almost instinctively decided to close the page, agents can be used to build games for sure, but that "one shot" and "fun" in the same sentence is really misleading
  - palmotea 2mo ago
    FYI, it looks like you've been banned for the last ~8-9 months, for no obvious reason I can see. I vouched for your comment to make it visible, but they appears to show up as [dead] by default.
    - ramon156 2mo ago
      i can see the comment now, so this seems undone?
    - ludamn 2mo ago
      [dead]
- xyst 2mo ago
  Probably fun defined in the ephemeral context. As-in it was "fun" to play in the context of the short form video attention span, social media brain rotted individual. Then it was thrown away and never played again. It's definitely not fun if you consider building an actual community around it and have a long term plan for it.
  - fragmede 2mo ago
xyst 2mo ago
I currently see a shit ton of work involved with "scaling" (aka cleaning up) the AI generated slop that was output in the past 2-3 years.
kreneskyp 2mo ago
My answer to this is adding an engineering assurance module to my specification and verification toolkit. Every project has a different definition of success and quality. My goal is to provide a consistent way to define, analyze, and report quality metrics tailored to each project. https://github.com/agent-ix/engineering-assurance https://github.com/agent-ix/engineering-assurance https://github.com/agent-ix/quoin https://github.com/agent-ix/quoin
jakeinspace 2mo ago
Kind of feels like we're in a similar place with AI programming as we are with self driving vehicles. Good enough for the typical case, maybe even better than most humans, but bad when dealing with novel situations or situations which require a really cohesive model of the system (code, business requirements, physical world, whatever).
deleted 2mo ago
[deleted]
2001zhaozhao 2mo ago
In the end, we already have processes that work for humans and we know the types of testing, verification and review that makes a codebase grow healthily. We just need to adapt these designs the best we can to AIs. AI have a lot of advantages over humans like speed, controllability, good defaults (e.g. Claude will write helpful comments unprompted which is better than most engineers), and the ability to implement best practices like comprehensive testing with much lower cost than a human team. Using AI in my personal projects, I am able to write software at a much lower bug rate than manually, because now I can make an automated test suite whereas previously I had no time to do so. However, AI also has downsides like rigidity, limited context window, and lack of creativity by default which needs to be resolved with harnessing (e.g. different agents with separate, well-compressed contexts) or worked around with human review layers. I suspect the best core pattern for coding automation will be to create an engineering manager agent layer that can flexibly apply a set of engineering practices to the concrete problems at hand. It would handle all coding agent pings before it reaches a human, with prompting so that it reliably considers the big picture when making decisions. Very rarely, it would escalate an issue or plan review directly to humans when it is truly blocking, but most likely it would organize the information it obtains from the lower layer coding agents and surface them as reports for humans to review.
6thbit 2mo ago
The things in the 'what should be self-driving' section are the exact things i give to juniors so they can build their mental model of the codebase. Now, I agree it'd be fantastic to have that automated, but how do we then share that context with the humans that will drive the non-self-driving actions?
billybones 2mo ago
Super exciting, and absolute killers on the customer testimonial page.
rafaelvasco 2mo ago
Is our fate to become more and more product designers and less and less software engineers? I feel that way. But I'm now able to actually create and release things. Before, I used to obsess over code architecture, optimizing every single line. I could and can craft beautiful code. And then I always burned out and lost interest (gemini). Now I don't touch code at all and have been actually creating and releasing the ideas I've always had in mind. Then I realized that's what I always wanted. The code was always a means to an end. But that's me.
hollowturtle 2mo ago
> Agents can oneshot games that are actually fun. HAHAHAHAHAHAHAHAAH here we go again
- bethekidyouwant 2mo ago
  Joust isn’t fun?
thisisauserid 2mo ago
Sounds like the next shitty Vercel ad at re:Invent.
saiyamshah1496 2mo ago
[dead]
dinobones 2mo ago
> Agents can oneshot games that are actually fun. Stopped reading right there.
zlokki 2mo ago
> Put another way: Detail’s secret plan is to create a product that bootstraps its own work, and then use that work to help engineering teams know what investments to make and how well they’re working. Sorry, guys, I still don't understand what are you selling. Prompts? Design manual for building testable system? In which way agent-friendly dev env differs from human-friendly dev env?

115 points by wilhelmklopp 2mo ago

rrook 2mo ago

My bet is that we'll see a second layer of harness emerge, as self-driving codebases become the target. There will be an application facing harness, orthogonal to the agent facing harness. The app harness will represent the software factory that is emergent for the specific application being developed. Anyway, here's mine, still wip: https://hale-lang.org/docs/dna/ https://hale-lang.org/docs/dna/ https://github.com/hale-lang/hale/issues/690 https://github.com/hale-lang/hale/issues/690

cyanydeez 2mo ago
I suspect it won't be a harness, but just a more specific LLM trained in the universe of user-selected context of vetted resources. Why? Because LLMs are always going to be dumb when they're trained at scale. Their ability to speak software diverges from their friendly user input layer. A harness won't overcome that, but an LLM saddle ontop of a larger model would provide the type of feedback loops you'd want to look into. I don't think you'll find two deterministic systems will produce much.
- verdverm 2mo ago
  fine-tuning may be a more scalable approach to LLM personalization than sending all the same context to two LLMs I'm working towards both in my homelab to see which works better with little qwen
- rrook 2mo ago
  I think that dissolves the "self driving" distinction, though? The mechanisms for driving the codebase must be present in the codebase itself. Otherwise its just a regular out-of-band development process.
- cindyllm 2mo ago
  [dead]
tdg5 2mo ago
I’ve definitely had a hunch about a software factory harness kind of thing. Your thoughts on the matter seem clearer than mine. I’m kind of relieved to see so many stages; that’s something I’ve grappled with conceptually. The volume of (sometimes optional) stages has made it hard to resist the temptation to hack something together with a GHA workflow. I salute your choice to support distributed; that’s a requirement in my book. I look forward to trying it out.
- rrook 2mo ago
  Expressing complex emergent workflows is fairly straightforward given how the workflow engine recurses: - a workflow is a series of steps - a step is one or more tasks that can be done in parallel - a step is complete when all of its tasks are complete - a task can be a unit of work, or a pointer to another workflow

sroerick 2mo ago

This article hints at it, but I think the blocker is environment. How can you self drive an app on windows? There's no clear UI framework, design pattern, nothing which can bullet proof your app. I bet you'd have better luck on Plan 9. Remove the choice. Make frameworks which have limited options which are usable by default.

TimVasil 2mo ago
Spot on about environment. I recently built a calculator to try to quantify the gains an org could expect as they become more "AI-native." I tried to account for a bunch of things including env, which I call "AI roadway," but I'm definitely missing codebase readiness (thank you, author), and also bottleneck analysis. Even without those considerations, though, gains are usually modest. It's hard to get to 2x. Tool is here if useful: https://timvasil.com/ai-native https://timvasil.com/ai-native

CharlieDigital 2mo ago

> It’s actually fine if agents make a lot of boneheaded mistakes. What’s not ok is if they keep making the same mistakes. I worked in life sciences for a bit. There is a process in clinical trials called corrective and preventative actions (CAPA). You'll also find this in other areas where failure tolerance is low (e.g. aircraft). It's simple: when a mistake happens, you run your CAPA process (Google CAPA form and see examples to extrapolate what that process might look like) and determine the root cause and the correction to the process that allowed the mistake to happen in the first place. (At least as a SaaS vendor in life sciences, when we had a CAPA (e.g. after a SEV0 failure), it would be folded into our SOPs and then we would be required to retrain on the SOP. Auditors would want to see our evidence of CAPAs, the versions of our SOPs, the records of training. All to extreme for most shops, but I add this for context/color) This is something most eng shops do not have the discipline for since it requires some diligence. Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered.

drob 2mo ago
Agreed that domain experts should be making these kinds of decisions, and the right way for an agent mistake to get caught is going to depend a lot on the cost and probability of the mistake. My only point here is that the agent side of the system needs to learn. The software production machine needs to improve over time.
skinfaxi 2mo ago
> Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered. I think a point we are grappling with is, what necessitates human intervention, like philosophically. Is it accountability? I was thinking about this in terms of code review and it's not like we would fire someone if they broke prod, so at what point do we need someone accountable. Maybe it's for approvals for certain classes of risk (like those associated with actual harm to life). It's also not lost on me that many human systems lack accountability.
- jaggederest 2mo ago

drob 2mo ago

Author here, hello! Happy to expand on how we're thinking about this if any of it is unclear. We wrote this post as part of a launch, which you can check out here: https://x.com/danlovesproofs/status/2095182189499711759 https://x.com/danlovesproofs/status/2095182189499711759

writeslowly 2mo ago

I suspect that if you're not careful with agent memory it creates a danger of agent-driven cargo-cult behavior. I've watched this in my own ad-hoc agent loops where it starts with something basic, like the first agent tried to run some gigantic dependency inspection command and OOMed the local JVM and eventually recorded a workaround (to enable it to run gigantic dependency inspection commands...), and by time I get a few more agents into the loop, agents have written entire paragraphs about testing and validating local dev environment memory configurations that are mostly irrelevant to whatever is being worked on. In general I've seen other issues like this where small errors and irrelevant comments in the codebase spin out into larger problems that consume annoying amounts of time/tokens. Maybe Anthropic and OpenAI don't notice this because they're in an "infinite monkeys with typewriters" scenario, but it's noticeable to me when the agent in my CLI has been spinning for 15 minutes contemplating irrelevant details

memjay 2mo ago
This happens way too often in my experience and I tried a lot of things. If the agent has access to an existing codebase it will always try to stay as close to it as possible. It doesn’t diverge from the initial direction the project/code went into, even if it’s already clear that this direction goes nowhere. Somehow agents aren’t able to detach themselves from existing code to come up with completely new solutions. Sometimes I feel like it would be best to only give the agent access to clear API boundaries (say public interfaces to certain modules) and let it work out a new system from scratch given the expected inputs and outputs. Then plug this independent solution into the system. Of course it can still overfit these interfaces but it’s less than having access to the entire codebase. Thinking about it, maybe it’s possible to let one agent extract those interfaces, then use them as grounding for a new session. One less nice way of achieving the same is to tell the agent once in a while that it should think completely from scratch (from first principles). But this relies heavily on instruction following in the reasoning part, which sometimes works and sometimes fails.

wrs 2mo ago

This would be more convincing if there were a simple stat on the home page of the dispositions of the reported bugs (fixed/no-repro/won't-fix). As a random sample of one, I looked at one of the bugs this reported on Tailscale (first thing on the homepage) [0], and the pull request ends with "Apologies for the lack of due diligence here. I'll go ahead and close this out." [0] https://github.com/tailscale/tailscale/pull/17843 https://github.com/tailscale/tailscale/pull/17843

drob 2mo ago
We should publish some stats. The fix rates on bugs are around 45% baseline, increasing over time as we learn from behavior within a given codebase. The merge rates for the codebase health work we do (e.g. deleting dead code) are very high, at least 90%.

aerodexis 2mo ago

Makes sense to me - I see it going down two separate routes in parallel: One is to take the human out the OODA loop for cyber defense. Servlet libs (for example) are going to become fluid, self-modifying things w/ contracts that operate much differenlty from how they do today. The engineering practice around these things will need to change. The second is that UX will be self-modifying. Just like how pi can modify itself, I can see this being a general practice for user-facing applications. Perhaps a text-box in which users can describe tweaks in can request changes to how the application functions. Engineering an application will focus on modeling the non-negotiables of how an application works, and providing the correct primitives for user-driven LLMs to modify software on the fly, as well as track and rollback changes. There'll also be funsies around how to ensure that ads get delivered regardless of the user trying to get rid of them.

smallmancontrov 2mo ago
> fluid, self-modifying things w/ contracts This smells like worm food. Those contracts had better be airtight.
- deleted 2mo ago
  [deleted]
- aerodexis 2mo ago
  What would you prefer: 99% of user requests to succeed but w/ 5% risk of getting O-day'd, or have 90% of user requests succeed but w/ 0.1% risk of getting 0-day'd.
  - aerodexis 2mo ago
    I could imagine it going so far as to have an LLM re-impl core libs every time a request is processed. Lightning never strikes the same place twice.
  - bigstrat2003 2mo ago
    The first and it isn't even close. Normal operation of the system is more important than preventing attacks, and trading it away at a 2:1 ratio is a bad trade.

zraiq 2mo ago

The missing piece in most versions of this is the feedback loop, not the harness. A codebase can only drive itself as far as it can tell whether it's still on the road: tests that fail for the right reason, error budgets, a way to notice that a change made something slower rather than broken. Without that, "self-driving" degrades into applying diffs that compile. With it, the interesting question becomes what the system is allowed to do when the signal goes red - revert, retry, or stop and ask - and that's a policy decision nobody has good defaults for yet.

Tatendaz 2mo ago
[flagged]
sfraxo 2mo ago
[flagged]

geooff_ 2mo ago

If you have a logo in the top left corner of your company blog the logo should route to the home page so I can see what product you're actually building.

ody2 2mo ago

I have integrated AI into my application. The source code is embedded into the binary. When someone questions the numbers in my app it can debug itself. If it truly finds a bug in itself it will tell the user and open a ticket. Soon we are going to let the application fix itself.

jorl17 2mo ago
How do you protect intellectual property? Or is this a case of the value being somewhere else, such as in your backend? If so, how does the agent debug frontend and backend? I presume it stops at frontend

jaksa 2mo ago

Not sure about you, but I've been writing real world software using agents for a while now. https://github.com/jaksa76/microfactory https://github.com/jaksa76/microfactory

owebmaster 2mo ago
Is this the "real world software" or a tool that creates "real world software"?

ModernMech 2mo ago

It's a losing proposition. Self-driving cars go from point A to point B, and it's really easy to specify point B. My experience so far using agentic AI 24/7 for almost a year has been the AI can drive almost anywhere, so the whole problem is specifying where to drive. This doesn't get solved with faster, better AI, because all that will do is allow it to autonomously drive your codebase to random places you didn't want it to go. Then you're still left with the problem of specifying the problem. But then there's the cases where the AI can't actually drive. What if you tell the agent to invent AGI? Or time travel? Where does it drive to? Does it tell you that it can't? How does it know it can't just drive here? How you you know? At the end of the day these are not wish granting machines, so someone connected to reality is still going to have to make decisions. And that person is going to be the one held liable for whatever the AI does so would they want a self-driving codebase in the first place?

Sivart13 2mo ago
Well put. I've found that agents are great at adding features to areas of the code that I already understand, where I can phrase my goals coherently and keep them on the rails. Any attempt to describe software by "specification" leaves enough ambiguity that the agent will do deranged stuff like add a ton of code to satisfy error cases that can't happen. Or, like you imply, it will confidently create a bunch of nonsense to "solve" a problem in a way that is not really possible.

vb-8448 2mo ago

> Global memory I don't get why we need global memory for code? Aren't code comments (even if invented for humans) the ideal place where to put "memories"?

rrr_oh_man 2mo ago
LLMs also do not have anything like memory, by design. Everything bolted on that smells like memory or is being called memory is a crutch, at best. Not being pedantic, I just think this aspect is lost on a lot of people.
- Terr_ 2mo ago
  Also, when we humans are tricked into perceiving a remote mind, we automatically assume it has a memory like us, or at least like large animals. In other words, there's a story-document between a SherlockHolmesBot and UserPlaceholder that's growing like a crystal formation. Some software sees "Sherlock Homes Says" and then "performs" the dialogue at us, and we then assume Sherlock Holmes exists with a mind and memory, rather than being a facet of a text-story.
- alansaber 2mo ago
  Exactly, this is why we see such minimalistic memory systems being used. Information retrieval is bad, but improving information availability by pre-computing information retrieval is not the obvious fix people think it is.
drob 2mo ago
Sometimes, but I think there are a few problems with this: 1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it. 2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look. 3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.

icase 2mo ago

“how can we make ourselves even MORE replaceable?”

lioeters 2mo ago
Reminds me of that Richard Scarry children's book illustration of a fox busily sawing off a tree branch he's sitting on.

tonymet 2mo ago

I'm hoping the "blogging about doing" format will fade away, now that doing is cheaper than blogging. It used to be that the doing part was expensive, and risky. Now you can just do 5 things and throw away 4. So I'd rather see the finished product than talk about it.

AIiscoming 2mo ago

I'm playing around with ecosystem driven development (hear me out ;)): So i want claude to build a small game for me so of course i create one big file in which i write everything then i tell claude to analyse it and grill me (grill me skill) to clarify all smaller details. This alone might lead to really good small sfotware but I still have to push it sometimes. Now instead of doing this, i tell claude to build a small tool which generates a dashboard and memory and which can save specs and ask me choicses (do you prefer this color over the other, what do you think about problem A? What solution would you prefer? A, b, c or something else). This does a few things: 1. claude doesn't has to save/store everything in a context 2. claude can now talk to that tool to ask it stuff 3. claude can now use the tool as a todo list 4. claude now can more easily spin up more agents in parallel 5. i have a nice interface and i can solve issues while claude works on unblocked tasks 6. I actually can follow the progress a lot easier The only problem with this is: with the next update, you have to reinvestigate how claude was finetuned and adjust. A few month ago /goal was really good, now you need it a lot less because claude will do something for an hour without /goal And the spec file only started to work after November/Opus moment but it got so good, that i can pack A LOT of stuff in a half structured markdown file and let it code what I need. The progress is still too fast for the whole ecosystem

calldacopsidc 2mo ago

Sorry to get hung up on what is probably just flavor text, but the first sentence.. "Agents can oneshot games that are actually fun", uhhh such as? Would love to be proven wrong here, feels like I've just seen the same basic one-level kart racer in like 3 model announcements. Not even sure if they were one-shotted, I certainly would not describe them as "fun"

ludamn 2mo ago
Agreed, I read this and almost instinctively decided to close the page, agents can be used to build games for sure, but that "one shot" and "fun" in the same sentence is really misleading
- palmotea 2mo ago
  FYI, it looks like you've been banned for the last ~8-9 months, for no obvious reason I can see. I vouched for your comment to make it visible, but they appears to show up as [dead] by default.
  - ramon156 2mo ago
    i can see the comment now, so this seems undone?
  - ludamn 2mo ago
    [dead]
xyst 2mo ago
Probably fun defined in the ephemeral context. As-in it was "fun" to play in the context of the short form video attention span, social media brain rotted individual. Then it was thrown away and never played again. It's definitely not fun if you consider building an actual community around it and have a long term plan for it.
- fragmede 2mo ago

xyst 2mo ago

I currently see a shit ton of work involved with "scaling" (aka cleaning up) the AI generated slop that was output in the past 2-3 years.

kreneskyp 2mo ago

My answer to this is adding an engineering assurance module to my specification and verification toolkit. Every project has a different definition of success and quality. My goal is to provide a consistent way to define, analyze, and report quality metrics tailored to each project. https://github.com/agent-ix/engineering-assurance https://github.com/agent-ix/engineering-assurance https://github.com/agent-ix/quoin https://github.com/agent-ix/quoin

jakeinspace 2mo ago

Kind of feels like we're in a similar place with AI programming as we are with self driving vehicles. Good enough for the typical case, maybe even better than most humans, but bad when dealing with novel situations or situations which require a really cohesive model of the system (code, business requirements, physical world, whatever).

deleted 2mo ago

[deleted]

2001zhaozhao 2mo ago

In the end, we already have processes that work for humans and we know the types of testing, verification and review that makes a codebase grow healthily. We just need to adapt these designs the best we can to AIs. AI have a lot of advantages over humans like speed, controllability, good defaults (e.g. Claude will write helpful comments unprompted which is better than most engineers), and the ability to implement best practices like comprehensive testing with much lower cost than a human team. Using AI in my personal projects, I am able to write software at a much lower bug rate than manually, because now I can make an automated test suite whereas previously I had no time to do so. However, AI also has downsides like rigidity, limited context window, and lack of creativity by default which needs to be resolved with harnessing (e.g. different agents with separate, well-compressed contexts) or worked around with human review layers. I suspect the best core pattern for coding automation will be to create an engineering manager agent layer that can flexibly apply a set of engineering practices to the concrete problems at hand. It would handle all coding agent pings before it reaches a human, with prompting so that it reliably considers the big picture when making decisions. Very rarely, it would escalate an issue or plan review directly to humans when it is truly blocking, but most likely it would organize the information it obtains from the lower layer coding agents and surface them as reports for humans to review.

6thbit 2mo ago

The things in the 'what should be self-driving' section are the exact things i give to juniors so they can build their mental model of the codebase. Now, I agree it'd be fantastic to have that automated, but how do we then share that context with the humans that will drive the non-self-driving actions?

billybones 2mo ago

Super exciting, and absolute killers on the customer testimonial page.

rafaelvasco 2mo ago

Is our fate to become more and more product designers and less and less software engineers? I feel that way. But I'm now able to actually create and release things. Before, I used to obsess over code architecture, optimizing every single line. I could and can craft beautiful code. And then I always burned out and lost interest (gemini). Now I don't touch code at all and have been actually creating and releasing the ideas I've always had in mind. Then I realized that's what I always wanted. The code was always a means to an end. But that's me.

hollowturtle 2mo ago

> Agents can oneshot games that are actually fun. HAHAHAHAHAHAHAHAAH here we go again

bethekidyouwant 2mo ago
Joust isn’t fun?

thisisauserid 2mo ago

Sounds like the next shitty Vercel ad at re:Invent.

saiyamshah1496 2mo ago

[dead]

dinobones 2mo ago

> Agents can oneshot games that are actually fun. Stopped reading right there.

zlokki 2mo ago

> Put another way: Detail’s secret plan is to create a product that bootstraps its own work, and then use that work to help engineering teams know what investments to make and how well they’re working. Sorry, guys, I still don't understand what are you selling. Prompts? Design manual for building testable system? In which way agent-friendly dev env differs from human-friendly dev env?