4 ms·

Yes but those are all semantic tasks which can be done and understood in time by the next iteration of models training at that meta-level of architectural analy

Yes but those are all semantic tasks which can be done and understood in time by the next iteration of models training at that meta-level of architectural analysis. AI deeply understands what Bun, Zig and Rust are, how they work, can conceive of the before and after architecture (probably the hard step here), can conceive of the goal of the rewrite, and can verify (using Lean and machine-checked proofs of the before and after expected states) the final result. They just needed us to ask. Build a sufficiently general research program that can find and iterate on ideas, and it will do the asking itself.

znnajdla 1mo ago
Sure, maybe step 2 can eventually be replaced by AI. But step 1? Why would the AI even conceive of the idea of doing a rewrite in the first place? And rewriting is just an example. There are tons of architectural decisions that need to be made every day in the building of advanced software. This is no "correct" way to do it that the AI can know in advance, it's a technical decision to be made by an engineer.
- satvikpendem 1mo ago
  Maybe it sees a bunch of segfaults in the Zig codebase and decides a memory safe language is better, just as the human did in Bun's rewrite. I'm not sure why you think it wouldn't have been able to conceive a rewrite.
  - znnajdla 1mo ago
    Such an AI would be unusable in production because there is no bound on the work that can be created and external side effects that could happen. All projects have real world constraints and side effects, budget, customer requirements, etc. Imagine a non-technical person prompting the AI "make the app faster and fix all the bugs" and the AI autonomously decides to rewrite the whole production app and all its dependencies in Rust that is live and serving thousands of users. Doing a full rewrite would take days to execute, and have all sorts of side effects on the actual users during deployment (even if it was done 100% correctly, work has to be paused, affairs need to be coordinated with real customers, etc.). Even if the AI has that capability (and I do believe it is possible, even with today's models), that's not what a business owner wants. You don't want to use an AI which, if you gave it a 3 word prompt, it could suddenly decide on its own to rewrite your whole entire business. An AI that had that much autonomy could just decide on its own to pivot your whole startup and sell something else. A business owner wouldn't even want to use such an AI which could have such large unbounded side effects.
    - dogcomplex 1mo ago
      You are essentially describing an over-eager developer having to run decisions up the chain of command to senior devs / product owners / company execs. I assure you, every one of those roles is being trained on to balance their own wide breadth of concerns, and is unlikely to advise something like a live rewrite of a production app or an entire product pivot - unless it actually made sense from an economics/management perspective. Every step bounded by its role and responsibilities. Turtles all the way up, and all the way down. AI getting this good at coding means it's a hop skip and a jump away from being this good at every white collar role.
      - znnajdla 1mo ago
        Yes, but the entire point is, you can't have a reasonable chain of command or any kind of sensible decision process without human judgement that has engineering expertise, so no, engineers are not going to be entirely replaced. The reason human judgement is necessary is not because it's "better". It's because the entire reason the business exists is to execute the human's judgement. There is no "right or wrong" way to build a startup if different paths are all profitable, but the human decides what they want to do. Could Bun have survived and done well even if they didn't rewrite the codebase in Rust? Yes, it would've been fine. But the human decided they wanted Rust, that's a different path they chose with different tradeoffs. The AI has no skin in the game , no ownership, it doesn't belong to the AI. The AI only exists to serve the human. And if the human wants to make good decisions, inevitably they need engineering expertise (or an engineer) to weigh the decision.
kodoman 1mo ago
I feel like leaving it to decide everything would not produce a good end product. It feels like just having it decide how GC should work, how the function call stack should work or any of these rather simple but actually requiring lots of decision and thinking how things fit together (and pick a good solution out of many seemingly good solutions that can come back to bite you), couple this with the tendency for AI agents to tend towards adding new code and building over features I don't think you end up with a good solution. On formal verification having done it only in an academic sense and looked over at projects like seL4 and quite interested in that project. It feels like actually proving useful properties of programs for real programs even ones with well defined domains and easier to model such as interpreters or compilers it seems that it will just prove theorems about properties that hardly matter or don't even matter at all. See how bad it still tends to be when trying to get it to write tests. I would be interested to know if their has been an agent that has actually utilized formal methods such as Lean or Coq or Isabella to prove properties of programs in an automated way as you suggest, I have only seen it proving mathematics and or searching for counter examples, not writing Curry-Howard style proofs.
- znnajdla 1mo ago
  > I don't think you end up with a good solution More precisely, a "perfect solution" doesn't exist. It's all tradeoffs given your goals. Someone needs to make the decision: 1. which tradeoffs are worth given your goals 2. which goals are worth defining or redefining. And do that effectively, you need to understand the problem, which goes back to engineering.
- dogcomplex 1mo ago
  Shoutout to Toph https://x.com/VictorTaelin https://x.com/VictorTaelin who's been building a programming language out of formal method Lean proof checkers, which probably would indeed have the scope you're looking for. I do think the current state of the AI just using rigorous tests and rendering inspection loops is still far more than most programmers did for the majority of apps, but yes I think rigorous formal verification will come too. Worst case it's gonna be something like a MechanicalTurk pipeline having us humans verify narrowed scopes the AI can't confidently inspect (yet. while also training on those results for next iteration) As for the decision stack of what makes a good end product - turtles all the way up/down. The ProductOwnerAI role decides those things, and it will likely do so with the same deep skill that programming AIs are currently hitting our profession with. Optimizing enormous breadths of concerns and simulating results is AI's main specialty.

znnajdla 1mo ago

Sure, maybe step 2 can eventually be replaced by AI. But step 1? Why would the AI even conceive of the idea of doing a rewrite in the first place? And rewriting is just an example. There are tons of architectural decisions that need to be made every day in the building of advanced software. This is no "correct" way to do it that the AI can know in advance, it's a technical decision to be made by an engineer.

satvikpendem 1mo ago
Maybe it sees a bunch of segfaults in the Zig codebase and decides a memory safe language is better, just as the human did in Bun's rewrite. I'm not sure why you think it wouldn't have been able to conceive a rewrite.
- znnajdla 1mo ago
  Such an AI would be unusable in production because there is no bound on the work that can be created and external side effects that could happen. All projects have real world constraints and side effects, budget, customer requirements, etc. Imagine a non-technical person prompting the AI "make the app faster and fix all the bugs" and the AI autonomously decides to rewrite the whole production app and all its dependencies in Rust that is live and serving thousands of users. Doing a full rewrite would take days to execute, and have all sorts of side effects on the actual users during deployment (even if it was done 100% correctly, work has to be paused, affairs need to be coordinated with real customers, etc.). Even if the AI has that capability (and I do believe it is possible, even with today's models), that's not what a business owner wants. You don't want to use an AI which, if you gave it a 3 word prompt, it could suddenly decide on its own to rewrite your whole entire business. An AI that had that much autonomy could just decide on its own to pivot your whole startup and sell something else. A business owner wouldn't even want to use such an AI which could have such large unbounded side effects.
  - dogcomplex 1mo ago
    You are essentially describing an over-eager developer having to run decisions up the chain of command to senior devs / product owners / company execs. I assure you, every one of those roles is being trained on to balance their own wide breadth of concerns, and is unlikely to advise something like a live rewrite of a production app or an entire product pivot - unless it actually made sense from an economics/management perspective. Every step bounded by its role and responsibilities. Turtles all the way up, and all the way down. AI getting this good at coding means it's a hop skip and a jump away from being this good at every white collar role.
    - znnajdla 1mo ago
      Yes, but the entire point is, you can't have a reasonable chain of command or any kind of sensible decision process without human judgement that has engineering expertise, so no, engineers are not going to be entirely replaced. The reason human judgement is necessary is not because it's "better". It's because the entire reason the business exists is to execute the human's judgement. There is no "right or wrong" way to build a startup if different paths are all profitable, but the human decides what they want to do. Could Bun have survived and done well even if they didn't rewrite the codebase in Rust? Yes, it would've been fine. But the human decided they wanted Rust, that's a different path they chose with different tradeoffs. The AI has no skin in the game , no ownership, it doesn't belong to the AI. The AI only exists to serve the human. And if the human wants to make good decisions, inevitably they need engineering expertise (or an engineer) to weigh the decision.

kodoman 1mo ago

I feel like leaving it to decide everything would not produce a good end product. It feels like just having it decide how GC should work, how the function call stack should work or any of these rather simple but actually requiring lots of decision and thinking how things fit together (and pick a good solution out of many seemingly good solutions that can come back to bite you), couple this with the tendency for AI agents to tend towards adding new code and building over features I don't think you end up with a good solution. On formal verification having done it only in an academic sense and looked over at projects like seL4 and quite interested in that project. It feels like actually proving useful properties of programs for real programs even ones with well defined domains and easier to model such as interpreters or compilers it seems that it will just prove theorems about properties that hardly matter or don't even matter at all. See how bad it still tends to be when trying to get it to write tests. I would be interested to know if their has been an agent that has actually utilized formal methods such as Lean or Coq or Isabella to prove properties of programs in an automated way as you suggest, I have only seen it proving mathematics and or searching for counter examples, not writing Curry-Howard style proofs.

znnajdla 1mo ago
> I don't think you end up with a good solution More precisely, a "perfect solution" doesn't exist. It's all tradeoffs given your goals. Someone needs to make the decision: 1. which tradeoffs are worth given your goals 2. which goals are worth defining or redefining. And do that effectively, you need to understand the problem, which goes back to engineering.
dogcomplex 1mo ago
Shoutout to Toph https://x.com/VictorTaelin https://x.com/VictorTaelin who's been building a programming language out of formal method Lean proof checkers, which probably would indeed have the scope you're looking for. I do think the current state of the AI just using rigorous tests and rendering inspection loops is still far more than most programmers did for the majority of apps, but yes I think rigorous formal verification will come too. Worst case it's gonna be something like a MechanicalTurk pipeline having us humans verify narrowed scopes the AI can't confidently inspect (yet. while also training on those results for next iteration) As for the decision stack of what makes a good end product - turtles all the way up/down. The ProductOwnerAI role decides those things, and it will likely do so with the same deep skill that programming AIs are currently hitting our profession with. Optimizing enormous breadths of concerns and simulating results is AI's main specialty.