3 ms·

I think the biggest difference here is that your code generator is probably deterministic and you likely are able to debug the results it produces rather than t

I think the biggest difference here is that your code generator is probably deterministic and you likely are able to debug the results it produces rather than treating it like a black box.

buu700 1y ago
Overloading of the term "generate" is probably creating some confused ideas here. An LLM/agent is a lot more similar to a human in terms of its transformation of input into output than it is to a compiler or code generator. I've been working on a recent project with heavy use of AI (probably around 100 hours of long-running autonomous AI sprints over the last few weeks), and if you tried to re-run all of my prompts in order, even using the exact same models with the exact same tooling, it would almost certainly fall apart pretty quickly. After the first few, a huge portion of the remaining prompts would be referencing code that wouldn't exist and/or responding to things that wouldn't have been said in the AI's responses. Meta-prompting (prompting agents to prepare prompts for other agents) would be an interesting challenge to properly encode. And how would human code changes be represented, as patches against code that also wouldn't exist? The whole idea also ignores that AI being fast and cheap compared to human developers doesn't make it infinitely fast or free, or put it in the same league of quickness and cheapness as a compiler. Even if this were conceptually feasible, all it would really accomplish is making it so that any new release of a major software project takes weeks (or more) of build time and thousands of dollars (or more) burned on compute. It's an interesting thought experiment, but the way I would put it into practice would be to use tooling that includes all relevant prompts / chat logs in each commit message. Then maybe in the future an agent with a more advanced model could go through each commit in the history one by one, take notes on how each change could have been better implemented based on the associated commit message and any source prompts contained therein, use those notes to inform a consolidated set of recommended changes to the current code, and then actually apply the recommendations in a series of pull requests.
tptacek 1y ago
People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking. There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.
- buu700 1y ago
  I assumed they were describing AI itself as a black box (contrasting it with deterministic code generation), not the output of AI.
  - tptacek 1y ago
    Right, I get that, and an LLM call by itself clearly is a black box. I just don't get why that's supposed to matter. It produces an artifact I can (and must) verify myself.
    - buu700 1y ago
      Because if the LLM is a black box and its output must ultimately be verified by humans, then you can't treat conversion of prompts into code as a simple build step as though an AI agent were just some sort of compiler. You still need to persist the actual code in source control. (I assume that isn't what you're actually arguing against, in which case at least one of us must have misread something from the parent chain.)
      - tptacek 1y ago
        Right, you definitely can't do that. People do talk as if the question was whether we could stick LLM calls into Makefiles. Nobody would ever do that, at least not with the technology we have at hand.
        layer8 1y ago
        But that’s exactly what the part of the article quoted in the root comment (https://news.ycombinator.com/item?id=44206758 https://news.ycombinator.com/item?id=44206758) is about, and in reference to future AI technology. That’s was this subthread was discussing.
        tptacek 1y ago
        Yep, you're 100% right. Sorry!

I think the biggest difference here is that your code generator is probably deterministic and you likely are able to debug the results it produces rather than treating it like a black box.

buu700 1y ago

Overloading of the term "generate" is probably creating some confused ideas here. An LLM/agent is a lot more similar to a human in terms of its transformation of input into output than it is to a compiler or code generator. I've been working on a recent project with heavy use of AI (probably around 100 hours of long-running autonomous AI sprints over the last few weeks), and if you tried to re-run all of my prompts in order, even using the exact same models with the exact same tooling, it would almost certainly fall apart pretty quickly. After the first few, a huge portion of the remaining prompts would be referencing code that wouldn't exist and/or responding to things that wouldn't have been said in the AI's responses. Meta-prompting (prompting agents to prepare prompts for other agents) would be an interesting challenge to properly encode. And how would human code changes be represented, as patches against code that also wouldn't exist? The whole idea also ignores that AI being fast and cheap compared to human developers doesn't make it infinitely fast or free, or put it in the same league of quickness and cheapness as a compiler. Even if this were conceptually feasible, all it would really accomplish is making it so that any new release of a major software project takes weeks (or more) of build time and thousands of dollars (or more) burned on compute. It's an interesting thought experiment, but the way I would put it into practice would be to use tooling that includes all relevant prompts / chat logs in each commit message. Then maybe in the future an agent with a more advanced model could go through each commit in the history one by one, take notes on how each change could have been better implemented based on the associated commit message and any source prompts contained therein, use those notes to inform a consolidated set of recommended changes to the current code, and then actually apply the recommendations in a series of pull requests.

tptacek 1y ago

People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking. There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.

buu700 1y ago
I assumed they were describing AI itself as a black box (contrasting it with deterministic code generation), not the output of AI.
- tptacek 1y ago
  Right, I get that, and an LLM call by itself clearly is a black box. I just don't get why that's supposed to matter. It produces an artifact I can (and must) verify myself.
  - buu700 1y ago
    Because if the LLM is a black box and its output must ultimately be verified by humans, then you can't treat conversion of prompts into code as a simple build step as though an AI agent were just some sort of compiler. You still need to persist the actual code in source control. (I assume that isn't what you're actually arguing against, in which case at least one of us must have misread something from the parent chain.)
    - tptacek 1y ago
      Right, you definitely can't do that. People do talk as if the question was whether we could stick LLM calls into Makefiles. Nobody would ever do that, at least not with the technology we have at hand.
      - layer8 1y ago
        But that’s exactly what the part of the article quoted in the root comment (https://news.ycombinator.com/item?id=44206758 https://news.ycombinator.com/item?id=44206758) is about, and in reference to future AI technology. That’s was this subthread was discussing.
        tptacek 1y ago
        Yep, you're 100% right. Sorry!