5 ms·

I think there's been a lot of fear-mongering on this topic and "the inevitable LLM take over" is not as inevitable as it might seem, perhaps depending on your d

I think there's been a lot of fear-mongering on this topic and "the inevitable LLM take over" is not as inevitable as it might seem, perhaps depending on your definition of "take over." I have personally used LLMs in my job to write boilerplate code, write tests, make mass renaming changes that were previously tedious to do without a lot of grep/sed-fu, etc. For these types of tasks, LLMs are already miles ahead of what I was doing before (do it myself by hand, or have a junior engineer do it and get annoyed/burnt out). However, I have yet to see an LLM that can understand an already established large codebase and reliably make well-designed additions to it, in the way that an experienced team of engineers would. I suppose this ability could develop over time with large increases in memory/compute, but even state-of-the-art models today are so far away from being able to act like an actual senior engineer that I'm not worried. Don't get me wrong, LLMs are incredibly useful in my day-to-day work, but I think of them more as a leap forward in developer tooling, not as an eventual replacement for me.

m_ke 2y ago
Those models will be here within a year. Long context is practically a solved problem and there's a ton of work now on test time reasoning motivated by o1 showing that it's not that hard to RL a model into superhuman performance as long as the task is easy / cheap to validate (and there's works showing that if you can define the problem you can use an LLM to validate against your criteria).
- angoragoats 2y ago
  I intentionally glossed over a lot in my first comment, but I should clarify that I don't believe that increased context size or RL is sufficient to solve the problem I'm talking about. Also "as long as the task is easy / cheap to validate" is a problematic statement if we're talking about the replacement of senior software engineers, because problem definition and development of validation criteria are core to the duties of a senior software engineer. All of this is to say: I could be completely wrong, but I'll believe it when I see it. As I said elsewhere in the comments to another poster, if your points could be expressed in easily testable yes/no propositions with a timeframe attached, I'd likely be willing to bet real money against them.
  - m_ke 2y ago
    Sorry I wasn't clear enough, the cheap to validate part is only needed to train a large base model that can handle writing individual functions / fix bugs. Planning a whole project, breaking it down into steps and executing each one is not something that current LLMs struggle at. Here's a recipe for a human level LLM software engineer: 1. Pretrain an LLM on as much code and text as you can (done already) 2. Fine tune it on synthetic code specific tasks like: (a) given a function, hide the body, make the model implement it and validate that it's functionally equivalent to the target function (output matching), can also have an objective to optimize the runtime of the implementation (b) introduce bugs in existing code and make the LLM fix it, (c) make LLM make up problems, write tests / spec for it, then have it attempt to implement it many times until it comes up with a method that passes the tests, (d-z) a lot of other similar tasks that use linters, parsers, AST modifications, compilers, unit tests, specs validated by LLMs, profilers to check that the produced code is valid 3. Distill this success / failure criteria validator to a value function that can predict probability of success at each token to give immediate reward instead of requiring full roll out, then optimize the LLM on that. 4. At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations. See papers like: https://arxiv.org/abs/2409.15254 https://arxiv.org/abs/2409.15254 or slides from NeurIPS that I mentioned here https://news.ycombinator.com/item?id=42431382 https://news.ycombinator.com/item?id=42431382
    - angoragoats 2y ago
      > At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations. If you're saying that it takes one software engineer one hour to produce comprehensive criteria that would allow this whole pipeline to work for a non-trivial software engineering task, this is where we violently disagree. For this reason, I don't believe I'll be convinced by any additional citations or research, only by an actual demonstration of this working end-to-end with minimal human involvement (or at least, meaningfully less human involvement than it would take to just have engineers do the work). edit: Put another way, what you describe here looks to me to be throwing a huge number of "virtual" low-skilled junior developers at the task and optimizing until you can be confident that one of them will produce a good-enough result. My contention is that this is not a valid methodology for reproducing/replacing the work of senior software engineers.

m_ke 2y ago

Those models will be here within a year. Long context is practically a solved problem and there's a ton of work now on test time reasoning motivated by o1 showing that it's not that hard to RL a model into superhuman performance as long as the task is easy / cheap to validate (and there's works showing that if you can define the problem you can use an LLM to validate against your criteria).

angoragoats 2y ago
I intentionally glossed over a lot in my first comment, but I should clarify that I don't believe that increased context size or RL is sufficient to solve the problem I'm talking about. Also "as long as the task is easy / cheap to validate" is a problematic statement if we're talking about the replacement of senior software engineers, because problem definition and development of validation criteria are core to the duties of a senior software engineer. All of this is to say: I could be completely wrong, but I'll believe it when I see it. As I said elsewhere in the comments to another poster, if your points could be expressed in easily testable yes/no propositions with a timeframe attached, I'd likely be willing to bet real money against them.
- m_ke 2y ago
  Sorry I wasn't clear enough, the cheap to validate part is only needed to train a large base model that can handle writing individual functions / fix bugs. Planning a whole project, breaking it down into steps and executing each one is not something that current LLMs struggle at. Here's a recipe for a human level LLM software engineer: 1. Pretrain an LLM on as much code and text as you can (done already) 2. Fine tune it on synthetic code specific tasks like: (a) given a function, hide the body, make the model implement it and validate that it's functionally equivalent to the target function (output matching), can also have an objective to optimize the runtime of the implementation (b) introduce bugs in existing code and make the LLM fix it, (c) make LLM make up problems, write tests / spec for it, then have it attempt to implement it many times until it comes up with a method that passes the tests, (d-z) a lot of other similar tasks that use linters, parsers, AST modifications, compilers, unit tests, specs validated by LLMs, profilers to check that the produced code is valid 3. Distill this success / failure criteria validator to a value function that can predict probability of success at each token to give immediate reward instead of requiring full roll out, then optimize the LLM on that. 4. At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations. See papers like: https://arxiv.org/abs/2409.15254 https://arxiv.org/abs/2409.15254 or slides from NeurIPS that I mentioned here https://news.ycombinator.com/item?id=42431382 https://news.ycombinator.com/item?id=42431382
  - angoragoats 2y ago
    > At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations. If you're saying that it takes one software engineer one hour to produce comprehensive criteria that would allow this whole pipeline to work for a non-trivial software engineering task, this is where we violently disagree. For this reason, I don't believe I'll be convinced by any additional citations or research, only by an actual demonstration of this working end-to-end with minimal human involvement (or at least, meaningfully less human involvement than it would take to just have engineers do the work). edit: Put another way, what you describe here looks to me to be throwing a huge number of "virtual" low-skilled junior developers at the task and optimizing until you can be confident that one of them will produce a good-enough result. My contention is that this is not a valid methodology for reproducing/replacing the work of senior software engineers.