4 ms·

> it can optimize its own code This is an overstatement. There are still humans in the loop to do the prompt, apply the patch, verify, write tests, and commit.

> it can optimize its own code This is an overstatement. There are still humans in the loop to do the prompt, apply the patch, verify, write tests, and commit. We're not even at intern-level autonomy here.

simonw 2y ago
Plugging DeepSeek R1 into a harness that can apply the changes, compile them, run the tests and loop to solve any bugs isn't hard. People are already plugging it into existing systems like Aider that can run those kinds of operations.
- mohsen1 2y ago
  Yes! I've done something like this here in my repo. This was nice while lasted (Deepseek is practically useless through the API since yesterday) https://github.com/bodo-run/yek/blob/main/.github/workflows/ailoop.yaml https://github.com/bodo-run/yek/blob/main/.github/workflows/... https://github.com/bodo-run/yek/blob/main/scripts/ai-loop.sh https://github.com/bodo-run/yek/blob/main/scripts/ai-loop.sh Using askds https://github.com/bodo-run/askds https://github.com/bodo-run/askds
  - lgats 2y ago
    added context, deepseek is having ddos issues https://status.deepseek.com/ https://status.deepseek.com/
    - skyyler 2y ago
      If you can't beat em on merit, pay criminals to take em offline. Good business.
  - mrtesthah 2y ago
    You can run it through Openrouter/Fireworks hosted in the US.
    - lizardking 2y ago
      For me it was so slow I found it to be unusable.
- casenmgreen 2y ago
  How do you know you've got a bug, to tell the AI to fix it?
  - simonw 2y ago
    You get really good at manual QA.
gejose 2y ago
How long do you see the humans in the loop being necessary?
- tantalor 2y ago
  Where companies depend on code for business critical applications? Forever. When your AI-managed codebase breaks, who are you going to ask to fix it? The AI?
  - minkzilla 2y ago
    Yes.
  - WXLCKNO 2y ago
    Absolutely the AI. At that point in the future I'm presuming that if something breaks it's because an external API or whatever dependency broke, not because the AI code has an inherent bug. But if it does it could still fix it. And you won't have to tell it anything, alerts will be sent if a test fails and it will fix it directly.
deleted 2y ago
[deleted]
cchance 2y ago
I mean currently yes, but writing a test/patch/benchmark loop, maybe with a seperate AI that generates the requests to the coder agent loop, should be doable to have the AI continually attempt to improve itself, its just no ones built the loop yet to my knowledge
tokioyoyo 2y ago
I'm very sorry, but the goalposts are moving so far ahead now, that's it's very hard to keep track of. 6 months ago the same comments were saying "AI generated code is complete garbage is useless, and I have to rewrite everything all the time anyways". Now we're onto "need to prompt, apply patch, verify" and etc. Come on guys, time to look at it a bit objectively, and decide where we're going with it.
- talldayo 2y ago
  Quite the contrary, really. We've been seeing "success stories" with AI translating function calls for years now, it just doesn't get any attention or make any headlines because it's so simple. SIMD optimization is pretty much the lowest-hanging fruit of modern computation; a middle schooler could write working SIMD code if they understood the problem. There's certainly a bit of irony in the PR, but the code itself is not complex enough to warrant any further hysteria. If you've written SIMD by hand you're probably well familiar with the fact that it's more drudgery than thought work.
  - tokioyoyo 2y ago
    It's been probably about 15 years since I've touched that, so I genuinely have no recollection of SIMD coding. But literally, that's the purpose of higher level automation? Like I don't know/remember it, I ask it to do stuff, it does, and the output is good enough. That's how a good chunk of companies operate - you get general idea of what to do, you write the code, then eventually it makes it to production. As we patch the holes in the AI-code delivery pipeline, those human-involved issues will be resolved as well. Slowly, painfully, but it's just a matter of time at this point?
- rybosworld 2y ago
  Couldn't agree more. Every time these systems get better, there are dozens of comments to the effect of "ya but...[insert something ai isn't great at yet]". It's a bit maddening to see this happening on a forum full of tech-literate folks. Ultimately, I think to stay relevant in software development, we are going to have accept that our role in the process could evolve to humans essentially never writing code. Take that one step further and humans may not even be reviewing code. I am not sure if accepting that is enough to guarantee job security. But I am fairly sure that those who do accept this eventuality will be more relevant for longer than those who prefer to hide behind their "I'm irreplaceable because I'm human" attitude. If your first instinct is to pick these systems apart and look for things that they aren't doing perfectly, then you aren't seeing the big picture.

simonw 2y ago

Plugging DeepSeek R1 into a harness that can apply the changes, compile them, run the tests and loop to solve any bugs isn't hard. People are already plugging it into existing systems like Aider that can run those kinds of operations.

mohsen1 2y ago
Yes! I've done something like this here in my repo. This was nice while lasted (Deepseek is practically useless through the API since yesterday) https://github.com/bodo-run/yek/blob/main/.github/workflows/ailoop.yaml https://github.com/bodo-run/yek/blob/main/.github/workflows/... https://github.com/bodo-run/yek/blob/main/scripts/ai-loop.sh https://github.com/bodo-run/yek/blob/main/scripts/ai-loop.sh Using askds https://github.com/bodo-run/askds https://github.com/bodo-run/askds
- lgats 2y ago
  added context, deepseek is having ddos issues https://status.deepseek.com/ https://status.deepseek.com/
  - skyyler 2y ago
    If you can't beat em on merit, pay criminals to take em offline. Good business.
- mrtesthah 2y ago
  You can run it through Openrouter/Fireworks hosted in the US.
  - lizardking 2y ago
    For me it was so slow I found it to be unusable.
casenmgreen 2y ago
How do you know you've got a bug, to tell the AI to fix it?
- simonw 2y ago
  You get really good at manual QA.

gejose 2y ago

How long do you see the humans in the loop being necessary?

tantalor 2y ago
Where companies depend on code for business critical applications? Forever. When your AI-managed codebase breaks, who are you going to ask to fix it? The AI?
- minkzilla 2y ago
  Yes.
- WXLCKNO 2y ago
  Absolutely the AI. At that point in the future I'm presuming that if something breaks it's because an external API or whatever dependency broke, not because the AI code has an inherent bug. But if it does it could still fix it. And you won't have to tell it anything, alerts will be sent if a test fails and it will fix it directly.

deleted 2y ago

[deleted]

cchance 2y ago

I mean currently yes, but writing a test/patch/benchmark loop, maybe with a seperate AI that generates the requests to the coder agent loop, should be doable to have the AI continually attempt to improve itself, its just no ones built the loop yet to my knowledge

tokioyoyo 2y ago

I'm very sorry, but the goalposts are moving so far ahead now, that's it's very hard to keep track of. 6 months ago the same comments were saying "AI generated code is complete garbage is useless, and I have to rewrite everything all the time anyways". Now we're onto "need to prompt, apply patch, verify" and etc. Come on guys, time to look at it a bit objectively, and decide where we're going with it.

talldayo 2y ago
Quite the contrary, really. We've been seeing "success stories" with AI translating function calls for years now, it just doesn't get any attention or make any headlines because it's so simple. SIMD optimization is pretty much the lowest-hanging fruit of modern computation; a middle schooler could write working SIMD code if they understood the problem. There's certainly a bit of irony in the PR, but the code itself is not complex enough to warrant any further hysteria. If you've written SIMD by hand you're probably well familiar with the fact that it's more drudgery than thought work.
- tokioyoyo 2y ago
  It's been probably about 15 years since I've touched that, so I genuinely have no recollection of SIMD coding. But literally, that's the purpose of higher level automation? Like I don't know/remember it, I ask it to do stuff, it does, and the output is good enough. That's how a good chunk of companies operate - you get general idea of what to do, you write the code, then eventually it makes it to production. As we patch the holes in the AI-code delivery pipeline, those human-involved issues will be resolved as well. Slowly, painfully, but it's just a matter of time at this point?
rybosworld 2y ago
Couldn't agree more. Every time these systems get better, there are dozens of comments to the effect of "ya but...[insert something ai isn't great at yet]". It's a bit maddening to see this happening on a forum full of tech-literate folks. Ultimately, I think to stay relevant in software development, we are going to have accept that our role in the process could evolve to humans essentially never writing code. Take that one step further and humans may not even be reviewing code. I am not sure if accepting that is enough to guarantee job security. But I am fairly sure that those who do accept this eventuality will be more relevant for longer than those who prefer to hide behind their "I'm irreplaceable because I'm human" attitude. If your first instinct is to pick these systems apart and look for things that they aren't doing perfectly, then you aren't seeing the big picture.