5 ms·

Have you tried it? In my experience they just go off on a hallucination loop, or blow up the code base with terrible re-implementations. Similarly Claude 3.5 w

Have you tried it? In my experience they just go off on a hallucination loop, or blow up the code base with terrible re-implementations. Similarly Claude 3.5 was stuck on TensorRT 8, and not even pointing it at the documentation for the updated 10 APIs for RAG could ever get it to correctly use the new APIs (not that they were very complex; bind tensors, execute, retrieve results). The whole concept of the self-reinforcing Agent loop is more of a fantasy. I think someone else likened it to a lawnmower that will run rampage over your flower bed at the first hiccup.

mountainriver 1y ago
Have you tried it? More than once? I’m getting massive productivity gains with Cursor and Gemini 2.5 or Claude 3.7. One-shotting whole features into my rust codebase.
- stefan_ 1y ago
  I use it all the time, multiple times daily. But the discussion is not being very honest, particularly for all the things that are being bolted on (agent mode, MCP). Like just upstream people dunk on others for pointing out that maybe giving the model an API call to read webpages isn't quite turning LLM into search engines. Just like letting it run shell commands has not made it into a full blown agent engineer. I tried it again just now with Claude 3.7 in Cursors Agent/Compose (they change this stuff weekly). Write a simple C++ TensorRT app that loads an engine and runs inference 100 times for a benchmark, use this file to source a toolchain. It generated code with the old API & a CMake file and (warning light turns on) a build script. The compile fails because of the old API, but this time it managed to fix it to use the new API. But now the linking fails, because it overwrote the TRT/CUDA directories in the CMakeLists with some home cooked logic (there was nothing to do, the toolchain script sets up the environment fully and just find_package would work). And this is where we go off the rails; it messes with the build script and CMakeLists more, but still it can not link. It thinks hey it looks like we are cross-compiling and creates a second build script "cross-compile.sh" that tries to use the compiler directly, but of course that misses things that the find_package in CMake would setup and so fails with include errors. It pretends its a 1970 ./configure script and creates source files "test_nvinfer.cpp" and "test_cudart.cpp" that are supposed to test for the presence of those libraries, then tries to compile them directly; again its missing directories and obviously fails. Next we create a mashup build script "cross-compile-direct.sh". Not sure anymore what this one tried to achieve, didn't work. Finally, and this is my favorite agent action yet, it decides fuck it, if the library won't link, why don't we just mock out all the actual TensorRT/CUDA functionality and print fake benchmark numbers to demonstrate LLMs can average a number in C++. So it writes, builds ands runs a "benchmark_mock.cpp" that subs out all the useful functionality for random data from std::mt19937. This naturally works, so the agent declares success and happily updates the README.md with all the crap it added and stops. This is what running the lawnmower over the flower bed means; you have 5 more useless source files and a bunch more shell scripts and a bunch of crap in a README that were all generated to try and fail to fix a problem it could not figure out, and this loop can keep going and generate more nonsense ad infinitum. (Why could it not figure out the linking error? We come back to the shitty bolted on integrations; it doesn't actually query the environment, search for files or look at what link directories are being used, as one would investigating a linking error. It could of course, but the balance in these integrations is 99% LLM and 1% tool use, and even context from the tool use often doesn't help)
  - tptacek 1y ago
    It's really weird for me to see people talking about using LLMs in coding situations in a frame where "agents" (we're not even at MCP yet!) are somehow an extra. People discussing the applicability of LLMs to programming, and drawing conclusions (even if only for themselves) about how well it works, should be experienced with a coding agent.
  - mountainriver 1y ago
    LLMs aren't good at deep ML problems yet. We know this, its what MLE-bench is for. That doesn't mean they aren't good at other coding problems
tptacek 1y ago
Yes, they're part of my daily toolset. And yes, they can spin out. I just hit the "reject" button when they do, and revise my prompt. Or, sometimes, I just take over and fill in some of the structure of the problem I'm trying to solve myself. I don't know about "self-reinforcing". I'm just saying: coding agents compile and lint the code they're running, and when they hallucinate interfaces, they notice. The same way any developer who has ever used ChatGPT knows that you can paste most errors into the web page and it will often (maybe even usually) come up with an apposite fix. I don't understand how anybody expects to convince LLM users this doesn't work; it obviously does work.
- steveklabnik 1y ago
  > I don't understand how anybody expects to convince LLM users this doesn't work; it obviously does work. This is really one of the hugest divides I've seen in the discourse about this: anti-LLM people saying very obviously untrue things, which is uh, kind of hilarious in a meta way. https://bsky.app/profile/caseynewton.bsky.social/post/3lo4tdybmgk2v https://bsky.app/profile/caseynewton.bsky.social/post/3lo4td... is an instance of this from a few days ago. I am still trying to sort out why experiences are so divergent. I've had much more positive LLM experiences while coding than many other people seem to, even as someone who's deeply skeptical of what's being promised about them. I don't know how to reconcile the two.
  - zoogeny 1y ago
    I think it is pretty simple: people tried it a few times a few months ago in a limited setting, formed an opinion based on those limited experiences and cannot imagine a world where they are wrong. That might sound snarky, but it probably works out for people in 99% of cases. AI and LLMs are advancing at a pace that is so different from any other technology that people aren't yet trained to re-evaluate their assumptions at the high rate necessary to form accurate new opinions. There are too many tools coming (and going, to be fair). HN (and certain parts of other social media) is a bubble of early adopters. We're on the front lines seeing the war in realtime and shaking our heads at what's being reported in the papers back home.
dboreham 1y ago
Someone gave me the tip to add "all source files should build without error", which you'd think would be implicit, but it seems not.
- tptacek 1y ago
  There's definitely a skill to using them well (I am not yet expert); my only frustration is with people who (like me) haven't refined the skill but have also concluded that there's no benefit to the tool. No, really, in this case, you're mostly just not holding it right. The tools will get better, but what I see happening with people who are good at using them (and from my own code, even in my degraded LLM usage), we have an existence proof of the value of the tools.
giovannibonetti 1y ago
> I think someone else likened it to a lawnmower that will run rampage over your flower bed at the first hiccup This reminds me of a scene from the recent animation movie "Wallace and Gromit: Vengeance Most Fowl" where Wallace actually uses a robot (Norbot) to do gardening tasks, and rampages over Gromit's flower bed. https://youtu.be/_Ha3fyDIXnc https://youtu.be/_Ha3fyDIXnc
zoogeny 1y ago
I mean, I have. I use them every day. You often see them literally saying "Oh there is a linter error, let me go fix it" and then a new code generation pass happens. In the worst case, it does exactly what you are saying, gets stuck in a loop. It eventually gets to the point where it says "let me try just once more" and then gives up. And when that happens I review the code and if it is bad then I "git revert". And if it is 90% of the way there I fix it up and move on. The question shouldn't be "are they infallible tools of perfection". It should be "do I get value equal to or greater than the time/money I spend". And if you use git appropriately you lose at most five minutes on a agent looping. And that happens a couple of times a week. And be honest with yourself, is getting stuck in a loop fighting a compiler, type-checker or lint something you have ever experienced in your pre-LLM days?

mountainriver 1y ago

Have you tried it? More than once? I’m getting massive productivity gains with Cursor and Gemini 2.5 or Claude 3.7. One-shotting whole features into my rust codebase.

stefan_ 1y ago
I use it all the time, multiple times daily. But the discussion is not being very honest, particularly for all the things that are being bolted on (agent mode, MCP). Like just upstream people dunk on others for pointing out that maybe giving the model an API call to read webpages isn't quite turning LLM into search engines. Just like letting it run shell commands has not made it into a full blown agent engineer. I tried it again just now with Claude 3.7 in Cursors Agent/Compose (they change this stuff weekly). Write a simple C++ TensorRT app that loads an engine and runs inference 100 times for a benchmark, use this file to source a toolchain. It generated code with the old API & a CMake file and (warning light turns on) a build script. The compile fails because of the old API, but this time it managed to fix it to use the new API. But now the linking fails, because it overwrote the TRT/CUDA directories in the CMakeLists with some home cooked logic (there was nothing to do, the toolchain script sets up the environment fully and just find_package would work). And this is where we go off the rails; it messes with the build script and CMakeLists more, but still it can not link. It thinks hey it looks like we are cross-compiling and creates a second build script "cross-compile.sh" that tries to use the compiler directly, but of course that misses things that the find_package in CMake would setup and so fails with include errors. It pretends its a 1970 ./configure script and creates source files "test_nvinfer.cpp" and "test_cudart.cpp" that are supposed to test for the presence of those libraries, then tries to compile them directly; again its missing directories and obviously fails. Next we create a mashup build script "cross-compile-direct.sh". Not sure anymore what this one tried to achieve, didn't work. Finally, and this is my favorite agent action yet, it decides fuck it, if the library won't link, why don't we just mock out all the actual TensorRT/CUDA functionality and print fake benchmark numbers to demonstrate LLMs can average a number in C++. So it writes, builds ands runs a "benchmark_mock.cpp" that subs out all the useful functionality for random data from std::mt19937. This naturally works, so the agent declares success and happily updates the README.md with all the crap it added and stops. This is what running the lawnmower over the flower bed means; you have 5 more useless source files and a bunch more shell scripts and a bunch of crap in a README that were all generated to try and fail to fix a problem it could not figure out, and this loop can keep going and generate more nonsense ad infinitum. (Why could it not figure out the linking error? We come back to the shitty bolted on integrations; it doesn't actually query the environment, search for files or look at what link directories are being used, as one would investigating a linking error. It could of course, but the balance in these integrations is 99% LLM and 1% tool use, and even context from the tool use often doesn't help)
- tptacek 1y ago
  It's really weird for me to see people talking about using LLMs in coding situations in a frame where "agents" (we're not even at MCP yet!) are somehow an extra. People discussing the applicability of LLMs to programming, and drawing conclusions (even if only for themselves) about how well it works, should be experienced with a coding agent.
- mountainriver 1y ago
  LLMs aren't good at deep ML problems yet. We know this, its what MLE-bench is for. That doesn't mean they aren't good at other coding problems

tptacek 1y ago

Yes, they're part of my daily toolset. And yes, they can spin out. I just hit the "reject" button when they do, and revise my prompt. Or, sometimes, I just take over and fill in some of the structure of the problem I'm trying to solve myself. I don't know about "self-reinforcing". I'm just saying: coding agents compile and lint the code they're running, and when they hallucinate interfaces, they notice. The same way any developer who has ever used ChatGPT knows that you can paste most errors into the web page and it will often (maybe even usually) come up with an apposite fix. I don't understand how anybody expects to convince LLM users this doesn't work; it obviously does work.

steveklabnik 1y ago
> I don't understand how anybody expects to convince LLM users this doesn't work; it obviously does work. This is really one of the hugest divides I've seen in the discourse about this: anti-LLM people saying very obviously untrue things, which is uh, kind of hilarious in a meta way. https://bsky.app/profile/caseynewton.bsky.social/post/3lo4tdybmgk2v https://bsky.app/profile/caseynewton.bsky.social/post/3lo4td... is an instance of this from a few days ago. I am still trying to sort out why experiences are so divergent. I've had much more positive LLM experiences while coding than many other people seem to, even as someone who's deeply skeptical of what's being promised about them. I don't know how to reconcile the two.
- zoogeny 1y ago
  I think it is pretty simple: people tried it a few times a few months ago in a limited setting, formed an opinion based on those limited experiences and cannot imagine a world where they are wrong. That might sound snarky, but it probably works out for people in 99% of cases. AI and LLMs are advancing at a pace that is so different from any other technology that people aren't yet trained to re-evaluate their assumptions at the high rate necessary to form accurate new opinions. There are too many tools coming (and going, to be fair). HN (and certain parts of other social media) is a bubble of early adopters. We're on the front lines seeing the war in realtime and shaking our heads at what's being reported in the papers back home.

dboreham 1y ago

Someone gave me the tip to add "all source files should build without error", which you'd think would be implicit, but it seems not.

tptacek 1y ago
There's definitely a skill to using them well (I am not yet expert); my only frustration is with people who (like me) haven't refined the skill but have also concluded that there's no benefit to the tool. No, really, in this case, you're mostly just not holding it right. The tools will get better, but what I see happening with people who are good at using them (and from my own code, even in my degraded LLM usage), we have an existence proof of the value of the tools.

giovannibonetti 1y ago

> I think someone else likened it to a lawnmower that will run rampage over your flower bed at the first hiccup This reminds me of a scene from the recent animation movie "Wallace and Gromit: Vengeance Most Fowl" where Wallace actually uses a robot (Norbot) to do gardening tasks, and rampages over Gromit's flower bed. https://youtu.be/_Ha3fyDIXnc https://youtu.be/_Ha3fyDIXnc

zoogeny 1y ago

I mean, I have. I use them every day. You often see them literally saying "Oh there is a linter error, let me go fix it" and then a new code generation pass happens. In the worst case, it does exactly what you are saying, gets stuck in a loop. It eventually gets to the point where it says "let me try just once more" and then gives up. And when that happens I review the code and if it is bad then I "git revert". And if it is 90% of the way there I fix it up and move on. The question shouldn't be "are they infallible tools of perfection". It should be "do I get value equal to or greater than the time/money I spend". And if you use git appropriately you lose at most five minutes on a agent looping. And that happens a couple of times a week. And be honest with yourself, is getting stuck in a loop fighting a compiler, type-checker or lint something you have ever experienced in your pre-LLM days?