3 ms·
I think what people are missing is that they work sometimes and sometimes they don't work. People think "Oh, it works better when somebody else does it" or "T
by PaulHoule 1y ago
I think what people are missing is that they work sometimes and sometimes they don't work.
People think "Oh, it works better when somebody else does it" or "There must be some model that does better than the one I am using" or "If I knew how to prompt better I'd get better results" or "There must be some other agentic IDE which is better than the one I am using."
All those things might be true but they just change the odds, they don't change the fact that it works sometimes and fails other times.
For instance I asked an agent to write me a screen to display some well-typed data. It came up with something great right away that was missing some fields and had some inconsistent formatting but it fixed all those problems when I mentioned them -- all speaking the language of product managers and end users. The code quality was just great, as good as if I wrote it, maybe better.
Plenty of times it doesn't work out like that.
I was working on some code where I didn't really understand the typescript types and fed it the crazy error messages I was getting and it made a try to understand them and didn't really, I used it as a "rubber duck" over the course of a day or two and working with it I eventually came to understand what was wrong and how to fix and I got into a place that I like and when there is an error I can understand it and it can understand it too.
Sometimes it writes something that doesn't typecheck and I tell it to run tsc and fix the errors and sometimes it does a job I am proud of and other times it adds lame typeguards like
if (x && typeof x === "object") x.someMethod()
Give it essentially the same problem, say writing tests in Java, and it might take very different approaches. One time it will use the same dependency injection framework used in other tests to inject mocks into private fields, other times it will write some a helper method to inject the mocks into private fields with introspection directly.
You might be able to somewhat tame this randomness with better techniques but sometimes it works and sometimes it doesn't and if I just told you about the good times or just told you about the bad times it would be a very different story.
- leptons 1y ago>I was working on some code where I didn't really understand the typescript types and fed it the crazy error messages I was getting and it made a try to understand them and didn't really, I used it as a "rubber duck" over the course of a day or two and working with it I eventually came to understand what was wrong and how to fix and I got into a place that I like and when there is an error I can understand it and it can understand it too. I have to wonder if you tried a simple google search and read through some docs if you couldn't have figured this out quicker than trying to coax a result out of the LLM.
- solarkraft 1y agoMy (not GP) intuitive answer would be hell no. Typescript messages are pretty hard to google and even parse manually and the LLM suggesting multiple approaches and ways to think about the problem does seem useful. It sometimes uncovers unknown unknowns you might never find otherwise. I have had cases in which a web search and some good old fashioned thinking have yielded better results than using an LLM, but on average I’m pretty sure the LLM has the edge.
- PaulHoule 1y agoPersonally I think Stack Overflow and Google are 95% trash for that kind of problem. The answers are in (i) the Typescript documentation and (ii) the documentation of libraries that I'm using. I could get lucky with a Google search and it could be worth trying, but I wouldn't expect it to work. Personally my preference is to have a language and libraries with great documentation (Python, Java, Typescript isn't too bad [1]) and really know that documentation like the back of my hand. If I hadn't had the LLM I would have probably figured it out the same way doing experiments, I might have asked my other "rubber duck" https://mastodon.social/@UP8/113935901671533690 https://mastodon.social/@UP8/113935901671533690 A tactic I didn't use, which helps in "legacy" systems where I am stuck, is start a fresh project in the IDE and try either reproducing the problem or building a tiny system which is problem free. I'm hesitant to say what speedup I got out of the "figuring out the types together with the LLM" but emotionally I felt supported and in the process I wrote a whole lot, like I was keeping track of the process in a notebook. I feel a lot times when I have good LLM conversations I wind up writing better code than I would otherwise, not necessarily write it faster -- it's like pair programming. [1] The typescript docs are great for the typescript stuff, MDN is good for Javascript and Javascript's stdlib
- leptons 1y ago"felt emotionally supported by an LLM" Wow. This sounds so foreign to me. Being emotionally supported by the proverbial "tin can". If that makes you happy, then I'm happy for you, I guess?
- rediscovery 1y agoBehavioral psychology goes a long way in explaining the weird distribution of responses to using LLMs to generate code: "Gambling-like behavior in pigeons: ‘jackpot’ signals promote maladaptive risky choice" https://www.nature.com/articles/s41598-017-06641-x https://www.nature.com/articles/s41598-017-06641-x The tech industry is actively promoting gambling addiction and the scary thing is that people are willingly walking into that trap. Take a look at this comment: https://news.ycombinator.com/item?id=44849147 https://news.ycombinator.com/item?id=44849147 "Most of what I've learned from talking to people about their workflows is counterintuitive and subtle." Seriously? Are we at the point of doing rain dances for these models and describing the moves as "counterintuitive and sublte"? This is some magical thinking level self delusion. Downvote all you like, or ignore this. Agency is being taken away from us, no one gets to say we didn't see it coming down the line because we did and we said something and our peers treated us like ignorant and self interested for pointing out the obvious.
- ath3nd 1y agoVery insightful. You are not alone: https://rgoldfinger.com/blog/2025-07-26-claude-code-is-a-slot-machine/ https://rgoldfinger.com/blog/2025-07-26-claude-code-is-a-slo... > Are we at the point of doing rain dances for these models and describing the moves as "counterintuitive and sublte" Yes, we are. LLM cultists can downvote as much as they like, but the reality is that with all the incantations and coaxing, we not only don't see a positive effect, we see a net negative effect on the quality of our software. On the ground level of OSS we are drowning in AI generated slop: https://www.theregister.com/2025/07/15/curl_creator_mulls_nixing_bug/ https://www.theregister.com/2025/07/15/curl_creator_mulls_ni... On the enterprise level we are creating more and worse code, faster. In the news, we are generating more and worse and factually wrong articles, faster. In our personal lives, guided by our sycophantic LLMs assistants, we offload more and more cognitive chores, becoming dumber and dumber. LLM-produced slop is like the plastic of text: cheap and easy to use and proliferate, and then 40 years later you realize it's everywhere in the environment and in your brain, and that it's toxic to all life.
- jama211 1y agoAnd like plastic, a fantastic material for many jobs, it’s just has its own pros and cons