7 ms·

I am approaching AI with caution. Shiny things don't generally excite me. Just this week I installed cursor, the AI-assisted VSCode-like IDE. I am working on a

I am approaching AI with caution. Shiny things don't generally excite me. Just this week I installed cursor, the AI-assisted VSCode-like IDE. I am working on a side project and decided to give it a try. I am blown away. I can describe the feature I want built, and it generates changes and additions that get me 90% there, within 15 or so seconds. I take those changes, and carefully review them, as if I was doing a code review of a super-junior programmer. Sometimes when I don't like the approach it took, I ask it to change the code, and it obliges and returns something closer to my vision. Finally, once it is implemented, I manually test the new functionality. Afterward, I ask it to generated a set of automated test cases. Again, I review them carefully, both from the perspective of correctness, and suitability. It over-tests on things that don't matter and I throw away a part of the code it generates. What stays behind is on-point. It has sped up my ability to write software and tests tremendously. Since I know what I want , I can describe it well. It generates code quickly, and I can spend my time revieweing and correcting. I don't need to type as much. It turns my abstract ideas into reasonably decent code in record time. Another example. I wanted to instrument my app with Posthog events. First, I went through the code and added "# TODO add Posthog event" in all the places I wanted to record events. Next, I asked cursor to add the instrumentation code in those places. With some manual copy-and pasting and lots of small edits, I instrumented a small app in <10 minutes. We are at the point where AI writes code for us and we can blindly accept it. We are at a point where AI can take care of a lot of the dreary busy typing work.

irisgrunn 2y ago
And this is the major problem. People will blindly trust the output of AI because it appears to be amazing, this is how mistakes slip in. It might not be a big deal with the app you're working on, but in a banking app or medical equipment this can have a huge impact.
- Gigachad 2y ago
  I feel like I’m being gaslit about these AI code tools. I’ve got the paid copilot through work and I’ve just about never had it do anything useful ever. I’m working on a reasonably large rails app and it can’t seem to answer any questions about anything, or even auto fill the names of methods defined in the app. Instead it just makes up names that seem plausible. It’s literally worse than the built in auto suggestions of vs code, because at least those are confirmed to be real names from the code. Maybe these tools work well on a blank project where you are building basic login forms or something. But certainly not on an established code base.
  - thewarrior 2y ago
    It’s writing most of my code now. Even if it’s existing code you can feed in the 1-2 files in question and iterate on them. Works quite well as long as you break it down a bit. It’s not gas lighting the latest versions of GPT, Claude, Lama have gotten quite good
    - Gigachad 2y ago
      These tools must be absolutely massively better than whatever Microsoft has then because I’ve found that GitHub copilot provides negative value, I’d be more productive just turning it off rather than auditing it’s incorrect answers hoping one day it’s as good as people market it as.
      - piker 2y ago
        Which languages do you use?
      - diggan 2y ago
        > These tools must be absolutely massively better than whatever Microsoft has then I haven't used anything from Microsoft (including Copilot) so not sure how it compares, but compared to any local model I've been able to load, and various other remote 3rd party ones (like Claude), no one comes near to GPT4 from OpenAI, especially for coding. Maybe give that a try if you can. It still produces overly verbose code and doesn't really think about structure well (kind of like a junior programmer), but with good prompting you can kind of address that somewhat.
        FridgeSeal 2y ago
        My experience was the opposite. GPT4 and variants would only respond in vagaries, and had to be endlessly prompted forward, Claude was the opposite, wrote actual code, answered in detail, zero vagueness, could appropriately re-write and hoist bits of code.
        2y ago
- svara 2y ago
  I don't think this criticism is valid at all. What you are saying will occasionally happen, but mistakes already happen today. Standards for quality, client expectations, competition for market share, all those are not going to go down just because there's a new tool that helps in creating software. New tools bring with them new ways to make errors, it's always been that way and the world hasn't ended yet...
- koliber 2y ago
  OP here. I am explicitly NOT blindly trusting the output of the AI. I am treating it as a suspicious set of code written by an inexperienced developer. Doing full code review on it.
DanHulton 2y ago
I sincerely worry about a future when most people act in this same manner. You have - for now - sufficient experience and understanding to be able to review the AI's code and decide if it was doing what you wanted it to. But what about when you've spent months just blindly accepting" what the AI tells you? Are you going to be familiar enough with the project anymore to catch its little mistakes? Or worse, what about the new generation of coders who are growing up with these tools, who NEVER had the expertise required to be able to evaluate AI-generated code, because they never had to learn it, never had to truly internalize it? It's late, and I think if I try to write any more just now, I'm going to go well off the rails, but I've gone into depth on this topic recently, if you're interested: https://greaterdanorequalto.com/ai-code-generation-as-an-agent-of-tech-debt-creation/ https://greaterdanorequalto.com/ai-code-generation-as-an-age... In the article, I posit a less than glowing experience with coding tools than you've had, it sounds like, but I'm also envisioning a more complex use case, like when you need to get into the meat of some you-specific business logic it hasn't seen, not common code it's been exposed to thousands of times, because that's where it tends to fall apart the most, and in ways that are hard to detect and with serious consequences. If you haven't run into that yet, I'd be interested to know if you do some day. (And also to know if you don't, though, to be honest! Strong opinions, loosely held, and all that.)
- wickedsight 2y ago
  You and I seem to live in very different worlds. The one I live and work in is full of over confident devs that have no actual IT education and mostly just copy and modify what they find on the internet. The average level of IT people I see daily is down right shocking and I'm quite confident that OP's workflow might be better for these people in the long run.
  - nyarlathotep_ 2y ago
    It's going to be very funny in the next few years when Accenture et al charge the government billions for a simple Java crud website thing that's entirely GPT-generated, and it'll still take 3 years and not be functional. Ironically, it'll be of better quality then they'd deliver otherwise. This is probably already happening.
t420mom 2y ago
I don't really want to increase the amount of time I spend doing code reviews. It's not the fun part of programming for me. Now, if you could switch it around so that I write the code, and the AI reviews it, that would be something. Imagine if your whole team got back the time they currently spend on performing code reviews or waiting for code reviews.
- gvurrdon 2y ago
  This would indeed be the best way around.The code reviews might even be better - currently, there's little time for them and we often have only one person in the team with much knowledge in the relevant language/framework/application, so reviews are often just "looks OK to me". It's not quite the same, but I'm reminded of seeing a documentary decades ago which (IIRC) mentioned that a factor in air accidents had been the autopilot flying the plane and human pilots monitoring it. Having humans fly and the computer warn them of potential issues was apparently safer.
- digging 2y ago
  > Now, if you could switch it around so that I write the code, and the AI reviews it, that would be something. I'm sort of doing that. I'm working on a personal project in a new language and asking Claude for help debugging and refactoring. Also, when I don't know how to create a feature, I might ask it to do so for me, but I might instead ask it for hints and an overview so I can enjoy working out the code myself.
layer8 2y ago
> I can spend my time revieweing and correcting. Do you really like spending most of your time reviewing AI output? I certainly don’t, that’s soul-crushing.
- koliber 2y ago
  That's essentially what many hands-on engineering managers or staff engineers do today. They spend significant portions of their day reviewing code from more junior team members. Reviewing and modifying code is more engaging than typing out the solution that is fully formed in my head. If the AI creates something close to what I have in my head from the description I gave it, I can work with it to get it even closer. I can also hand-edit it.
- sgu999 2y ago
  Not much more than reviewing the code of any average dev who doesn't bother doing their due diligence. At least with an AI I immediately get an answer with "Oh yes, you're right, sorry for the oversight" and a fix. Instead of some bullshit explanation to try to convince me that their crappy code is following the specs and has no issues. That said, I'm deeply saddened by the fact that I won't be passing on a craft I spent two decades refining.
- woah 2y ago
  I think there are two types of developers: those who are most excited about building things, and those who are most excited about the craft of programming. If I can build things faster, then I'm happy to spend most of my time reviewing AI code. That doesn't mean that I never write code. Some things the AI is worse at, or need to be exactly write and its faster to do them manually.
  - samcat116 2y ago
    I think we could see a lot of these AI code tools start to pivot towards product folks for just this reason. They aren't meant for the people who find craft in what they do.
  - koliber 2y ago
yread 2y ago
I use it for simple tasks where spotting a mistake is easy. Like writing language binding for a REST API. It's a bunch of methods that look very similar, simple bodies. But it saves quite some work Or getting keywords to read about from a field I know nothing about, like caching with zfs. Now I know what things to put in google to learn more to get to articles like this one https://klarasystems.com/articles/openzfs-all-about-l2arc/ https://klarasystems.com/articles/openzfs-all-about-l2arc/ which for some reason doesn't appear in top google results for "zfs caching" for me
syncr0 2y ago
"I say your civilization, because as soon as we started thinking for you it really became our civilization, which is of course what this is all about." - Agent Smith "Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." - Dune
- hn_throwaway_99 2y ago
  But I think that quote is a pretty gross mischaracterization of the parent comment. I similarly am a big fan of Cursor. But I don't "turn [my] thinking over to machines". Even though I review every piece of code it generates and make sure I understand it, it still saves me a ton of time. Heck, some of the most value I get from Cursor isn't even it generating code for me, it's getting to ask questions about a very large codebase with many maintainers where I'm unfamiliar with large chunks. E.g. asking questions like "I would like to do X, are there any places in this codebase that already do this?" I'm also skeptical of LLMs ever being able to live up to their hype ("AGI is coming sooooon!!!!"), but I still find them to be useful tools in context that can save me a lot of time.
BrouteMinou 2y ago
If you are another "waterboy" doing crud applications, the problem has been solved a long time ago. What I mean by that is, the "waterboy" (crud "developer") is going to fetch the water (sql query in the database), then bring the water (Clown Bob layer) to the UI... The size of your Clown Bob layer may vary from one company to another... This has been solved a long time ago. It has been a well-paid clerk job that is about to come to an end. If you are doing pretty much anything else, the AI is pathetically incapable of doing any piece of code that makes sense. Another great example, yesterday, I wanted to know if VanillaOs was using systemD or not. I did scroll through their frontpage but I didn't see anything, so I tried the AI Chat from duckduckgo. This is a frontend for AI chatbots that includes ChatGPT, Llama, Claude and another one... I started my question by: "can you tell me if VanillaOS is using runit as the init system?"... I wanted initially ask if it was using systemd, but I didn't want to _suggest_ systemd at first. And of course, all of them told me: "Yeah!! It's using runit!". Then for all of them I replied, without any fact in hands: "but why on their website they are mentioning to use systemctl to manage the services then?". And... of course! All of them answered: "Ooouppsss, my mistake, VanillaOS uses systemD, blablabla".... So at the end, I still don't know which init VanillaOS is using. If you are trusting the AI as you seem to do, I wish you the best luck my friend... I just hope you will realize the damage you are doing to yourself by "stopping" coding and letting something else do the job. That skill, my friend, is easily lost with time; don't let it evaporate from your brain for some vaporware people are trying to sell you. Take care.
smm11 2y ago
I was in the newspaper field a year or two before desktop publishing took off, then a few years into that evolution. Rooms full of people and Linotype/Compugraphic equipment were replaced by one Mac and a printer. I shot film cameras for years, and we had a darkroom, darkroom staff, and a film/proofsheet/print workflow. One digital camera later and that was all gone. Before me publications were produced with hot lead. Get off my lawn. https://www.nytimes.com/2016/06/02/insider/1966-2016-the-last-hot-type-printer-puts-down-his-tools.html https://www.nytimes.com/2016/06/02/insider/1966-2016-the-las...
latexr 2y ago
> We are at the point where AI writes code for us and we can blindly accept it. I’m waiting for the day we’ll get the first major breach because someone did exactly that. This is not a case of “if”, it is very much a “when”. I’ve seen enough buggy LLM-generated code and enough people blindly accepting it to be confident in that assertion. I do hope it doesn’t happen, but I think it will.

irisgrunn 2y ago

And this is the major problem. People will blindly trust the output of AI because it appears to be amazing, this is how mistakes slip in. It might not be a big deal with the app you're working on, but in a banking app or medical equipment this can have a huge impact.

Gigachad 2y ago
I feel like I’m being gaslit about these AI code tools. I’ve got the paid copilot through work and I’ve just about never had it do anything useful ever. I’m working on a reasonably large rails app and it can’t seem to answer any questions about anything, or even auto fill the names of methods defined in the app. Instead it just makes up names that seem plausible. It’s literally worse than the built in auto suggestions of vs code, because at least those are confirmed to be real names from the code. Maybe these tools work well on a blank project where you are building basic login forms or something. But certainly not on an established code base.
- thewarrior 2y ago
  It’s writing most of my code now. Even if it’s existing code you can feed in the 1-2 files in question and iterate on them. Works quite well as long as you break it down a bit. It’s not gas lighting the latest versions of GPT, Claude, Lama have gotten quite good
  - Gigachad 2y ago
    These tools must be absolutely massively better than whatever Microsoft has then because I’ve found that GitHub copilot provides negative value, I’d be more productive just turning it off rather than auditing it’s incorrect answers hoping one day it’s as good as people market it as.
    - piker 2y ago
      Which languages do you use?
    - diggan 2y ago
      > These tools must be absolutely massively better than whatever Microsoft has then I haven't used anything from Microsoft (including Copilot) so not sure how it compares, but compared to any local model I've been able to load, and various other remote 3rd party ones (like Claude), no one comes near to GPT4 from OpenAI, especially for coding. Maybe give that a try if you can. It still produces overly verbose code and doesn't really think about structure well (kind of like a junior programmer), but with good prompting you can kind of address that somewhat.
      - FridgeSeal 2y ago
        My experience was the opposite. GPT4 and variants would only respond in vagaries, and had to be endlessly prompted forward, Claude was the opposite, wrote actual code, answered in detail, zero vagueness, could appropriately re-write and hoist bits of code.
        2y ago
svara 2y ago
I don't think this criticism is valid at all. What you are saying will occasionally happen, but mistakes already happen today. Standards for quality, client expectations, competition for market share, all those are not going to go down just because there's a new tool that helps in creating software. New tools bring with them new ways to make errors, it's always been that way and the world hasn't ended yet...
koliber 2y ago
OP here. I am explicitly NOT blindly trusting the output of the AI. I am treating it as a suspicious set of code written by an inexperienced developer. Doing full code review on it.

DanHulton 2y ago

I sincerely worry about a future when most people act in this same manner. You have - for now - sufficient experience and understanding to be able to review the AI's code and decide if it was doing what you wanted it to. But what about when you've spent months just blindly accepting" what the AI tells you? Are you going to be familiar enough with the project anymore to catch its little mistakes? Or worse, what about the new generation of coders who are growing up with these tools, who NEVER had the expertise required to be able to evaluate AI-generated code, because they never had to learn it, never had to truly internalize it? It's late, and I think if I try to write any more just now, I'm going to go well off the rails, but I've gone into depth on this topic recently, if you're interested: https://greaterdanorequalto.com/ai-code-generation-as-an-agent-of-tech-debt-creation/ https://greaterdanorequalto.com/ai-code-generation-as-an-age... In the article, I posit a less than glowing experience with coding tools than you've had, it sounds like, but I'm also envisioning a more complex use case, like when you need to get into the meat of some you-specific business logic it hasn't seen, not common code it's been exposed to thousands of times, because that's where it tends to fall apart the most, and in ways that are hard to detect and with serious consequences. If you haven't run into that yet, I'd be interested to know if you do some day. (And also to know if you don't, though, to be honest! Strong opinions, loosely held, and all that.)

wickedsight 2y ago
You and I seem to live in very different worlds. The one I live and work in is full of over confident devs that have no actual IT education and mostly just copy and modify what they find on the internet. The average level of IT people I see daily is down right shocking and I'm quite confident that OP's workflow might be better for these people in the long run.
- nyarlathotep_ 2y ago
  It's going to be very funny in the next few years when Accenture et al charge the government billions for a simple Java crud website thing that's entirely GPT-generated, and it'll still take 3 years and not be functional. Ironically, it'll be of better quality then they'd deliver otherwise. This is probably already happening.

t420mom 2y ago

I don't really want to increase the amount of time I spend doing code reviews. It's not the fun part of programming for me. Now, if you could switch it around so that I write the code, and the AI reviews it, that would be something. Imagine if your whole team got back the time they currently spend on performing code reviews or waiting for code reviews.

gvurrdon 2y ago
This would indeed be the best way around.The code reviews might even be better - currently, there's little time for them and we often have only one person in the team with much knowledge in the relevant language/framework/application, so reviews are often just "looks OK to me". It's not quite the same, but I'm reminded of seeing a documentary decades ago which (IIRC) mentioned that a factor in air accidents had been the autopilot flying the plane and human pilots monitoring it. Having humans fly and the computer warn them of potential issues was apparently safer.
digging 2y ago
> Now, if you could switch it around so that I write the code, and the AI reviews it, that would be something. I'm sort of doing that. I'm working on a personal project in a new language and asking Claude for help debugging and refactoring. Also, when I don't know how to create a feature, I might ask it to do so for me, but I might instead ask it for hints and an overview so I can enjoy working out the code myself.

layer8 2y ago

> I can spend my time revieweing and correcting. Do you really like spending most of your time reviewing AI output? I certainly don’t, that’s soul-crushing.

koliber 2y ago
That's essentially what many hands-on engineering managers or staff engineers do today. They spend significant portions of their day reviewing code from more junior team members. Reviewing and modifying code is more engaging than typing out the solution that is fully formed in my head. If the AI creates something close to what I have in my head from the description I gave it, I can work with it to get it even closer. I can also hand-edit it.
sgu999 2y ago
Not much more than reviewing the code of any average dev who doesn't bother doing their due diligence. At least with an AI I immediately get an answer with "Oh yes, you're right, sorry for the oversight" and a fix. Instead of some bullshit explanation to try to convince me that their crappy code is following the specs and has no issues. That said, I'm deeply saddened by the fact that I won't be passing on a craft I spent two decades refining.
woah 2y ago
I think there are two types of developers: those who are most excited about building things, and those who are most excited about the craft of programming. If I can build things faster, then I'm happy to spend most of my time reviewing AI code. That doesn't mean that I never write code. Some things the AI is worse at, or need to be exactly write and its faster to do them manually.
- samcat116 2y ago
  I think we could see a lot of these AI code tools start to pivot towards product folks for just this reason. They aren't meant for the people who find craft in what they do.
- koliber 2y ago

yread 2y ago

I use it for simple tasks where spotting a mistake is easy. Like writing language binding for a REST API. It's a bunch of methods that look very similar, simple bodies. But it saves quite some work Or getting keywords to read about from a field I know nothing about, like caching with zfs. Now I know what things to put in google to learn more to get to articles like this one https://klarasystems.com/articles/openzfs-all-about-l2arc/ https://klarasystems.com/articles/openzfs-all-about-l2arc/ which for some reason doesn't appear in top google results for "zfs caching" for me

syncr0 2y ago

"I say your civilization, because as soon as we started thinking for you it really became our civilization, which is of course what this is all about." - Agent Smith "Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." - Dune

hn_throwaway_99 2y ago
But I think that quote is a pretty gross mischaracterization of the parent comment. I similarly am a big fan of Cursor. But I don't "turn [my] thinking over to machines". Even though I review every piece of code it generates and make sure I understand it, it still saves me a ton of time. Heck, some of the most value I get from Cursor isn't even it generating code for me, it's getting to ask questions about a very large codebase with many maintainers where I'm unfamiliar with large chunks. E.g. asking questions like "I would like to do X, are there any places in this codebase that already do this?" I'm also skeptical of LLMs ever being able to live up to their hype ("AGI is coming sooooon!!!!"), but I still find them to be useful tools in context that can save me a lot of time.

BrouteMinou 2y ago

If you are another "waterboy" doing crud applications, the problem has been solved a long time ago. What I mean by that is, the "waterboy" (crud "developer") is going to fetch the water (sql query in the database), then bring the water (Clown Bob layer) to the UI... The size of your Clown Bob layer may vary from one company to another... This has been solved a long time ago. It has been a well-paid clerk job that is about to come to an end. If you are doing pretty much anything else, the AI is pathetically incapable of doing any piece of code that makes sense. Another great example, yesterday, I wanted to know if VanillaOs was using systemD or not. I did scroll through their frontpage but I didn't see anything, so I tried the AI Chat from duckduckgo. This is a frontend for AI chatbots that includes ChatGPT, Llama, Claude and another one... I started my question by: "can you tell me if VanillaOS is using runit as the init system?"... I wanted initially ask if it was using systemd, but I didn't want to _suggest_ systemd at first. And of course, all of them told me: "Yeah!! It's using runit!". Then for all of them I replied, without any fact in hands: "but why on their website they are mentioning to use systemctl to manage the services then?". And... of course! All of them answered: "Ooouppsss, my mistake, VanillaOS uses systemD, blablabla".... So at the end, I still don't know which init VanillaOS is using. If you are trusting the AI as you seem to do, I wish you the best luck my friend... I just hope you will realize the damage you are doing to yourself by "stopping" coding and letting something else do the job. That skill, my friend, is easily lost with time; don't let it evaporate from your brain for some vaporware people are trying to sell you. Take care.

smm11 2y ago

I was in the newspaper field a year or two before desktop publishing took off, then a few years into that evolution. Rooms full of people and Linotype/Compugraphic equipment were replaced by one Mac and a printer. I shot film cameras for years, and we had a darkroom, darkroom staff, and a film/proofsheet/print workflow. One digital camera later and that was all gone. Before me publications were produced with hot lead. Get off my lawn. https://www.nytimes.com/2016/06/02/insider/1966-2016-the-last-hot-type-printer-puts-down-his-tools.html https://www.nytimes.com/2016/06/02/insider/1966-2016-the-las...

latexr 2y ago

> We are at the point where AI writes code for us and we can blindly accept it. I’m waiting for the day we’ll get the first major breach because someone did exactly that. This is not a case of “if”, it is very much a “when”. I’ve seen enough buggy LLM-generated code and enough people blindly accepting it to be confident in that assertion. I do hope it doesn’t happen, but I think it will.