11 ms·
I use Cursor for most of my development these days. This article aligns pretty closely with my experiences. A few additional observations: 1. Anecdotally, AI a
by ikerino 2y ago
I use Cursor for most of my development these days. This article aligns pretty closely with my experiences. A few additional observations:
1. Anecdotally, AI agents feel stuck somewhere circa ~2021. If I install newer packages, Claude will revert to outdated packages/implementations that were popular four years ago. This is incredibly frustrating to watch and correct for. Providing explicit instructions for which packages to use can mitigate the problem, but it doesn't solve it.
2. The unpredictability of these missteps makes them particularly challenging. A few months ago, I used Claude to "one-shot" a genuinely useful web app. It was fully featured and surprisingly polished. Alone, I think it would've taken a couple weeks or weekends to build. But, when I asked it to update the favicon using a provided file, it spun uselessly for an hour (I eventually did it myself in a couple minutes). A couple days ago, I tried to spin up another similarly scoped web app. After ~4 hours of agent wrangling I'm ready to ditch the code entirely.
3. This approach gives me the brazenness to pursue projects that I wouldn't have the time, expertise, or motivation to attempt otherwise. Lower friction is exciting, but building something meaningful is still hard. Producing a polished MVP still demands significant effort.
4. I keep thinking about The Tortoise and The Hare. Trusting the AI agent is tempting because progress initially feels so much faster. At the end of the day, though, I'm usually left with the feeling I'd have made more solid progress with slower, closer attention. When building by hand, I rarely find myself backtracking or scrapping entire approaches. With an AI-driven approach, I might move 10x faster but throw away ~70% of the work along the way.
> These experiences mean that by no stretch of my personal imagination will we have AI that writes 90% of our code autonomously in a year. Will it assist in writing 90% of the code? Maybe.
Spot on. Current environment feels like the self-driving car hype cycle. There have been a lot of bold promises (and genuine advances), but I don't see a world in the next 5 years where AI writes useful software by itself.
- Aurornis 2y ago> It feels like my AI agents are stuck somewhere circa ~2021. If I install newer packages or more recent versions, Claude will often revert to outdated packages/implementations that were popular four years ago. My experience is the same, though the exact dates differ. I assume LLMs gravitate toward solutions that are most represented in their training material. It's hard to keep them pulled toward newer versions without explicitly mentioning it all the time.
- spwa4 2y agoBut they can't really be low-latency if they have to search for new versions ... and that makes such a big difference in how usable they are.
- dkarl 2y ago> At the end of the day, though, I'm usually left with the feeling I'd have made more solid progress with slower, closer attention Does you think this feeling reflects the usual underestimation we're all guilty of, or do you think it's accurate?
- ikerino 2y agoIn terms of absolute progress (e.g. towards finishing a feature or MVP) I think it it could have to do with a usual underestimation (optimism) for timelines. I'm using Cursor mostly for exploratory/weekend projects. I usually opt for stacks/libraries I'm less familiar with, so I think there's some optimism/uncertainty to account for there. I think there's another aspect to progress involving learning/becoming fluent in a codebase. When I build something from scratch, I become the expert, so familiar that later features become very easy/obvious to implement. I haven't had this experience when I take a heavily agent-driven approach. I'm steering, but I'm not learning much. The more I progress, the harder new features feel to implement. I don't think this is unique to working with AI. I guess the takeaway is that attention and familiarity matter.
- deleted 2y ago[deleted]
- bigs 2y agoThe tortoise and the hare is an important point to raise - this is exactly how I feel. But how long before AI is Good Enough that the point is irrelevant?
- Hasu 2y agoChecks my notes from 3 years ago, 2.5 years ago, 2 years ago, 1.5 years ago, 1 year ago, and 6 months ago when we had this exact same discussion It says here that it'll only be another 6 months!
- hnlurker22 2y agoIt's all true but I'm surprised it took so long to realize this. As someone who is not an early adopter, I decided to give AI a shot in helping with an existing project a few days ago. I immediately realized almost everything mentioned here. However, the sentiment of everything i've read before was that AI can already replace me. In reality I enjoy steering it to reach a solution.
- sReinwald 2y ago> In reality I enjoy steering it to reach a solution. That resonates with me. It actually brings back some nostalgic memories about setting up and constantly tweaking bots in World of Warcraft (sorry, I was young). There's something incredibly engaging and satisfying about configuring them just right and then sitting back, watching your minions run around and do your bidding. I get a similar vibe when I'm working with AI coding assistants. It's less about the (currently unrealistic) hype of full replacement and more about trying to guide these powerful, and often erratic, tools to do what they're supposed to. For me, it taps into that same core enjoyment of automating and observing a process unfold that I got with those WoW bots. Perhaps unsurprisingly, automation is now a fairly big part of my job.
- kristopolous 2y ago#1 has an easy fix. Clone the dependency you want to use in the directory of your code. Instruct it to go into the directory and look at that code in order to complete task X: "I've got a new directory xyz, in it contains a library to do feature abc. I'll need to include it here to do A to function B and so on" The weird version mixing bug will disappear. If it's closed source, then just do the documentation. You need to line up the breadcrumbs right. #2 is "create a patch file that does X. Do not apply it". Followed by "apply the previous patch file". Manually splitting the task fixes the attention. Another method is to modify the code. Don't use "to-do" it will get confused. Instead use something meaningless like 1gwvDn, then at the appropriate place [1gwvDn: insert blah here] Then go to the agent and say "I've changed the file and given you instructions in the form [1gwvDn:<instructions>]. Go through the code and do each individually. Then the breadcrumbs are right and it doesn't start deleting giant blocks of code and breaking things #3 You will never start anything unless you convince yourself it's going to be easy. I know some people will disagree with this. They're wrong. You need to tell yourself it's doable before you attempt it. #4 is because we lose ownership of the code and end up playing manager. So we do the human thing of asking the computer to do increasingly trivial things because it's "the computer's" code. Realize you're doing that and don't be dumb about it.
- moqizhengz 2y agoThis is a very typical reply when we see someone pointing out the flaws of AI coding tools. "You are using it wrong, AI can do everything if you properly prompt it" Yes, it can write everything if I provide enough context, but it ain't 'Intelligence' if context ~= output. The point here is providing enough context itself is challenging and requires expertise, this makes AI ides unusable for many scenarios.
- infecto 2y agoThis is a very typical reply when we see someone excited about AI and then a luddite needs to come along and tell them why they should not be so excited and helpful. I mostly jest but your comment comes off quite unhelpful and negative. The person you replied to wasn’t blaming the parent comment, just offering helpful tips. I agree that today’s AI tools aren’t perfect, but I also think it’s important for developers to invest time refining their toolset. It’s no different from customizing IDE shortcuts. These tools will improve, but if devs aren’t willing to tinker with what they use, what’s the point?
- emseetech 2y ago#1 is going to be an issue until we have another breakthrough or genuinely innovative approach. We all know that 2 years is a lifetime in tech (for better or for worse), and we've all trained ourselves to keep up with a rapidly changing industry in a way that's more efficient than fully retraining a model with considerably more novel data. For instance, enough people have started to move away from React for more innovative or standards-based approaches. HTML and CSS alone have come a long way since 2013 when React was a huge leap forward. But while those of us doing the development might have that realization, the training data won't reflect that for a good amount of time. So until then, trying to build a non-React approach will involve wrestling with the LLM until the point when the model has caught up. At which point, we will likely still be ahead of the curve in terms of the solutions it provides.
- beepbooptheory 2y agoBut doesnt this just ultimately "solve" itself? If everybody is just going to use LLMs like this, they will keep using React longer, and 2 years is not going to feel like that much of a lifetime anymore. How would, even, a developer inside their AI ecosystem like this even know or care about new frameworks or new ways to do things?
- aprilthird2021 2y agoNo because big companies see far more value from actually innovating and producing better technology (see Bytedance with their new cross-platform mobile framework Lynx) that speeds up things and provides better User Experience at their scale than they save in developer salaries by just using what an AI is most familiar with
- jb_briant 2y agoWhile you give your process feedback, here is my emotions related one. When I dev with LLM, I don't face my own limits in term of reasoning and architecturing, but I face the limit of the model to interpret prompts. Instead of trying to be a better engineer, I'm frustratingly prompting an unintelligent human-like interface. I'm not fuding LLM, I use it everyday, all the time. But it won't make me a better engineer. And I deeply believe that becoming a good engineer helped me becoming a better human, because how the job make you face your own limits, train you to be humble and constantly learning. Vibe coding won't lead to that same and sane mindset.
- jgilias 2y agoI feel like this is the big question now. How to find the correct balance that lets you preserve and improve your skills. I haven’t yet found the answer, but I feel like it should be in being very deliberate about what you let the agent do, and what kind of work you do in the “artisanal” way, so not even AI-enabled auto-complete. But in order to be able to find the right balance, one does need to learn fully what the agent can do, and have a lot of experience with that way of coding. Otherwise the mental model is wrong.
- jb_briant 2y agoAgreed. Auto-complete on steroid isn't what I call using LLM for coding. It's just a convenience. What I do is to write pure functions with LLM. Once I designed the software and I have the API, I can tell the model to write a function which does a specific job where I know the inputs and outputs but I'm lazy to write the code itself.
- liveoneggs 2y agoit will make you a worse thinker
- chrisco255 2y agoI don't see the 2021 problem as much with Grok, but it is wired up to X for realtime learning, it can also fetch and crawl pages. Occasionally I'll use dev branch of some codebase, like Zig, and I just have to let it know the version I'm on and remind it to use the dev version. Occasionally I'll have to paste in the new API and then it works out what the answer should be. A lot of the other problems mentioned still hold though.
- nukem222 2y agoI agree with all of this. I want to offer a tiny bit more hope, though: > There have been a lot of bold promises (and genuine advances), but I don't see a world in the next 5 years where AI writes useful software by itself. I actually think the opposite: that within five years, we will be seeing AI one-shot software, not because LLMs will experience some kind of revolution in auditing output, but because we will move the goalposts to ensure the rough spots of AI are massaged out. Is this cheating? Kind of, but any effort to do this will also ease humans accomplishing the same thing. It's entirely possible, in other words, that LLMs will force engineers to be honest about the ease of tasks they ask developers to tackle, resulting in more easily composable software stacks. I also believe that use of LLMs will force better naming of things. Much of the difficulty of complex projects comes from simply tracking the existence and status of all the moving parts and the wires that connect them. It wouldn't surprise me at all if LLMs struggle to manage without a clear shared ontology (that we naturally create and internalize ourselves).
- lioeters 2y ago> we will move the goalposts to ensure the rough spots of AI are massaged out Totally agree with this point. Software engineering will adapt to work better with LLMs. It will influence how we think about programming language design, as an interface to human readers/writers as well as for machines to "understand", generate, and refine. There was a recent article about how LLMs will stifle innovation due to its cutoff point, where it's more productive using older or more mature frameworks and libraries whose documentation is part of the training data. I'm already seeing how this is affecting technical decisions at companies. But then again, it's similar to how such decisions are often made to maximize the labor pool, for example, choosing a more popular language due to availability of experts. One thing I hope for is that we'll see more emphasis on thorough and precisely worded documentation. Similarly with API design and user interfaces in general, possibly leading to improvements in accessibility also. Another aspect I think about is the recursive cycle of LLM-generated code and documentation being consumed by future LLMs, influencing what kinds of new frameworks and libraries may emerge that are particularly suited for this new kind of programming, purely AI or human/AI symbiosis.
- 2y ago
- petesergeant 2y ago> Spot on. Current environment feels like the self-driving car hype cycle. There have been a lot of bold promises (and genuine advances), but I don't see a world in the next 5 years where AI writes useful software by itself. My worry is that we get an overnight sea-change like 4o image generation. The current tools aren't good enough for anything other than one-shotting, and then suddenly overnight, they're good enough to put a lot of people out of work.
- impjohn 2y agoHm. The 'prompt obedience' was definitely a huge step up, but there's a huge number of acceptable results to an image generation prompt, while for coding usually there a handful, many times just one right solution. So I don't think this parallel here is telling
- re-thc 2y ago> Current environment feels like the self-driving car hype cycle. Is that really hype? I mean there companies or person(s) hyping it up, but there is also Waymo and Pingshan (Baidu) for example actually rolling it out. It's a lot less hype than AI coding. > Anecdotally, AI agents feel stuck somewhere circa ~2021. That's only part of the problem. It's also stuck or dead set on certain frameworks / libraries where it has training data. > I keep thinking about The Tortoise and The Hare. This implies that AI is currently "smart". The hare is "smarter" and just takes breaks i.e. it can actually get the job done. With the current "AI" there are still quirks where it can get stuck.
- ikerino 2y ago> Is that really hype? I'm thinking back to various promises self-driving would be widespread by 2016. These set a certain expectation for how our roads would look that I don't think has been realized a decade later (even as I've ridden in Waymos/FSD Teslas.)
- dingnuts 2y agoif Elon Musk hadn't convinced the world in 2015 that self driving cars were right around the corner, that could work with just a few sensors and machine learning, maybe we'd have smart highways now. Slightly less fancy technology communicating hazards and such to cars from sensors on the road. But no, we have Waymo, I guess, so that means Musk was right?
- BigParm 2y agoThe best choice is to do it right the first time, instead of spending billions in infrastructure which soon becomes obsolete.
- Nemi 2y agoI don’t do much development anymore, but this is always how I felt when ORM’s like Hibernate came out. At first they were amazing and were used for everything. Eventually it became obvious that as the complexity of the code went up linearly, the difficulty of troubleshooting and optimizing the code went up exponentially. I learned to only used ORM’s for basic stuff, which they are very much useful, but when things got a little bit complicated to drop back to hand coding SQL.
- jdkoeck 2y agoOne of the things I love most about AI coding tools is how they make it easier than ever to move away from complex, fragile abstractions like ORMs. The main reason we’ve often avoided writing raw SQL repositories by hand is that it’s tedious, but now, much of that boilerplate can be generated by AI. More broadly, wherever you might have relied on a heavyweight dependency, you can often replace it with AI-generated code tailored to the task at hand. You might think this would increase the maintenance burden, but au contraire: reviewing and taking ownership of AI-generated code is often simpler and more sustainable in the long term than dealing with complex libraries you don’t fully control.
- aprilthird2021 2y ago> A few months ago, I used Claude to "one-shot" a genuinely useful web app. It was fully featured and surprisingly polished. Alone, I think it would've taken a couple weeks or weekends to build. But, when I asked it to update the favicon using a provided file, it spun uselessly for an hour (I eventually did it myself in a couple minutes). This reminds me of when cross-platform was becoming big for mobile apps and all of us new app developers would put up templates on GitHub which gave a great base to start on, but you really quickly realized you'd have to change a lot of it for your use case anyways.