7 ms·

My social feeds are full of tech bros who keep telling people AI codes everything for them. AI obviously has some impressive coding skills, but for me it never

• HL33tibCe7 2y ago
  Current-gen AI can write obvious code well, but fails at anything that involves complexity or subtlety in my experience
• frodo8sam 2y ago
  For me ai has been pretty useful. Difference is I'm not a software engineer, I just write scripts to help me do my job. If I wrote bigger applications I doubt llms could help me.
  • andix 2y ago
    AI is awesome for small coding tasks, with a defined scope. I don't write shell (or powershell) scripts anymore, AI does it now for me. But once a project has more than 20 source files, most AI tools seem to be unable to grasp the context of the project. In my experience AI is really bad at multi threading code and distributed systems. It seems to be unable to build its "mental model" for those kind of problems.
    • Earw0rm 2y ago
      This. It's good at Cmake, and mostly good at dealing with COM boilerplate (although hallucinations are still a problem). But threading and asynchronous code are implicit - there's a lot going on that you can't see on the page, you need to think about what the system is actually doing rather than simply the words to make it do the thing.
  • dahousecat 2y ago
    They are useful for small tasks like refactoring a method however big the whole project is
• rco8786 2y ago
  Current gen AI can spit out some very, very basic web sites (I won't even elevate to the word "app") with some handholding and poking and prodding to get it to correct its own mistakes. There is no one out there building real, marketable production apps where AI "codes everything for them". At least not yet, but even in the future it seems infeasible because of context. I think even the most pro-AI people out there are vastly underestimating the amount of context that humans have and need to manage in order to build fully fledged software. It is pretty great as a ridealong pair programmer though. I've been using Cursor as my IDE and can't imagine going back to a non-AI coding experience.
• jgilias 2y ago
  I think it’s that AI unlocks the ability to code something up and test an idea for people who’re technical enough to get it working, but not really developers themselves. It’s not (yet at least) a substitute for a good dev team that knows what they’re doing. But this is still huge, and shouldn’t be disregarded.
• loveparade 2y ago
  In my experience, AI is good at building stuff in two scenarios: - You have zero engineering background and you use an LLM to build an MVP from scratch. As long as the MVP is sufficiently simple there is plenty of training data for LLM to do well. E.g. some kind of React website with a simple REST API backend. This works as long as the app is simple enough, but it'll start breaking down as the app becomes more complex and requires domain-specific business knowledge or more sophisticated engineering techniques. Because you don't understand what the LLM is doing, you can't debug or extend any of it. - You are an experienced developer and know EXACTLY what you want. You then use an LLM to write all the boilerplate for you. I was surprised at how much of my daily engineering work is actually just boilerplate. Using an LLM has made me a significantly more productive. This only works if you know what you're doing, can spot mistakes immediately, and can describe in detail HOW an LLM should be doing the task. For use cases in middle, LLMs kind of suck. So I think the comparison to a (very) junior engineer is quit apt. If the task is simple you can just let them do it. If the task is hard or requires a lot of context, you need to give them step by step instructions on how to go about it, and that requires that you know how to do it yourself.
  • bdangubic 2y ago
    these are exactly my experiences as well. senior devs on my team and rocking and rolling with the AI. my junior devs have all but given up using it even after numerous retros etc…
• wanderingbort 2y ago
  I think it’s selection bias. Marketers are going to post the proof-of-concept that it works (if only in a small isolated scenario), algorithms are going to emphasize the more amazing “toys” this produces, over the boring rebuttals. In the end, you will see hundreds of examples where it worked and not the the thousands where it produced buggy or dangerous code. That attention does not map well to the important, hard, and more valuable parts of development. Anecdotally, I still find it to be useful and it’s improving. I do think it’s going to be an huge impact in time. Hype is part of the industry and it can be distracting to users, developers, and investors BUT it can also be useful (and I don’t know how to replace it) so, we live with it.
• javier2 2y ago
  Same experience. It has become pretty good at writing creative SQL queries though. Its actually rather good at that. When I am working on something niche, it does not help either. I have tried to make it build modern UI applications for myself using modern Java, but it just can't. It hallucinates libs and functions that does not exists, and I cant really get it to produce what I want. I have had better experiences with languages that are simpler and more predictable (Go), and languages with huge amounts of learning material available (Typescript / React). But I have been trying to build open source UI apps in JavaFX and GTK, it just can not help me when I am stuck
• williamcotton 2y ago
  https://github.com/williamcotton/webdsl https://github.com/williamcotton/webdsl Made almost entirely with Cursor and Claude 3.5 Sonnet. 11k lines of C and counting.
  • andix 2y ago
    Thanks, that's exactly what I'm looking for.
  • thomasfromcdnjs 2y ago
    I used Windsurf mostly on a feature to build out user authentication and then another tool to generate the PR documentation entirely. https://github.com/jsonresume/jsonresume.org/pull/176 https://github.com/jsonresume/jsonresume.org/pull/176 Meets my good enough standards fo sure
  • BeefWellington 2y ago
    It's funny that this is MIT licensed, expecting credit for uncopyrightable work.
    • williamcotton 2y ago
      I work in copyright law. Familiarize yourself with the AFC test concept and I'm willing to have a conversation about what would be copyrightable in this project of mine. https://en.wikipedia.org/wiki/Abstraction-Filtration-Comparison_test https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari...
      • BeefWellington 2y ago
        You misunderstand, the repo lists an MIT license which requires attribution. You want people to give you credit if they use this LLM-generated code. LLMs which were trained on the works of thousands of other developers with similar licenses, who are offered no similar credit here. It also claims copyright of the code as though you have authored it, but you're claiming here to have used LLMs to generate it. Seems like trying to have it both ways.
• dagw 2y ago
  AI isn't great at creating software, but it is great at writing functions. I often ask AI to "write a function that takes A, looks up B in a SQL database, and returns C, or write a function that implements the FooBar algorithm in C++" and on the whole that works pretty well. Asking it to write documentations for those functions also works really well. Asking it to write unit tests for those functions works pretty well (although you have to be extra careful, because sometimes the tests are wrong). What you have to do, and what AI cannot do well, is to decide where in the codebase to put those functions, and decide how to structure the code around those functions. You have to decide how and when and why to call each of those functions.
  • javier2 2y ago
    When I have to be that specific with it, it would be faster for me to just write it directly in my normal IDE with great auto complete
    • dagw 2y ago
      it would be faster for me to just write it directly in my normal IDE Then you are a much better developer than me (which you may very well be). I'd like to think I'm pretty good, and I've many times spent hours trying to think through complex SQL queries or getting all the details right in some tricky equation or algorithm. Writing the same code with an AI often takes 2-20 minutes. If it's faster for me, it might not be faster for everybody, but it is probably faster for many people.
      • jprete 2y ago
        The way to get better is to do it a lot. Every time you dig through a problem to solve it, you're not just learning about that problem - you're learning about every problem near it in the problem space, including the meta-problems about how to get information, solve problems, and test solutions. In a sense you're slowly building the LLM in your head, but it's more valuable there because of the much-better idea evaluation, and lack of network/GPU overhead.
      • 2y ago
• mft_ 2y ago
  I experimented with Cursor over christmas, with writing a simple-ish Swift/SwiftUI app on iOS as the challenge. I can code fairly well in Python, moderately in JS, and almost not at all in Swift. I was using Cursor on a Mac, in parallel to XCode. Basically, it worked, but not without issues: - The biggest issue was debugging: because the bugs appeared in XCode, not Cursor, it either meant laboriously describing/transcribing errors into Cursor, or manually fixing them. - The 'parallel' work between Cursor and XCode was clunky, especially when Cursor created new files. It took a while to figure out a halfway-decent workflow. - At one point something screwed up somewhere deep in the confusing depths of XCode, and the app refused to compile altogehter. Neither Cursor nor I could figure it out, but a new project with the files transferred over worked just fine. But... after a few short hours' chatting, learning, and fixing, I had a functional app. It wasn't free of frustrations, and it's pretty far from the level where a non-coder could do the same, but it impressed me that it's already at the level where it's a decent multiplier of someone's abilities.
• jstummbillig 2y ago
  aider (AI assistant, that will do coding for/with you, depending on how you use it) has one of the more illuminating pieces of information on this. Here is a graph of percentage contribution of aider to aider development itself over time: https://aider.chat/HISTORY.html https://aider.chat/HISTORY.html
• cejast 2y ago
  I don't think it is an illusion. It can remove a lot of barriers to entry for some people, and this is probably what you're seeing in the anecdata. For example, my brother. He is what I'd refer to as 'tech-aligned' - he can and has written code before, but does not do it for a living and only ever wrote basic Python scripts every now and then to help with his actual work. LLM's have enabled him to build out web apps in perhaps 1/5 of the time it would have taken him if he tried to learn and build them out from scratch. I don't think he would have even attempted it without an LLM. Now it doesn't 'code everything' - he still has to massage the output to get what he wants, and there is still a learning curve to climb. But the spring-board that LLM's can give people, particularly those who don't have much experience in software development, should not be underestimated.
  • andix 2y ago
    There is a big gap between being able to create a somehow working application and shipping a product to a customer. Those claims are about being able to create a profitable product with 10x efficiency.