8 ms·

A lot of software developers have an initial bad experience and assume it's terrible and give up on it. I feel bad for people who haven't yet experienced how u

A lot of software developers have an initial bad experience and assume it's terrible and give up on it. I feel bad for people who haven't yet experienced how useful these models are for programming. Some also just prefer manually entering everything. Those people I will never understand.

panarky 2y ago
It doesn't matter how useful it is, there will always be naysayers who can't see it, or who refuse to see it. That's okay. It's not my responsibility to convince or convert them. I prefer to just let them be and not engage.
- gilbetron 2y ago
  I hear you, I'm tired of getting people that don't care to care. It's the people that should know how cool this stuff is and don't - they frustrate me!
  - jryan49 2y ago
    You people frustrate me because you don't listen that I've tried to use AI to do help with my job and it fails horribly in every way. I see that it is useful to you, and that's great, but that doesn't make it useful for everybody... I don't understand why you must have everyone agree with you, and that it's "tires" you out to hear other people's contracting opinions. It feels like a religion.
    - gilbetron 2y ago
      I mean, it is trivial to show that it can do things literally impossible even 5 years ago. And you don't acknowledge that fact, and that's what drives me crazy. It's like showing someone from 1980 a modern smart phone and them saying, yeah but it can't read my mind.
      - heavyset_go 2y ago
        It's more like showing someone from 1980 a modern smart phone you call a Portable Mind Reader and them saying, "yeah, but it can't read my mind."
        School-Cotton 2y ago
        Are people really this hung up on the term “AI”? Who cares? The fact that this is a shockingly useful piece of technology has nothing to do with what it’s called.
        matkoniecz 2y ago
        Because marketing keeps deeply overpromising things. Maybe hype is overly beneficial to them but if you promise me 1500 and I get 1100 then I will underwhelmed. And especially around LLM marketing hype is fairly extreme.
        School-Cotton 2y ago
        If you lived before the invention of cars, and if when they were invented, marketers all said "these will be able to fly soon" (which of course, we know now wouldn't have been true), you would be underwhelmed? You wouldn't think it was extremely transformative technology?
        arkh 2y ago
        Because the AI term makes people anthropomorphize those tools. They "hallucinate", they "know", they "think". They're just the result of matrix calculus on which your own pattern recognition capacities fool you into thinking there is intelligence there. There isn't. They don't hallucinate, their output is wrong. The worst example I've seen of anthropomorphism was the blog from a searcher working on adverse prompting. The tool spewing "help me" words made them think they were hurting a living organism https://www.lesswrong.com/posts/MnYnCFgT3hF6LJPwn/why-white-box-redteaming-makes-me-feel-weird-1 https://www.lesswrong.com/posts/MnYnCFgT3hF6LJPwn/why-white-... Speaking with AI proponents feels like speaking with cryptocurrencies proponents: the more you learn about how things work, the more you understand they don't and just live in lalaland.
    - handfuloflight 2y ago
      Have you considered you are using it incorrectly?
      - jryan49 2y ago
        Yes I have. I did some research and read some blog posts, and nothing really helped. Do you have any good resources I could look at maybe?
        handfuloflight 2y ago
        I think you would benefit from a personalized approach. If you like, send me a Loom or similar of you attempting to complete one software task with AI, that fails as you said, and I'll give you my feedback. Email in profile.
        ukuina 2y ago
        This is the correct approach. Also see: https://ezyang.github.io/ai-blindspots/ https://ezyang.github.io/ai-blindspots/
      - Xelynega 2y ago
        Have you considered that the problems you encounter in daily life just happen to be more present in the training data than problems other users encounter? Stitching together well-known web technologies and protocols in well-known patterns, probably a good success rate. Solving issues in legacy codebases using proprietary technologies and protocols, and non-standard patterns. Probably not such a good success rate.
        handfuloflight 2y ago
        Have you considered you have no knowledge of how I'm using AI?
- menzoic 2y ago
  Well said
tempestn 2y ago
Far from just programming too. They're useful for so many things. I use it for quickly coming up with shell scripts (or even complex piped commands (or if I'm being honest even simple commands since it's easier than skimming the man page)). But I also use it to bounce ideas off of when negotiating contracts. Or to give me a spoiler-free reminder of a plot point I'd forgotten in a book or TV series. Or to explain legal or taxation issues (which I of course verify, but it points me in the right direction). Or any number of other things. As the parent says, while far from perfect, they're an incredible aid in so many areas. When used well, they help you produce not just faster but also better results. The only trick really is that you need to treat it as a (very knowledgeable but overconfident) collaborator rather than an oracle.
01100011 2y ago
how much time do I need to devote to see anything but garbage? For reference, I program systems code in C/C++ in a large, proprietary codebase. My experiences with OpenAI(a year ago or more), and more recently, Cursor, Grok-v3 and Deepseek-r1, were all failures. The later two started out OK and got worse over time. What I haven't done is asked "AI" to whip up a more standard application. I have some ideas(an ncurses frontend to p4 written in python similar to tig, for instance), but haven't gotten around to it. I want this stuff to work, but so far it hasn't. Now I don't think "programming" a computer in english is a very good idea anyway, but I want a competent AI assistant to pair program with. To the degree that people are getting results, to me it seems they are leveraging very high-level APIs/libraries of code which are not written by AI and solving well-solved, "common" problems(simple games, simple web or phone apps). Sort of like how people gloss over the heavy lifting done by language itself when they praise the results from LLMs in other fields. I know it eventually will work. I just don't know when. I also get annoyed by the hype of folks who think they can become software engineers because they can talk to an LLM. Most of my job isn't programming. Most of my job is thinking about what the solution should be, talking to other people like me in meetings, understanding what customers really want beyond what they are saying, and tracking what I'm doing in various forms(which is something I really do want AI to help me with). Vibe coding is aptly named because it's sort of the VB6 of the modern era. Holy cow! I wrote a Windows GUI App!!!. It's letting non-programmers and semi-programmers(the "I write glue code in Python to munge data and API ins/outs" crowd) create usable things. Cool! So did spreadsheets. So did Hypercard. Andrej tweeting that he made a phone app was kinda cool but also kinda sad. If this is what the hundreds of billions spent on AI(and my bank account thanks you for that) delivers then the bubble is going to pop soon.
- sothatsit 2y ago
  I think there is a big problem of expectations. People are told that it is great for software development, so they try to use it on big existing software projects, and it sucks. Usually that's because of context: LLMs are not very good at understanding a very large amount of context, but if you don't give LLMs enough context, they can't magically figure it out on their own. This relegates AI to only really being useful for pretty self-contained examples where the amount of code is small, and you can provide all the context it needs to do its job in a relatively small amount of text (few thousand words or lines of code at most). That's why I think LLMs are only useful right now in real-world software development for things like one-off functions, new prototypes, writing small scripts, or automating lots of manual changes you have to do. For example, I love using o3-mini-high to take existing tests that I have and modifying them to make a new test case. Often this involves lots of tiny changes that are annoying to write, and o3-mini-high can make those changes pretty reliably. You just give it a TODO list of changes, and it goes ahead and does it. But I'm not asking these models how to implement a new feature in our codebase. I think this is why a lot of software developers have a bad view of AI. It's just not very good at the core software development work right now, but it's good enough at prototypes to make people freak out about how software development is going to be replaced. That's not to mention that often when people first try to use LLMs for coding, they don't give the LLMs enough context or instructions to do well. Sometimes I will spend 2-3 minutes writing a prompt, but I often see other people putting the bare minimum effort into it, and then being surprised when it doesn't work very well.
tombert 2y ago
I use ChatGPT all the time. I really like it. It's not perfect; how I've described it (and I doubt that I'm unique in this): it's like having a really smart and eager intern at your disposal. I say "intern" in the sense that its error-prone and kind of inexperienced, but also generally useful. I can ask it to automatically create a lot of the bootstrapping or tedious code that I always dread writing so that I can focus on the fun stuff, which is often the stuff that's pawned off onto interns and junior-level engineers. I think for the most part, when you treat it like that, it lives up to and sometimes even surpasses expectations. I mean, I can't speak for everyone, but whenever I begin a new project, a large percentage of the first ~3 hours is simply copying and pasting and editing from documentation, either an API I have to call or some bootstrapping code from a framework or just some cruft to make built-in libraries work how you want. I hate doing all that, it actively makes me not want to start a new project. Being able to get ChatGPT to give me stuff that I need to actually get started on my project has made coding a lot more fun for me again. At this point, you can take my LLM from my cold dead hands. I do think it will keep getting better, but I'm also at a point where even if it never improves I will still keep using it.
billy99k 2y ago
I love using it to boilerplate code for a new API I want to integrate. Much better than having to manually search. In the near future, not knowing how to effectively use AI to enhance productivity will be a disadvantage to potential employers.

panarky 2y ago

It doesn't matter how useful it is, there will always be naysayers who can't see it, or who refuse to see it. That's okay. It's not my responsibility to convince or convert them. I prefer to just let them be and not engage.

gilbetron 2y ago
I hear you, I'm tired of getting people that don't care to care. It's the people that should know how cool this stuff is and don't - they frustrate me!
- jryan49 2y ago
  You people frustrate me because you don't listen that I've tried to use AI to do help with my job and it fails horribly in every way. I see that it is useful to you, and that's great, but that doesn't make it useful for everybody... I don't understand why you must have everyone agree with you, and that it's "tires" you out to hear other people's contracting opinions. It feels like a religion.
  - gilbetron 2y ago
    I mean, it is trivial to show that it can do things literally impossible even 5 years ago. And you don't acknowledge that fact, and that's what drives me crazy. It's like showing someone from 1980 a modern smart phone and them saying, yeah but it can't read my mind.
    - heavyset_go 2y ago
      It's more like showing someone from 1980 a modern smart phone you call a Portable Mind Reader and them saying, "yeah, but it can't read my mind."
      - School-Cotton 2y ago
        Are people really this hung up on the term “AI”? Who cares? The fact that this is a shockingly useful piece of technology has nothing to do with what it’s called.
        matkoniecz 2y ago
        Because marketing keeps deeply overpromising things. Maybe hype is overly beneficial to them but if you promise me 1500 and I get 1100 then I will underwhelmed. And especially around LLM marketing hype is fairly extreme.
        School-Cotton 2y ago
        If you lived before the invention of cars, and if when they were invented, marketers all said "these will be able to fly soon" (which of course, we know now wouldn't have been true), you would be underwhelmed? You wouldn't think it was extremely transformative technology?
        arkh 2y ago
        Because the AI term makes people anthropomorphize those tools. They "hallucinate", they "know", they "think". They're just the result of matrix calculus on which your own pattern recognition capacities fool you into thinking there is intelligence there. There isn't. They don't hallucinate, their output is wrong. The worst example I've seen of anthropomorphism was the blog from a searcher working on adverse prompting. The tool spewing "help me" words made them think they were hurting a living organism https://www.lesswrong.com/posts/MnYnCFgT3hF6LJPwn/why-white-box-redteaming-makes-me-feel-weird-1 https://www.lesswrong.com/posts/MnYnCFgT3hF6LJPwn/why-white-... Speaking with AI proponents feels like speaking with cryptocurrencies proponents: the more you learn about how things work, the more you understand they don't and just live in lalaland.
  - handfuloflight 2y ago
    Have you considered you are using it incorrectly?
    - jryan49 2y ago
      Yes I have. I did some research and read some blog posts, and nothing really helped. Do you have any good resources I could look at maybe?
      - handfuloflight 2y ago
        I think you would benefit from a personalized approach. If you like, send me a Loom or similar of you attempting to complete one software task with AI, that fails as you said, and I'll give you my feedback. Email in profile.
        ukuina 2y ago
        This is the correct approach. Also see: https://ezyang.github.io/ai-blindspots/ https://ezyang.github.io/ai-blindspots/
    - Xelynega 2y ago
      Have you considered that the problems you encounter in daily life just happen to be more present in the training data than problems other users encounter? Stitching together well-known web technologies and protocols in well-known patterns, probably a good success rate. Solving issues in legacy codebases using proprietary technologies and protocols, and non-standard patterns. Probably not such a good success rate.
      - handfuloflight 2y ago
        Have you considered you have no knowledge of how I'm using AI?
menzoic 2y ago
Well said

tempestn 2y ago

Far from just programming too. They're useful for so many things. I use it for quickly coming up with shell scripts (or even complex piped commands (or if I'm being honest even simple commands since it's easier than skimming the man page)). But I also use it to bounce ideas off of when negotiating contracts. Or to give me a spoiler-free reminder of a plot point I'd forgotten in a book or TV series. Or to explain legal or taxation issues (which I of course verify, but it points me in the right direction). Or any number of other things. As the parent says, while far from perfect, they're an incredible aid in so many areas. When used well, they help you produce not just faster but also better results. The only trick really is that you need to treat it as a (very knowledgeable but overconfident) collaborator rather than an oracle.

01100011 2y ago

how much time do I need to devote to see anything but garbage? For reference, I program systems code in C/C++ in a large, proprietary codebase. My experiences with OpenAI(a year ago or more), and more recently, Cursor, Grok-v3 and Deepseek-r1, were all failures. The later two started out OK and got worse over time. What I haven't done is asked "AI" to whip up a more standard application. I have some ideas(an ncurses frontend to p4 written in python similar to tig, for instance), but haven't gotten around to it. I want this stuff to work, but so far it hasn't. Now I don't think "programming" a computer in english is a very good idea anyway, but I want a competent AI assistant to pair program with. To the degree that people are getting results, to me it seems they are leveraging very high-level APIs/libraries of code which are not written by AI and solving well-solved, "common" problems(simple games, simple web or phone apps). Sort of like how people gloss over the heavy lifting done by language itself when they praise the results from LLMs in other fields. I know it eventually will work. I just don't know when. I also get annoyed by the hype of folks who think they can become software engineers because they can talk to an LLM. Most of my job isn't programming. Most of my job is thinking about what the solution should be, talking to other people like me in meetings, understanding what customers really want beyond what they are saying, and tracking what I'm doing in various forms(which is something I really do want AI to help me with). Vibe coding is aptly named because it's sort of the VB6 of the modern era. Holy cow! I wrote a Windows GUI App!!!. It's letting non-programmers and semi-programmers(the "I write glue code in Python to munge data and API ins/outs" crowd) create usable things. Cool! So did spreadsheets. So did Hypercard. Andrej tweeting that he made a phone app was kinda cool but also kinda sad. If this is what the hundreds of billions spent on AI(and my bank account thanks you for that) delivers then the bubble is going to pop soon.

sothatsit 2y ago
I think there is a big problem of expectations. People are told that it is great for software development, so they try to use it on big existing software projects, and it sucks. Usually that's because of context: LLMs are not very good at understanding a very large amount of context, but if you don't give LLMs enough context, they can't magically figure it out on their own. This relegates AI to only really being useful for pretty self-contained examples where the amount of code is small, and you can provide all the context it needs to do its job in a relatively small amount of text (few thousand words or lines of code at most). That's why I think LLMs are only useful right now in real-world software development for things like one-off functions, new prototypes, writing small scripts, or automating lots of manual changes you have to do. For example, I love using o3-mini-high to take existing tests that I have and modifying them to make a new test case. Often this involves lots of tiny changes that are annoying to write, and o3-mini-high can make those changes pretty reliably. You just give it a TODO list of changes, and it goes ahead and does it. But I'm not asking these models how to implement a new feature in our codebase. I think this is why a lot of software developers have a bad view of AI. It's just not very good at the core software development work right now, but it's good enough at prototypes to make people freak out about how software development is going to be replaced. That's not to mention that often when people first try to use LLMs for coding, they don't give the LLMs enough context or instructions to do well. Sometimes I will spend 2-3 minutes writing a prompt, but I often see other people putting the bare minimum effort into it, and then being surprised when it doesn't work very well.

tombert 2y ago

I use ChatGPT all the time. I really like it. It's not perfect; how I've described it (and I doubt that I'm unique in this): it's like having a really smart and eager intern at your disposal. I say "intern" in the sense that its error-prone and kind of inexperienced, but also generally useful. I can ask it to automatically create a lot of the bootstrapping or tedious code that I always dread writing so that I can focus on the fun stuff, which is often the stuff that's pawned off onto interns and junior-level engineers. I think for the most part, when you treat it like that, it lives up to and sometimes even surpasses expectations. I mean, I can't speak for everyone, but whenever I begin a new project, a large percentage of the first ~3 hours is simply copying and pasting and editing from documentation, either an API I have to call or some bootstrapping code from a framework or just some cruft to make built-in libraries work how you want. I hate doing all that, it actively makes me not want to start a new project. Being able to get ChatGPT to give me stuff that I need to actually get started on my project has made coding a lot more fun for me again. At this point, you can take my LLM from my cold dead hands. I do think it will keep getting better, but I'm also at a point where even if it never improves I will still keep using it.

billy99k 2y ago

I love using it to boilerplate code for a new API I want to integrate. Much better than having to manually search. In the near future, not knowing how to effectively use AI to enhance productivity will be a disadvantage to potential employers.