5 ms·

> I did a lot of AI assisted coding this week Are you new to it? There's a pretty standard arc that starts with how great it is and ends with all the "giving u

by andy99 1y ago

> I did a lot of AI assisted coding this week Are you new to it? There's a pretty standard arc that starts with how great it is and ends with all the "giving up on AI" blog posts you see. I went through it to. I still use a chatbot as a better stack overflow, but I've stopped actually having AI write any code I use - it's not just the quality, it's the impact on my thinking and understanding that ultimately doesn't improve outcomes over just doing it myself.

PaulHoule 1y ago
I've been doing it for a while. I never really liked stack overflow though it always seemed like a waste of time versus learning how to look up the real answers in the documentation. I never really liked agents because they go off for 20 minutes and come back with complete crap But if I can ask a question get an answer in 20 seconds and iterate again I find that's pretty efficient. I’ve usually been skeptical about people who get unreasonably good results and not surprised when they wake up a few weeks later and are disappointed. One area where I am consistently disappointed is when there are significant changes across versions: I had one argue over where I could write a switch in Java that catches null (you can in JDK 21) and lots of trouble with SQLAlchemy in Python which changed a lot between versions. I shudder to think what would happen if you asked questions about react-router but actually I shudder to think about react-router at all.
resonious 1y ago
I've been back and forth, and currently heavily relying on AI-written code. It all depends on knowing what the AI can and can't do ahead of time. And what it can do often overlaps with grunt work that I don't enjoy doing.
whynotminot 1y ago
When's the last time you "went through the loop" ? I feel like with this stuff I have to update my priors about every three or four months. I've been using AI regularly since GPT 4 first came out a couple years ago. Over that time, various models from Sonnet to Gemini to 4o have generally been good rubber ducks. Good to talk to and discuss approaches and tradeoffs, and better in general than google + stack overflow + pouring over verbose documentation. But I couldn't really "hand the models the wheel." They weren't trustworthy enough, easily lost the plot, failed to leverage important context right in front of them in the codebase, etc. You could see that there was potential there, but it felt pretty far away. Something changed this spring. Gemini 2.5 Pro, Claude 4 models, o3 and o4-mini -- I'm starting to give the models the wheel now. They're good. They understand context. They understand the style of the codebase. And they of course bring the immense knowledge they've always had. It's eerie to see, and to think about what comes with the next wave of models coming very soon. And if the last time you really gave model-driven programming a go was 6 months or more ago, you probably have no idea what's about to happen.
- andy99 1y ago
  Interesting point, I agree that things change so fast that experience from a few months ago is out of date. I'm sceptical there has been a real step change (especially based on the snippets I see claude 4 writing in answer to questions) but it never hurts to try again. My most recent stab at this was Claude code with 3.7, circa March this year. To be fair though, a big part of the issue for me is that having not done the work or properly thought through how a project is structured and how the code works, it comes back to bite later. A better model doesn't change this.
  - whynotminot 1y ago
    If you give it another try, my goto right now is Sonnet 4 Thinking. There's a pretty massive difference in intelligence by switching from just plain 4 to 4 Thinking. It's still pretty fast, and I think hits the right balance between speed and useful intelligence. However, at least in my experience, nothing beats o3 for raw intelligence. It's a little too slow to use as the daily driver though. It's kind of fun seeing the models all have their various pros and cons. > To be fair though, a big part of the issue for me is that having not done the work or properly thought through how a project is structured and how the code works, it comes back to bite later. A better model doesn't change this. Yes, even as I start to leverage the tools more, I try to double down on my own understanding of the problem being solved, at least at a high level. Need to make sure you don't lose the plot yourself.
  - theptip 1y ago
    There has been a big change with Claude 4.0 in my opinion. Probably depends on your environment, but it’s the first time I’ve been able to get hundreds of lines of Python that just works when vibe coding a new project. It’s still slower going as the codebase increases in size, and this is my hypothesis for the huge variance; I was getting giddy at how fast I blew through the first 5 hours of a small project (perhaps in 30 mins with Claude) but quickly lost velocity when I started implementing tests and editing existing code.
- ryandrake 1y ago
  Just one person's opinion: I can't get into the mode of programming where you "chat" with something and have it build the code. By the time I have visualized in my head and articulated into English what I want to build and the data structures and algorithms I need, I might as well just type the code in myself. That's the only value I've found from AI: It's a great autocomplete as you're typing. To me, programming is a solo activity. "Chatting" with someone or something as I do it is just a distraction.
positron26 1y ago
> There's a pretty standard arc that starts with how great it is and ends with all the "giving up on AI" blog posts you see. Wouldn't be shocked if this is related to getting ramped up. Switch to a language you don't know. How will that affect your AI usage? Will it go up or down over time? Had similar experiences during the SO early days or when diving into open source projects. Suddenly you go from being stuck in your bubble to learning all kinds of things. Your code gets good. The returns diminish and you no longer curiously read unless the library is doing something you can't imagine.
benreesman 1y ago
I'm hanging in there with it. The article was remarkably dispassionate about something that gets everyone all hot around here: easy stuff benefits from it a lot more than hard stuff. If your whole world is Node.js? Yeah, not a lot of deep water energy discovery geology data mining at exabyte scale or whatever. It pushed the barrier to entry on making a website with a Supabase backend from whatever it was to nearly zero. And I want to be really clear that frontend work is some of my favorite work: I love doing the web interface, it's visual and interactive and there's an immediacy that's addictive, the compiler toolchains are super cool, it's great stuff. I worked on a web browser, got some patents for it. I like this stuff. But getting extreme concurrency outcomes or exotic hardware outcomes or low latency stuff or heavy numerics or a million other things? It's harder, the bots aren't as good. This is the divide: it can one shot a web page, even the older ones can just smoke a tailwind theme, headshot. A kernel patch to speed up a path on my bare metal box in a latency contest? Nah, not really. But I see a lot of promise in the technology even if the sort of hype narrative around it seems pretty much false at this point: I still use the LLMs a lot. Part of that is that search just doesn't work anymore (although... Yandex got a lot better recently out of the blue of all things), and part of it is that I see enough exciting glimpses of like, if I got it hooked up to the right programming language and loaded the context right, wow, once in a while it just slams and it's kinda rare but frequent enough that I'm really interested in figuring out how to reproduce it reliably. And I think I'm getting better outcomes a little bit at a time, getting it dialed in. Two or three months ago an even with claude code would have me yelling at the monitor, now it's like, haha, i see you.

by andy99 1y ago

PaulHoule 1y ago

I've been doing it for a while. I never really liked stack overflow though it always seemed like a waste of time versus learning how to look up the real answers in the documentation. I never really liked agents because they go off for 20 minutes and come back with complete crap But if I can ask a question get an answer in 20 seconds and iterate again I find that's pretty efficient. I’ve usually been skeptical about people who get unreasonably good results and not surprised when they wake up a few weeks later and are disappointed. One area where I am consistently disappointed is when there are significant changes across versions: I had one argue over where I could write a switch in Java that catches null (you can in JDK 21) and lots of trouble with SQLAlchemy in Python which changed a lot between versions. I shudder to think what would happen if you asked questions about react-router but actually I shudder to think about react-router at all.

resonious 1y ago

I've been back and forth, and currently heavily relying on AI-written code. It all depends on knowing what the AI can and can't do ahead of time. And what it can do often overlaps with grunt work that I don't enjoy doing.

whynotminot 1y ago

When's the last time you "went through the loop" ? I feel like with this stuff I have to update my priors about every three or four months. I've been using AI regularly since GPT 4 first came out a couple years ago. Over that time, various models from Sonnet to Gemini to 4o have generally been good rubber ducks. Good to talk to and discuss approaches and tradeoffs, and better in general than google + stack overflow + pouring over verbose documentation. But I couldn't really "hand the models the wheel." They weren't trustworthy enough, easily lost the plot, failed to leverage important context right in front of them in the codebase, etc. You could see that there was potential there, but it felt pretty far away. Something changed this spring. Gemini 2.5 Pro, Claude 4 models, o3 and o4-mini -- I'm starting to give the models the wheel now. They're good. They understand context. They understand the style of the codebase. And they of course bring the immense knowledge they've always had. It's eerie to see, and to think about what comes with the next wave of models coming very soon. And if the last time you really gave model-driven programming a go was 6 months or more ago, you probably have no idea what's about to happen.

andy99 1y ago
Interesting point, I agree that things change so fast that experience from a few months ago is out of date. I'm sceptical there has been a real step change (especially based on the snippets I see claude 4 writing in answer to questions) but it never hurts to try again. My most recent stab at this was Claude code with 3.7, circa March this year. To be fair though, a big part of the issue for me is that having not done the work or properly thought through how a project is structured and how the code works, it comes back to bite later. A better model doesn't change this.
- whynotminot 1y ago
  If you give it another try, my goto right now is Sonnet 4 Thinking. There's a pretty massive difference in intelligence by switching from just plain 4 to 4 Thinking. It's still pretty fast, and I think hits the right balance between speed and useful intelligence. However, at least in my experience, nothing beats o3 for raw intelligence. It's a little too slow to use as the daily driver though. It's kind of fun seeing the models all have their various pros and cons. > To be fair though, a big part of the issue for me is that having not done the work or properly thought through how a project is structured and how the code works, it comes back to bite later. A better model doesn't change this. Yes, even as I start to leverage the tools more, I try to double down on my own understanding of the problem being solved, at least at a high level. Need to make sure you don't lose the plot yourself.
- theptip 1y ago
  There has been a big change with Claude 4.0 in my opinion. Probably depends on your environment, but it’s the first time I’ve been able to get hundreds of lines of Python that just works when vibe coding a new project. It’s still slower going as the codebase increases in size, and this is my hypothesis for the huge variance; I was getting giddy at how fast I blew through the first 5 hours of a small project (perhaps in 30 mins with Claude) but quickly lost velocity when I started implementing tests and editing existing code.
ryandrake 1y ago
Just one person's opinion: I can't get into the mode of programming where you "chat" with something and have it build the code. By the time I have visualized in my head and articulated into English what I want to build and the data structures and algorithms I need, I might as well just type the code in myself. That's the only value I've found from AI: It's a great autocomplete as you're typing. To me, programming is a solo activity. "Chatting" with someone or something as I do it is just a distraction.

positron26 1y ago

> There's a pretty standard arc that starts with how great it is and ends with all the "giving up on AI" blog posts you see. Wouldn't be shocked if this is related to getting ramped up. Switch to a language you don't know. How will that affect your AI usage? Will it go up or down over time? Had similar experiences during the SO early days or when diving into open source projects. Suddenly you go from being stuck in your bubble to learning all kinds of things. Your code gets good. The returns diminish and you no longer curiously read unless the library is doing something you can't imagine.

benreesman 1y ago

I'm hanging in there with it. The article was remarkably dispassionate about something that gets everyone all hot around here: easy stuff benefits from it a lot more than hard stuff. If your whole world is Node.js? Yeah, not a lot of deep water energy discovery geology data mining at exabyte scale or whatever. It pushed the barrier to entry on making a website with a Supabase backend from whatever it was to nearly zero. And I want to be really clear that frontend work is some of my favorite work: I love doing the web interface, it's visual and interactive and there's an immediacy that's addictive, the compiler toolchains are super cool, it's great stuff. I worked on a web browser, got some patents for it. I like this stuff. But getting extreme concurrency outcomes or exotic hardware outcomes or low latency stuff or heavy numerics or a million other things? It's harder, the bots aren't as good. This is the divide: it can one shot a web page, even the older ones can just smoke a tailwind theme, headshot. A kernel patch to speed up a path on my bare metal box in a latency contest? Nah, not really. But I see a lot of promise in the technology even if the sort of hype narrative around it seems pretty much false at this point: I still use the LLMs a lot. Part of that is that search just doesn't work anymore (although... Yandex got a lot better recently out of the blue of all things), and part of it is that I see enough exciting glimpses of like, if I got it hooked up to the right programming language and loaded the context right, wow, once in a while it just slams and it's kinda rare but frequent enough that I'm really interested in figuring out how to reproduce it reliably. And I think I'm getting better outcomes a little bit at a time, getting it dialed in. Two or three months ago an even with claude code would have me yelling at the monitor, now it's like, haha, i see you.