3 ms·

Indeed. I started using Sonnet for coding only about a month ago. It's been great in that I've finally written scripts I had floating around my brain for years,

Indeed. I started using Sonnet for coding only about a month ago. It's been great in that I've finally written scripts I had floating around my brain for years, and very rapidly. But the feeling of skill atrophy is very real. The other day I needed to write a function that recursively traverses a Python dictionary, making modifications along the way. It's the perfect task to give an LLM. I can easily see that if I always give this task to an LLM, a few years down the road I'll be really slow in writing it on my own, and will fail most coding interviews. Also, while there is a high in producing a lot in a short amount of time, there is no feeling of satisfaction that you get from programming. And debugging its bugs is as much fun as debugging your coworkers' bugs - except at least with the coworkers you can have a more intelligent conversation on what they were trying to do.

hosh 2y ago
This is exactly why I still play Go, practice martial arts, archery and use the command line for my dev workflow. Those are all arguably less efficient and obsolete. The AlphaGo series can defeat the strongest human Go players and firearms are more effective than unarmed martial arts and archery. GUI is easier for most people than the command line. Yet, I practice all these to develop my mind and body. I don't have to be a world-class Go player to benefit from learning to play Go. This happens a lot in the natural world in ecosystems. For example, many people plant trees and add a drip system. The trees grow to depend on the drip system, and never stretch and develop their roots -- and the relationship they have with the soil microbiome. They make the trees prone to get knocked down when an unusually strong gust of wind come through.
- MrMcCall 2y ago
  You might have my favorite profile page here, with the exception of DonHopkins. Just beautiful concepts, inspiring, and well-represented in this comment. Peace be with you, my friend.
- BeetleB 2y ago
  > This is exactly why I still play Go, practice martial arts, archery and use the command line for my dev workflow. And this is what worries me. Pre-LLM, I'd get all my practice done during work hours. I fear that with LLMs, I'll need to do all this practice on my free time, effectively reducing my truly "free" time.
  - alephnerd 2y ago
    > Pre-LLM, I'd get all my practice done during work hours. I fear that with LLMs, I'll need to do all this practice on my free time If you can attach a valid business reason/excuse then it's fairly easy to get time dedicated to experimentation and learning. I haven't been a coder for several years now, but I still try to justify a personal lab environment in order to understand the market and customer problems. If you're an Engineer or EM, I think it would be even easier (attach experimentation to some sort of critical engineering initiative). Even with Code Gen Copilots becoming a thing, between the lines most of us in engineering and business leadership recognize it cannot replace architecture or design. If it can, then you've solved the Turing or Chinese Room problem and that's a whole other story. That said, if your day job isn't only "write this terraform/python script" you shouldn't be at risk.
    - BeetleB 2y ago
      > If you can attach a valid business reason/excuse then it's fairly easy to get time dedicated to experimentation and learning. This has the same problem as "doing it in free time". I have a finite number of goodie points with management, and I'd be using some when I request this - which means I have fewer things I can request from them in the future. > That said, if your day job isn't only "write this terraform/python script" you shouldn't be at risk. I'm not worried about the present, or even the near future. The reason I have other valuable skills is because of the years I spent on writing code. Eventually, I'll stop growing if I rely too much on LLMs. I may have to rely on them if everyone else is and being more productive.
    - CamperBob2 2y ago
      The Turing test can be considered well and truly passed at this point, partially because it has always said more about the human taking the test than it did about the machine. As for the Chinese Room, it was never anything but a pointless exercise in question-begging. If anyone ever considered it relevant, the advent of LLMs should have immediately convinced them otherwise. Searle knew nothing about embedding vectors, never mind attention. Great points otherwise, though. A strong focus on identifying, understanding, and solving customer problems is, in modern parlance, all you need.
MoonGhost 2y ago
Coding is actually a hard skill which requires practice. Regular litcoding for the sake of it should help. The problem with that is it takes the whole brain and breaks other thoughts chain. I'm thinking about to dedicate full days for small tasks like this.
- BeetleB 2y ago
  > Regular litcoding for the sake of it should help. I got where I am without regular leetcoding. For me, regular leetcoding will only marginally improve my skills. And I definitely don't want to do regular leetcoding just to maintain my skills. You know how many people love jogging outdoors but hate treadmills? Leetcoding is like treadmills. You may have to do it, but it sucks. Yes, some people love treadmills, but they're in the minority.
  - QuadmasterXLII 2y ago
    There’s two kinds of leetcoding. Leetcoding via memorization obviously sucks, but leetcoding by clicking next problem until you get one thats totally alien and then letting it mull in the back of the head for a few days to crack it without reference material is pretty fun
    - mechagodzilla 2y ago
      After being a professional programmer for ~20 years, and recently playing around with leetcode - my main issue with leetcode is that there's almost no overlap between leetcode problems and the problems I actually encounter in the wild. The validation tests often have silly corner cases that force you into a single answer to avoid timing out. It's frequently as much work to understand what the problem is actually asking you as it is to implement a solution. Just like I've found ChatGPT to be pretty mediocre at writing the sort of code I work on, but others swear by it, maybe some peoples' dayjob actually looks like writing leetcode all day? I know a lot of interviewers use it, but it feels so disconnected from actual engineering work.
      - 2y ago
medhir 2y ago
the flip side to that “high” that comes with working super quickly for me has been a crash associated with a sinking feeling that I’ve outsourced too much.
CamperBob2 2y ago
I started using a C compiler for coding about 30 years ago. It's been great, but the feeling of skill atrophy is very real. I probably couldn't write useful code in x86 assembly any more, at least not without refreshing my memory first. And you know what? That's just peachy keen. I don't need to write x86 assembly anymore. In fact, these days I do a lot of coding for ARM platforms, but I never learned ARM assembly at all. So it would take more than just a refresher course to bring me up to speed in that. I don't anticipate any such need, fortunately. So... if I still need to write C in 10 years, why in the world would I consider that a good thing? Things in this business are supposed to progress toward higher and higher levels of abstraction.
- BeetleB 2y ago
  I have extremely high faith that the C compiler will produce very reliable x86 assembly code. I am extremely pessimistic that LLMs will ever reach that level of reliability. Or even close. They are a great helper, but that's all they'll be.
  - CamperBob2 2y ago
    What are some examples of prompts and responses that have made you pessimistic about the technology's ability to improve? Reliability has gotten insanely better over the past couple of years, and it's still improving.
    - BeetleB 2y ago
      > What are some examples of prompts and responses that have made you pessimistic about the technology's ability to improve? I'm not saying it won't improve. I think its lack of determinism puts a natural upper bound on reliability - something we don't have to worry about with compilers. As for examples: Oh wow. As much as I love and use them for coding, almost every project I work on has cases of coming up with complex solutions to a task that do not work, where 1-2 lines of (simple!) code would have solved the problem. It's not just that it fails, but it kept failing even after I told it how to do it right. Sometimes it does a great job and solves a hard problem for me. Other times I lose a lot of time getting it to do beginner level stuff. It has huge gaps/blind spots.

hosh 2y ago

This is exactly why I still play Go, practice martial arts, archery and use the command line for my dev workflow. Those are all arguably less efficient and obsolete. The AlphaGo series can defeat the strongest human Go players and firearms are more effective than unarmed martial arts and archery. GUI is easier for most people than the command line. Yet, I practice all these to develop my mind and body. I don't have to be a world-class Go player to benefit from learning to play Go. This happens a lot in the natural world in ecosystems. For example, many people plant trees and add a drip system. The trees grow to depend on the drip system, and never stretch and develop their roots -- and the relationship they have with the soil microbiome. They make the trees prone to get knocked down when an unusually strong gust of wind come through.

MrMcCall 2y ago
You might have my favorite profile page here, with the exception of DonHopkins. Just beautiful concepts, inspiring, and well-represented in this comment. Peace be with you, my friend.
BeetleB 2y ago
> This is exactly why I still play Go, practice martial arts, archery and use the command line for my dev workflow. And this is what worries me. Pre-LLM, I'd get all my practice done during work hours. I fear that with LLMs, I'll need to do all this practice on my free time, effectively reducing my truly "free" time.
- alephnerd 2y ago
  > Pre-LLM, I'd get all my practice done during work hours. I fear that with LLMs, I'll need to do all this practice on my free time If you can attach a valid business reason/excuse then it's fairly easy to get time dedicated to experimentation and learning. I haven't been a coder for several years now, but I still try to justify a personal lab environment in order to understand the market and customer problems. If you're an Engineer or EM, I think it would be even easier (attach experimentation to some sort of critical engineering initiative). Even with Code Gen Copilots becoming a thing, between the lines most of us in engineering and business leadership recognize it cannot replace architecture or design. If it can, then you've solved the Turing or Chinese Room problem and that's a whole other story. That said, if your day job isn't only "write this terraform/python script" you shouldn't be at risk.
  - BeetleB 2y ago
    > If you can attach a valid business reason/excuse then it's fairly easy to get time dedicated to experimentation and learning. This has the same problem as "doing it in free time". I have a finite number of goodie points with management, and I'd be using some when I request this - which means I have fewer things I can request from them in the future. > That said, if your day job isn't only "write this terraform/python script" you shouldn't be at risk. I'm not worried about the present, or even the near future. The reason I have other valuable skills is because of the years I spent on writing code. Eventually, I'll stop growing if I rely too much on LLMs. I may have to rely on them if everyone else is and being more productive.
  - CamperBob2 2y ago
    The Turing test can be considered well and truly passed at this point, partially because it has always said more about the human taking the test than it did about the machine. As for the Chinese Room, it was never anything but a pointless exercise in question-begging. If anyone ever considered it relevant, the advent of LLMs should have immediately convinced them otherwise. Searle knew nothing about embedding vectors, never mind attention. Great points otherwise, though. A strong focus on identifying, understanding, and solving customer problems is, in modern parlance, all you need.

MoonGhost 2y ago

Coding is actually a hard skill which requires practice. Regular litcoding for the sake of it should help. The problem with that is it takes the whole brain and breaks other thoughts chain. I'm thinking about to dedicate full days for small tasks like this.

BeetleB 2y ago
> Regular litcoding for the sake of it should help. I got where I am without regular leetcoding. For me, regular leetcoding will only marginally improve my skills. And I definitely don't want to do regular leetcoding just to maintain my skills. You know how many people love jogging outdoors but hate treadmills? Leetcoding is like treadmills. You may have to do it, but it sucks. Yes, some people love treadmills, but they're in the minority.
- QuadmasterXLII 2y ago
  There’s two kinds of leetcoding. Leetcoding via memorization obviously sucks, but leetcoding by clicking next problem until you get one thats totally alien and then letting it mull in the back of the head for a few days to crack it without reference material is pretty fun
  - mechagodzilla 2y ago
    After being a professional programmer for ~20 years, and recently playing around with leetcode - my main issue with leetcode is that there's almost no overlap between leetcode problems and the problems I actually encounter in the wild. The validation tests often have silly corner cases that force you into a single answer to avoid timing out. It's frequently as much work to understand what the problem is actually asking you as it is to implement a solution. Just like I've found ChatGPT to be pretty mediocre at writing the sort of code I work on, but others swear by it, maybe some peoples' dayjob actually looks like writing leetcode all day? I know a lot of interviewers use it, but it feels so disconnected from actual engineering work.
    - 2y ago

medhir 2y ago

the flip side to that “high” that comes with working super quickly for me has been a crash associated with a sinking feeling that I’ve outsourced too much.

CamperBob2 2y ago

I started using a C compiler for coding about 30 years ago. It's been great, but the feeling of skill atrophy is very real. I probably couldn't write useful code in x86 assembly any more, at least not without refreshing my memory first. And you know what? That's just peachy keen. I don't need to write x86 assembly anymore. In fact, these days I do a lot of coding for ARM platforms, but I never learned ARM assembly at all. So it would take more than just a refresher course to bring me up to speed in that. I don't anticipate any such need, fortunately. So... if I still need to write C in 10 years, why in the world would I consider that a good thing? Things in this business are supposed to progress toward higher and higher levels of abstraction.

BeetleB 2y ago
I have extremely high faith that the C compiler will produce very reliable x86 assembly code. I am extremely pessimistic that LLMs will ever reach that level of reliability. Or even close. They are a great helper, but that's all they'll be.
- CamperBob2 2y ago
  What are some examples of prompts and responses that have made you pessimistic about the technology's ability to improve? Reliability has gotten insanely better over the past couple of years, and it's still improving.
  - BeetleB 2y ago
    > What are some examples of prompts and responses that have made you pessimistic about the technology's ability to improve? I'm not saying it won't improve. I think its lack of determinism puts a natural upper bound on reliability - something we don't have to worry about with compilers. As for examples: Oh wow. As much as I love and use them for coding, almost every project I work on has cases of coming up with complex solutions to a task that do not work, where 1-2 lines of (simple!) code would have solved the problem. It's not just that it fails, but it kept failing even after I told it how to do it right. Sometimes it does a great job and solves a hard problem for me. Other times I lose a lot of time getting it to do beginner level stuff. It has huge gaps/blind spots.