5 ms·

I see so many people say that llms have improved their programming and i am truly baffled by how. They are useful on occasion and i do use chatgpt and copilot d

I see so many people say that llms have improved their programming and i am truly baffled by how. They are useful on occasion and i do use chatgpt and copilot daily but if they disappeared today it would be at worst a minor annoyance.

dontreact 3y ago
It’s a tool you learn to use and improve with. How many hours of practice have you given the tool? Personally I’ve been practicing for almost 3 years now going back to CoPilot so I would guess I have at very minimum a few hundred hours of thinking about how (what to prompt for, what to check the output for) and probably more importantly when (how to decompose the task I am doing into parts for which some the LLM can reliably execute) to use the tool.
- stavros 3y ago
  I don't think it's a failure of the prompting, but of the tool itself. It just isn't good for any higher-level tasks, and those I know how to do anyway. It's good for menial API work, as a sibling comment says, and using it I can avoid reading API docs, but that's about it. I tried to have it rewrite in Go a small Python script (a hundred or so lines) that I wrote, and it basically refused, saying "here's the start, you write the rest now". As others said, if it went away it'd be a minor inconvenience of me having to read a bit more documentation.
  - dontreact 3y ago
    Yes to me that’s obviously not a good use, and I have experience to know not to attempt hundred line rewrites of files. I order to reflect on my mental about this. It’s that i more often break down parts problem into sub routines that are 30 or so lines. These are nailed about 95% of the time and when there’s something wrong it’s obvious.
    - dontreact 3y ago
      I also wonder if it’s a language effect and the models are just way better at python
bun_terminator 3y ago
Same here. The most use I made of it were 1) making it write some basic code for an API I don't know. Some windows API calls in particular 2) Abusing it as a search engine since google barely qualifies for one anymore For actual refactoring, the usual tools still blow it out of the water IMO. Same for quality code. It just doesn't compile half the time.
Balgair 3y ago
I mean, I'm not so much of a programmer as a person that can hack together code. Granted, my job is not 'programmer', it's 'R&D'. For example, I had to patch together some excel files the other day. Data in one file referenced data in ~60 other files in a pretty complicated way, but a standardized one. About 80,000 rows needed to be processed across 60,000 columns. Not terrible, not fun though. Now, I'm good at excel, but not VBA good. And I'm a 'D-' in python. Passing, but barely. Writing a python program to do the job would take me about a full work day, likely two. But, when I fired up GPT3.5 (I'm a schlub, I know!) and typed in the request, I got a pretty good working answer in about, ohhh, 15 seconds. Sure, it didn't work the first time, but I was able to hack at it, iterate with GPT3.5 only twice, and I got my work done just fine. Total time? About 15 minutes. Likely a 64X increase in productivity. So, maybe when we say that our programming is improved, what we actually mean is that our productivity is improved, not our innate abilities to program.
mattkevan 3y ago
I’m a designer who likes to code things for fun and would make a terrible programmer. For me llms are fantastic as they enable me to build things I never could have without. I imagine they wouldn’t be as useful for a skilled programmer as they can do it all already. Can’t remember the source, but I read a paper a while back that looked at how much ChatGPT actually helped people in their work. For above average workers it didn’t make much difference, but it made a big improvement for below average workers, bringing them up to the level of their more experienced peers.
shriek 3y ago
Yeah, I'm curious how others are using too. For boilerplate code and configs, they're great in a sense that I don't have to open docs for reference but other than that I feel like maybe I'm not fully using it to its full potential like others are mentioning.

dontreact 3y ago

It’s a tool you learn to use and improve with. How many hours of practice have you given the tool? Personally I’ve been practicing for almost 3 years now going back to CoPilot so I would guess I have at very minimum a few hundred hours of thinking about how (what to prompt for, what to check the output for) and probably more importantly when (how to decompose the task I am doing into parts for which some the LLM can reliably execute) to use the tool.

stavros 3y ago
I don't think it's a failure of the prompting, but of the tool itself. It just isn't good for any higher-level tasks, and those I know how to do anyway. It's good for menial API work, as a sibling comment says, and using it I can avoid reading API docs, but that's about it. I tried to have it rewrite in Go a small Python script (a hundred or so lines) that I wrote, and it basically refused, saying "here's the start, you write the rest now". As others said, if it went away it'd be a minor inconvenience of me having to read a bit more documentation.
- dontreact 3y ago
  Yes to me that’s obviously not a good use, and I have experience to know not to attempt hundred line rewrites of files. I order to reflect on my mental about this. It’s that i more often break down parts problem into sub routines that are 30 or so lines. These are nailed about 95% of the time and when there’s something wrong it’s obvious.
  - dontreact 3y ago
    I also wonder if it’s a language effect and the models are just way better at python

bun_terminator 3y ago

Same here. The most use I made of it were 1) making it write some basic code for an API I don't know. Some windows API calls in particular 2) Abusing it as a search engine since google barely qualifies for one anymore For actual refactoring, the usual tools still blow it out of the water IMO. Same for quality code. It just doesn't compile half the time.

Balgair 3y ago

I mean, I'm not so much of a programmer as a person that can hack together code. Granted, my job is not 'programmer', it's 'R&D'. For example, I had to patch together some excel files the other day. Data in one file referenced data in ~60 other files in a pretty complicated way, but a standardized one. About 80,000 rows needed to be processed across 60,000 columns. Not terrible, not fun though. Now, I'm good at excel, but not VBA good. And I'm a 'D-' in python. Passing, but barely. Writing a python program to do the job would take me about a full work day, likely two. But, when I fired up GPT3.5 (I'm a schlub, I know!) and typed in the request, I got a pretty good working answer in about, ohhh, 15 seconds. Sure, it didn't work the first time, but I was able to hack at it, iterate with GPT3.5 only twice, and I got my work done just fine. Total time? About 15 minutes. Likely a 64X increase in productivity. So, maybe when we say that our programming is improved, what we actually mean is that our productivity is improved, not our innate abilities to program.

mattkevan 3y ago

I’m a designer who likes to code things for fun and would make a terrible programmer. For me llms are fantastic as they enable me to build things I never could have without. I imagine they wouldn’t be as useful for a skilled programmer as they can do it all already. Can’t remember the source, but I read a paper a while back that looked at how much ChatGPT actually helped people in their work. For above average workers it didn’t make much difference, but it made a big improvement for below average workers, bringing them up to the level of their more experienced peers.

shriek 3y ago

Yeah, I'm curious how others are using too. For boilerplate code and configs, they're great in a sense that I don't have to open docs for reference but other than that I feel like maybe I'm not fully using it to its full potential like others are mentioning.