7 ms·

Salient point: > Would I have been able to do it without ChatGPT? Certainly yes, but the most interesting thing is not the fact that it would have taken me lon

Salient point: > Would I have been able to do it without ChatGPT? Certainly yes, but the most interesting thing is not the fact that it would have taken me longer: the truth is that I wouldn't even have tried, because it wouldn't have been worth it. This is the true enabling power of LLMs for code assistance -- reducing the activation energy of new tasks enough that they are tackled (and finished) when they otherwise would have been left on the pile of future projects indefinitely. I think the internet and the open source movement had a similar effect, in that if you did not attempt a project that you had some small interest in, it would only be a matter of time before someone else did enough of a similar problem for you to reuse or repurpose their work, and this led to an explosion of (often useful, or at least usable) applications and libraries. I agree with the author that LLMs are not by themselves very capable but provide a force multiplier for those with the basic skills and motivation.

somewhereoutth 3y ago
However could it be that the 'entertainment effect' of using a new (and trendy) technology like LLMs provides the activation energy for otherwise mundane tasks?
- apwell23 3y ago
  This is true. I write way more documentation now since llm does all the formatting, structure , diagrams ect. I just guide it at a high level.
  - robluxus 3y ago
    Do you use a specialized LLM for diagrams?
    - apwell23 3y ago
      I just ask chatgpt to generate text diagrams that i can embed in my markdown.
    - danielbln 3y ago
      I'm not OP, but I just ask GPT to turn code or process or whatever else into a mermaid diagram. Most of the time I don't even need to few-shot prompt it with examples. Then you dump the resulting text into something like https://mermaid.live/ https://mermaid.live/ and voilà.
- times_trw 3y ago
  No. I've completed projects that have been on the back burner for years in hours. They weren't on the back burner for lack of interest but mainly for lack of expertise in a specific stupid area. It's not an exaggeration to say that I now do two weeks of programming a night. Of course a lot of times the result gets thrown away because the fundamental idea was flawed in a non-obvious way. But learning that is also worth while.
  - Klathmon 3y ago
    It's revolutionized our in house tooling at work. No longer do I need to give PR feedback more than a couple times, because we can just ask chatgpt to come up with a lint rule that detects and sometimes auto-fixes the issue. I use it to write or change Jenkins jobs, scaffold out tests, diagram ideas from a monologue brain dump, write alerting and monitoring code, write and clean up documentation. Most recently I wanted to get some "end of year" stats for the teams, normally it would never have happened because I don't have half a day to dedicate to relearning the git commands, and how to count the lines of code and attribute changes to teams and script the whole process to work across 20 repos. 20 minutes later with chatgpt I had results I could share within the company. It's just allowed me to skip almost all of the boring and time consuming parts of handling small things like that, and instead turns me into a code reviewer who makes a few changes to make it good enough then pushes it out
    - lifeisstillgood 3y ago
      Wait what? ChatGPT does diagrams for you? Writes documentation?
      - Klathmon 3y ago
        Absolutely! I found a plugin a while back called "AI Diagrams" that generates whimsical.com diagrams for me. Combined with the "speech to text" systems in chatgpt means I can just start babbling about some topic and let it write it all down, collect it into documentation, and even spit out a few diagrams from it. I generally have to spend like 10 minutes cleaning them up and rearranging them to look a bit more sane, but it's been a godsend! Similarly I sometimes paste a bunch of code in and tell it to write some starter docs for this code, then I can go from there and clean it up manually, or just tell it what changes need to be made. (technically I tend to use editor plugins these days not copy+paste, but the idea is the same) Other times I'll paste in docs and have it reformat them into something better. Like I recently took our ~3 year old README in a monorepo that goes over all the build and lint commands and had it rearrange everything into sets of markdown tables which displayed the data in a much easier to understand format.
- datagram 3y ago
  In my experience, it genuinely lowers the activation (and total) energy for certain tasks, since LLMs are great at writing repetitive code that would otherwise be tedious to write by hand. For instance, writing a bunch of similar test cases.
vidarh 3y ago
I find that even beyond "activation energy", a lot of my exploration with ChatGPT is around things I don't necessarily initially even intend to take forward, but is just curious about, and then realise I can do with much less effort than I expected. You can often get much of the same effect by bouncing ideas of someone, who doesn't necessarily need to know the problem space well enough to solve things but just well enough to give meaningful input. But people with the right skills aren't available at the click of a button 24/7.
moffkalast 3y ago
Agreed, I've now got a sizable project going that I would probably procrastinate on attempting for years without GPT 4 laying the initial groundwork, along with informing me of a few libraries I had never heard of that reduced wheel reinventing quite a bit.
xnorswap 3y ago
First automation acts as a powerful lever and enabler, then it replaces you. In 5 years time you may well be more productive than ever. In 15 years I doubt there'll be many programming jobs in the form they are recognisable today.
- jgilias 3y ago
  Historically, automation has always made society richer and better off as a result. There are two guys outside of my window at this very moment getting rid of a huge pile of dirt. One is in an excavator, the other in a truck. There’s a bunch of piles that they’ve taken care of today. Two centuries ago this work would’ve taken a dozen people, a few animals of burden, and a lot more time. Where are the other ten hypothetical people? I don’t know, but chances are they’ve been absorbed by the rest of the economy doing something else worthwhile that they are getting paid for.
  - dash2 3y ago
    This comment and the grandparent can both be right. Society gets richer because automation replaces jobs, leaving us more time to do other jobs.
  - daxfohl 3y ago
    Those ten hypothetical people would likely have actually been zero because the task wouldn't have been worth hiring ten people.
    - thatjoeoverthr 3y ago
      Important, and often ignored! Technological improvements change the cost structure of activities. Consider texting. We send billions of texts every day. This does not replace billions of couriers. Rather, more people work to maintain our phone supply and telecom than we ever had running around with letters. It's not a promise but it is a pattern.
  - andsoitis 3y ago
    also, two centuries ago, we didn't have trucks and the concomitant infrastructure, jobs, wealth, etc. that machines brought along.
nerdponx 3y ago
I feel very left out of all this LLM hype. It's helped me with a couple of things, but usually by the time I'm at a point where I don't know what I'm doing, the model doesn't know any better than I do. Otherwise, I have a hard time formulating prompts faster than I can just write the damn code myself. Am I just bad at using these tools?
- packetlost 3y ago
  No, you're likely just a better programmer than those relying on these tools.
  - ParetoOptimal 3y ago
    That could be the case and likely is in areas where they are strongest, just like the articles example of how its not as useful for systems programming because he is an expert. If you ask it about things you don't know it was likely trained on high quality data for and get bad answers, you likely need to improve your writing/prompting.
    - packetlost 3y ago
      Except it's most dangerous when used in areas that you're weakest because it will confidently spit out subtly wrong answers to everything. It is not a fact engine.
      - ParetoOptimal 3y ago
        That only matters if you use it for things where failure actually causes damage.
- countWSS 3y ago
  Its useful for writing generic code/template/boilerplate, then customizing it by inserting your own code. For something you already know better, there isn't a magic prompt to express it, since the code is not generic enough for LLM to understand as a prompt. Its best usecase is when you're not a domain expert, need quickly to run some unknown API/library inside your program inserting code like "write a function for loading X with Y in language Z" when you barely have an idea what is X,Y,Z. Its possible in theory to break-down everything to "write me a function for N" but the quality of such functions is not worth the prompting in most situations and you better ask it to explain how to write a function X,Y,Z step-by-step.
kodablah 3y ago
I had a project recently: building an advanced Java class file analyzer. I knew a lot about ow2 asm libraries, but it saved me a lot of digging time remembering exact descriptor formats. Also it helped me understand why other static analysis libraries weren't good enough for me for stateful reasons. For me, ChatGPT is doing two things: 1) saving trivial StackOverflow and library code walking to answer specific question, and 2) helping the initial project research stage to grasp feasibility of approaches I may take before starting.
michaelbrave 3y ago
to add to that I've heard a lot of people who have things like ADHD saying LLM's are life changing more than say neurotypical people. For those of us with bad ADHD doing simple things are the hardest, just taking out the trash or opening the mail is nearly impossible, your internal dialog is screaming at you for hours to just do the thing, but your body won't listen. They call it executive dysfunction, but it's the bane of my life LLM's helping to just start the thing is actually a huge deal.
- kolinko 3y ago
  Also, just throwing a bunch of disorganised thoughts into it, and letting it organise everything.
vmfunction 3y ago
>I think the internet and the open source movement had a similar effect, in that if you did not attempt a project that you had some small interest in, it would only be a matter of time before someone else did enough of a similar problem for you to reuse or repurpose their work, and this led to an explosion of (often useful, or at least usable) applications and libraries. Being developing before when Mac had b&w screen (system 6?). Had so many good ideas just didn't work on them for whatever reason, and eventually someone did them. Maybe it is just tech people run into the same problem, or collective tech unconscious, the longer you have being around, eventually people in the industry will try to solve the problems. These occurrences goes to show me that there are really aren't many true original ideas out there, a lot of it comes down to implementation, PR, funding, geopolitics, your users and luck.

somewhereoutth 3y ago

However could it be that the 'entertainment effect' of using a new (and trendy) technology like LLMs provides the activation energy for otherwise mundane tasks?

apwell23 3y ago
This is true. I write way more documentation now since llm does all the formatting, structure , diagrams ect. I just guide it at a high level.
- robluxus 3y ago
  Do you use a specialized LLM for diagrams?
  - apwell23 3y ago
    I just ask chatgpt to generate text diagrams that i can embed in my markdown.
  - danielbln 3y ago
    I'm not OP, but I just ask GPT to turn code or process or whatever else into a mermaid diagram. Most of the time I don't even need to few-shot prompt it with examples. Then you dump the resulting text into something like https://mermaid.live/ https://mermaid.live/ and voilà.
times_trw 3y ago
No. I've completed projects that have been on the back burner for years in hours. They weren't on the back burner for lack of interest but mainly for lack of expertise in a specific stupid area. It's not an exaggeration to say that I now do two weeks of programming a night. Of course a lot of times the result gets thrown away because the fundamental idea was flawed in a non-obvious way. But learning that is also worth while.
- Klathmon 3y ago
  It's revolutionized our in house tooling at work. No longer do I need to give PR feedback more than a couple times, because we can just ask chatgpt to come up with a lint rule that detects and sometimes auto-fixes the issue. I use it to write or change Jenkins jobs, scaffold out tests, diagram ideas from a monologue brain dump, write alerting and monitoring code, write and clean up documentation. Most recently I wanted to get some "end of year" stats for the teams, normally it would never have happened because I don't have half a day to dedicate to relearning the git commands, and how to count the lines of code and attribute changes to teams and script the whole process to work across 20 repos. 20 minutes later with chatgpt I had results I could share within the company. It's just allowed me to skip almost all of the boring and time consuming parts of handling small things like that, and instead turns me into a code reviewer who makes a few changes to make it good enough then pushes it out
  - lifeisstillgood 3y ago
    Wait what? ChatGPT does diagrams for you? Writes documentation?
    - Klathmon 3y ago
      Absolutely! I found a plugin a while back called "AI Diagrams" that generates whimsical.com diagrams for me. Combined with the "speech to text" systems in chatgpt means I can just start babbling about some topic and let it write it all down, collect it into documentation, and even spit out a few diagrams from it. I generally have to spend like 10 minutes cleaning them up and rearranging them to look a bit more sane, but it's been a godsend! Similarly I sometimes paste a bunch of code in and tell it to write some starter docs for this code, then I can go from there and clean it up manually, or just tell it what changes need to be made. (technically I tend to use editor plugins these days not copy+paste, but the idea is the same) Other times I'll paste in docs and have it reformat them into something better. Like I recently took our ~3 year old README in a monorepo that goes over all the build and lint commands and had it rearrange everything into sets of markdown tables which displayed the data in a much easier to understand format.
datagram 3y ago
In my experience, it genuinely lowers the activation (and total) energy for certain tasks, since LLMs are great at writing repetitive code that would otherwise be tedious to write by hand. For instance, writing a bunch of similar test cases.

vidarh 3y ago

I find that even beyond "activation energy", a lot of my exploration with ChatGPT is around things I don't necessarily initially even intend to take forward, but is just curious about, and then realise I can do with much less effort than I expected. You can often get much of the same effect by bouncing ideas of someone, who doesn't necessarily need to know the problem space well enough to solve things but just well enough to give meaningful input. But people with the right skills aren't available at the click of a button 24/7.

moffkalast 3y ago

Agreed, I've now got a sizable project going that I would probably procrastinate on attempting for years without GPT 4 laying the initial groundwork, along with informing me of a few libraries I had never heard of that reduced wheel reinventing quite a bit.

xnorswap 3y ago

First automation acts as a powerful lever and enabler, then it replaces you. In 5 years time you may well be more productive than ever. In 15 years I doubt there'll be many programming jobs in the form they are recognisable today.

jgilias 3y ago
Historically, automation has always made society richer and better off as a result. There are two guys outside of my window at this very moment getting rid of a huge pile of dirt. One is in an excavator, the other in a truck. There’s a bunch of piles that they’ve taken care of today. Two centuries ago this work would’ve taken a dozen people, a few animals of burden, and a lot more time. Where are the other ten hypothetical people? I don’t know, but chances are they’ve been absorbed by the rest of the economy doing something else worthwhile that they are getting paid for.
- dash2 3y ago
  This comment and the grandparent can both be right. Society gets richer because automation replaces jobs, leaving us more time to do other jobs.
- daxfohl 3y ago
  Those ten hypothetical people would likely have actually been zero because the task wouldn't have been worth hiring ten people.
  - thatjoeoverthr 3y ago
    Important, and often ignored! Technological improvements change the cost structure of activities. Consider texting. We send billions of texts every day. This does not replace billions of couriers. Rather, more people work to maintain our phone supply and telecom than we ever had running around with letters. It's not a promise but it is a pattern.
- andsoitis 3y ago
  also, two centuries ago, we didn't have trucks and the concomitant infrastructure, jobs, wealth, etc. that machines brought along.

nerdponx 3y ago

I feel very left out of all this LLM hype. It's helped me with a couple of things, but usually by the time I'm at a point where I don't know what I'm doing, the model doesn't know any better than I do. Otherwise, I have a hard time formulating prompts faster than I can just write the damn code myself. Am I just bad at using these tools?

packetlost 3y ago
No, you're likely just a better programmer than those relying on these tools.
- ParetoOptimal 3y ago
  That could be the case and likely is in areas where they are strongest, just like the articles example of how its not as useful for systems programming because he is an expert. If you ask it about things you don't know it was likely trained on high quality data for and get bad answers, you likely need to improve your writing/prompting.
  - packetlost 3y ago
    Except it's most dangerous when used in areas that you're weakest because it will confidently spit out subtly wrong answers to everything. It is not a fact engine.
    - ParetoOptimal 3y ago
      That only matters if you use it for things where failure actually causes damage.
countWSS 3y ago
Its useful for writing generic code/template/boilerplate, then customizing it by inserting your own code. For something you already know better, there isn't a magic prompt to express it, since the code is not generic enough for LLM to understand as a prompt. Its best usecase is when you're not a domain expert, need quickly to run some unknown API/library inside your program inserting code like "write a function for loading X with Y in language Z" when you barely have an idea what is X,Y,Z. Its possible in theory to break-down everything to "write me a function for N" but the quality of such functions is not worth the prompting in most situations and you better ask it to explain how to write a function X,Y,Z step-by-step.

kodablah 3y ago

I had a project recently: building an advanced Java class file analyzer. I knew a lot about ow2 asm libraries, but it saved me a lot of digging time remembering exact descriptor formats. Also it helped me understand why other static analysis libraries weren't good enough for me for stateful reasons. For me, ChatGPT is doing two things: 1) saving trivial StackOverflow and library code walking to answer specific question, and 2) helping the initial project research stage to grasp feasibility of approaches I may take before starting.

michaelbrave 3y ago

to add to that I've heard a lot of people who have things like ADHD saying LLM's are life changing more than say neurotypical people. For those of us with bad ADHD doing simple things are the hardest, just taking out the trash or opening the mail is nearly impossible, your internal dialog is screaming at you for hours to just do the thing, but your body won't listen. They call it executive dysfunction, but it's the bane of my life LLM's helping to just start the thing is actually a huge deal.

kolinko 3y ago
Also, just throwing a bunch of disorganised thoughts into it, and letting it organise everything.

vmfunction 3y ago

>I think the internet and the open source movement had a similar effect, in that if you did not attempt a project that you had some small interest in, it would only be a matter of time before someone else did enough of a similar problem for you to reuse or repurpose their work, and this led to an explosion of (often useful, or at least usable) applications and libraries. Being developing before when Mac had b&w screen (system 6?). Had so many good ideas just didn't work on them for whatever reason, and eventually someone did them. Maybe it is just tech people run into the same problem, or collective tech unconscious, the longer you have being around, eventually people in the industry will try to solve the problems. These occurrences goes to show me that there are really aren't many true original ideas out there, a lot of it comes down to implementation, PR, funding, geopolitics, your users and luck.