8 ms·

I won't deliberately use AI for programming

44 points by mhdi_kr99 3y ago

kushie 3y ago
i think everyone learns differently, and one person's "toggling between 30 tabs" is another's "having this explained to me personally over and over"
jebarker 3y ago
I strongly disagree with this perspective. I have found GPT4 in particular to be very helpful in learning new programming languages and improving my overall coding. I still solve problems myself and I still read books, but GPT4 is the TA that I can have a conversation with about a piece of code. It's all about how you use it. If you just ask it to tell you complete answers or generate code on a blank page then you're doing yourself a disservice.
- dmarchand90 3y ago
  Yeah this logic would have us writing everything in assembly and only using textbooks.
  - userbinator 3y ago
    I wish more people would know Asm just so they can realise how much computing power they're wasting (multiplied by however many users...) when they've seen the limits of the machine.
  - mhdi_kr99 3y ago
    I believe there's a inherent benefit in learning assembly and using textbooks because it helps you better understand the software you are writing with any other language. learning hard things the hard way is good.
    - dmarchand90 3y ago
      Oh for sure! But you wouldn't argue to do this exclusively though
- dgroshev 3y ago
  It's a really bad TA that has no self awareness of its limitations. You are well aware that it hallucinates, right? You probably have observed it yourself in domains you're familiar with. But how would you recognise a hallucination in something that you have no familiarity with? "Learning something new" from LLMs is like trying to learn something new from a con artist. Sure, they might actually tell you the truth, but having little knowledge of something is exactly the worst possible state to be in while trying to figure out what's true and what's hallucinated.
  - jebarker 3y ago
    My experience has been that the hallucinations in GPT4 are actually pretty rare. But in any case, if I choose to use code it suggests I ask it for explanations and then I verify those myself by other means, e.g. tests. I think it's too strong to call it a really bad TA. I'd say it's an imperfect TA and you need to check it's work, but it's work still has great value.
    - dgroshev 3y ago
      It's not just things that can be caught with tests. You wouldn't know if you got recommended an obsolete API, a dated pattern, or a charged foot gun, and if you're looking up docs for every suggestion I doubt that'd save you much time compared to reading docs through to begin with.
      - jebarker 3y ago
        I can definitely see those things being problems, but they are much more likely to occur when asking an LLM to generate code for you. That's not the best coding use case for LLMs IMO, it's much better to provide it code already written and ask it for explanations, bug fixes, refactoring etc. Then it rarely introduces new API calls etc
  - netruk44 3y ago
    When it comes to programming, when an LLM hallucinates something in an area I'm not familiar with, it's usually very obvious that it is not correct because it just does not work at all. For me, programming hallucinations have never fallen into the case where it appears like everything is working but actually isn't doing the correct thing. So then I just forward that back to the model, or in the very worst case use my own brain to fix the problem the LLM generated. In either case, I'm still better off than had the LLM not existed, because even an incorrect solution is closer to the correct solution than nothing at all. I now have something that's close to solving my problem that I wouldn't have been able to come up with solely on my own. It just needs a little tweaking to be correct, which I can muster my way through with a little help from the LLM or Google.
- piva00 3y ago
  To me GPT-4 has been very valuable when I already know the gist of what I need done, as in: I've done it before, I know the techniques and the right keywords I need, I know the tools I'll use, etc. but can't remember exactly the right incantation and would need to go read references, search Google, and so on. Instead I pop by ChatGPT and ask it what I need done, the tools I'm using and let it write me an answer that I double-check to see if it matches what I'm looking for. It's definitely saved me a lot of time, I still go read the documentation but I have a much narrower point to start from. I would never just copy-paste whatever it spits out, I don't trust it to generate code nor will I ever trust it to be fed with intellectual property in the prompt (mine or my employer's) but as a general assist for techniques and tools that I've been familiar with, and which could require some researching I do think it has pretty good value.
- elesiuta 3y ago
  It's basically a form of rubber duck debugging. I also use for quickly writing tiny programs to isolate and test a specific piece of functionality.
- monkpit 3y ago
  I find it really helpful when dabbling in a language that you haven’t used much. For example: If you’re an expert in language A, and you want to do something common in A in language B, you can ask GPT to show you the idiomatic way of doing it in B and it will usually lead you down the right path (or tell you it’s not possible).
Traubenfuchs 3y ago
What a luddite argument! AI can show you previously unknown ways, often the pragmatic and idiomatic ways to solve problems. It can also explain code in depth. I don‘t know how I‘d have quickly mastered the shitty "drool syntax" I had the displeasure to work with today. No other tool allowed one to become fluent in a new language or framework this fast, ever before. Sounds the same like people who rejected IDEs. Or even syntax highlighting. Yes, those people existed and some might still rawdog xml and java in vi today.
- sharemywin 3y ago
  49 20 70 72 65 66 65 72 20 74 6F 20 77 72 69 74 65 20 6D 79 20 63 6F 64 65 20 69 6E 20 48 65 78 2E
  - Kim_Bruning 3y ago
    01100010 01101001 01101110 01100001 01110010 01111001 00100000 01101111 01110010 00100000 01100010 01110101 01110011 01110100 00100001 (ps. vi might not necessarily be an ide by itself (though plugins can change that) , but the unix command line in its totality definitely can count as one. Sometimes it's just faster to jump into vi, fix a problem, and then continue. )
adamsb6 3y ago
I’ve found that GPT-4 now fills the role that Google once did for me when learning new things. For whatever reason the Google results just aren’t as useful as they used to be. Highly specific queries return very generalized results, and it takes a lot of effort to find something relevant. With ChatGPT I can ask it how to do just about anything in any language and it nearly always gives me exactly what I need.
- svaha1728 3y ago
  Some of this has to do with Google pushing sites like geeksforgeeks above more helpful blogs.
mattpallissard 3y ago
I do so many things manually that have been successfully automated for a long time. Washing dishes, splitting firewood, grinding coffee, owning manual transmission cars, I could go on. Some things are subjectively better like splitting firewood; It's more work but quieter. Some things are objectively better; washing dished by hand are cleaner. I don't think about it too much. I guess I'll just add programming to that list and not think about it too much.
- jebarker 3y ago
  But you do use a car, instead of walking everywhere.
  - haswell 3y ago
    This is a bit beside the point. They also use dishes instead of leaves, presumable a modern axe instead of a piece of stone, etc. To me it’s about finding the sweet spot between staying connected to the thing that you’re doing for the purpose of keeping life interesting/fulfilling and utilizing your time wisely. Driving stick is like shooting manual photos. Not necessary in 2023, but can be a lot more fun than the alternative, and enables levels of control that cannot be found with automated solutions. Choosing to walk is like choosing to forego fire altogether.
    - jebarker 3y ago
      My point is that AI is just a tool for programming and that I see it as more comparable to the difference between walking and driving than the difference between driving stick and automatic.
      - haswell 3y ago
        Eh, I can’t really agree, but I suppose this is relative to each person’s interests. To me, the analogy to walking would be like foregoing the computer entirely and using an abacus, or sticks in the sand. The end result of writing code manually or automatically is still a program made from code that must be maintained. Some people like more control over the code. The end result of using an automatic or manual transmission is still driving. Some people like more control over the operation of the transmission because of the finer grained control over the output of the car that it enables.
baz00 3y ago
I have gone past the point of giving a shit. Unless it's something I really care about the minuscule details for I'll get ChatGPT to write the main chunk of code and then fix the bugs. It's no different than having a junior dev working for you, except it's quicker and doesn't piss off to lunch or go sick when you need them.
- reportgunner 3y ago
  > doesn't piss off to lunch or go sick when you need them It doesn't until it does I think.
  - baz00 3y ago
    I'll worry when it starts a union :)
- quonn 3y ago
  I never need this. Just the work to get the code out of ChatGPT correctly is usually more effort than writing it. And besides - most of the time there is a huge amount of context and ChatGPT knows nothing about it. I touch dozens if not hundreds of files every day. Doing a little tweak here or there which rarely takes more than a few seconds to do. And the rest of the time I spend thinking and designing.
- michaelt 3y ago
  > It's no different than having a junior dev working for you In my experience it's completely different. A junior dev who wants to produce a PDF automatically will have to spend a few hours evaluating all the library options and learning their usage. But the code will certainly be free of syntax errors and will produce not-obviously-wrong output before they send it for code review. ChatGPT will select a library and know how to use it instantaneously - and yet, might call methods that don't exist. Which any human developer would have caught on the very first test run. They do have certain things in common though - ChatGPT certainly comments code like a beginner programmer :)
  - dgroshev 3y ago
koryk 3y ago
I agree with the author. The idea of "collateral knowledge" or "chain of learning" is definitely important for personal understanding and learning. However, I think depending on the person, ChatGPT actually helps the chain of learning rather than hinders.
userbinator 3y ago
Me neither. If I'm responsible for all the code I write, then "the AI did it" is not an acceptable excuse if it happens to be wrong in some subtle way.
taylodl 3y ago
You can use AI for learning. I know a lot of people who learn by looking at code and then playing with it to understand it. Generative AI is great for those people - they can quickly get code they can start with. Different strokes for different folks.
- jasfi 3y ago
  Agreed. Learning is one of AI's stronger points. You can get examples and explanations for code that's seemingly non-existent according to Google.
bitwize 3y ago
Some people (like, for example "Conscience of a Hacker" author Loyd Blankenship) still like to make furniture by hand. Despite the widespread availability of flatpacked IKEA style furniture, there is still the desire for the sheer joy of craftsmanship... and there is still demand for the end product.
- mhdi_kr99 3y ago
  That's right!
DigitalNoumena 3y ago
IMO learning is at the service of building. I learn what I need to build. If GPT can do the first 30% of any project, then I can focus my learning on the remaining 70%. Just think of GPT as another layer of abstraction. Sure, pure learning is enjoyable but also ephemeral. Building/doing = learning, yes, but no need to do everyhting.
dgroshev 3y ago
I used Copilot full time for about four months writing Rust and Python, and then stopped completely. Problem is, LLMs are really good at uninsightful, repetitive, boilerplate-y bits of code that shouldn't be there. The moment I get to anything remotely complex or subtle, it either doesn't grasp what I'm doing, needs more coaxing than it's worth, or worst of all introduces subtle bugs. It just incentivises me to write mountains of pointless code instead of stopping and thinking if it even needs to be written in the first place. What we get from code writing LLMs are exactly the same kind of "improved productivity" we got a decade ago with IDEs autogenerating Java boilerplate. It might help in the short term, but long term it just masks the pain that drives tools and skills getting better, more concise, less repetitive, more insightful. On the other hand, I'm really looking forward to tools that will be able to figure out how much "insight per line of code" our code has, some sort of "Kolmogorov complexity" but with LLMs. Anything too predictable, anything LLMs can easily fill in is a good code smell signal.
eigenhombre 3y ago
I have mixed feelings about this. I switch back and forth between two editors: Emacs for Clojure and other lisps (and Org Mode and Magit), and VSCode with Copilot for, well, most everything else. This ad hoc setup provides a little bit of "personal A/B testing" -- feeling the difference Copilot makes. When I was learning Go I had Copilot on most of the time. I was amazed at how much progress I could make despite being new to the language. And, when switching to Emacs, I'd notice myself reflexively waiting for Copilot to supply autocompletions that weren't there. Yet, strangely enough, I can actually write Go code now (with or without Copilot). All the troubleshooting, refactoring, redesigning, bug fixing, etc. that I still had to go through saw to that. (Presumably having programmed already for many years didn't hurt.) Repeated and quick exposure to the relevant idioms supplied by my tooling seems to have helped at least as much as it set me back. On the lisp (Clojure, Elisp, ...) side of things, I have found Copilot (and ChatGPT) much less helpful, presumably because of the relative paucity of training data. Simple things such as matching parentheses seem to elude these tools, and hallucinations are worse and more frequent. I'm still feeling out how this difference w.r.t VSCode/Copilot and other languages affects my experience and the quality of my output, but it definitely feels different (and, frankly, a little clumsier). Over time I've learned how helpful different kind of tools are for making software. Learning one's editor well. Writing good (and fast) tests. Using linters and code formatters. Using type systems when available. The new AI tools certainly involve trade-offs, but as I gain experience with them I am becoming convinced that putting them aside will make programming a different, and probably much slower, experience than what most programmers come to experience. (Slow can be good, but probably not what most of us want.)
- jimmaswell 3y ago
  > Repeated and quick exposure to the relevant idioms supplied by my tooling seems to have helped This sounds exactly like what we know about how children learn language. The recommendation now for adults focuses on a large portion of full immersion in content like tv/movies with no translation. We pick up on patterns and rules from repeated exposure. Maybe this will inform how we teach programming too. I had Copilot on learning Rust and it alternately felt distracting and like cheating so I turned it off, but I think I'll turn it back on based on your experience.
aldarisbm 3y ago
As anything in life, this isn't a binary choice. You can use an llm, while learning yourself. I agree with the take on copilot being a hindrance on using your brain. While learning a new language it was really hard to understand whatever is being put in front of me (as autocomplete) is truly best practice or something tied together very loosely. The extra time it takes you to google, and learn about strategies on how to code, and patterns, and design. This is what is actually valuable, being able to speak to these things in conversation when defending the code you wrote on a PR etc, it makes a difference.
gorjusborg 3y ago
I think that for some types of programmers, AI-assisted programming feels no different from what they already do. Those types feel that the end goal of 'working code' is all that matters. Their definition of 'working' meaning that the functional requirements of the feature are met. That mindset is an extreme. A developer's job is to solve a problem for someone, not to code, but copy/pasting code without understanding as a consistent behavior is a recipe for eventual collapse of a codebase. I see Copilot and other LLM code generation as methamphetamine for that sort of programmer. They will feel great, but the situation around them will degrade, those surrounding them will be unhappy, and eventually it will be clear the behavior was unsustainable. From a different angle, I would feel uneasy giving others access to my proprietary data (code), and also accepting LLM generated code into my companies products. The legal rules don't seem fully sorted.
- jimmaswell 3y ago
  As a front-end lead and long time game programmer, I've been using copilot at work and home since it was invite-only. This assessment isn't accurate, at least in my case. Copilot mostly: - Writes the same code (often verbatim) I was about to write, often multiple lines of it. "Ok, next I'll loop over the rigidbodies and apply the force to the ones in range- oh look the loop is already there" - Gives me some API calls I didn't know about, much easier than finding a stackoverflow post. There's generally not much enlightenment to be had diving deep into the formatting of java.date or some such - if I just want to format a date and move on, it's awesome being able to type a comment "//format the date as.." and get the code. The IDE will yell at me right away if it gave me something deprecated, and I'll hover over the unfamiliar bits it put there anyway to make sure everything is sensible. - Solves the small problem I'm looking at in a way I didn't think of, or even alerts me to an error in my thinking frequently enough - "Why is it doing that? Oh yeah, I really should've done that.." My codebases haven't suffered, and I'm still diligent to keep them maintainable, well-tested, and fundamentally sound. Overall, it speeds things up a lot and frees mental space for me to think about the big picture more. I generally understand the code being put out just as well as any other code I've written - be honest, you don't read a book on every API call you copy from Stack Overflow. The beauty of a good abstraction is that we generally don't have to. If anything I think it's helped me become a better programmer over the past year, elevating the mental level of abstraction I work at somewhat. I can see what you said being a problem for somebody just starting out though. And as the author said (and I found trying to learn Rust with Copilot on) it can even be an annoying distraction in that case. As an aside, copilot and gpt have certainly degraded lately with hallucinations way up (copilot chat has been extra disappointing lately), maybe to save server costs. Could be why we're seeing more negative sentiment.
jmarchello 3y ago
I used Copilot for several months and there were times when I was “wow”ed and others where it’s just got in the way. I recently decided to make my editor much more minimal and even went so far as to disable autocomplete entirely. So far I haven’t noticed a reduction in my productivity and actually find it much easier to think about what I’m writing since I’m not having to correct autocomplete and Copilot all the time. All the extra things popping up on the screen were really distracting and I find that for me, a minimal approach gives me more space to think deeply.
- mwint 3y ago
  The main thing I find copilot good for is things that are easy to validate, but tedious (not hard!) to write. Stuff like colors for different statuses, or error messages. I actually love it for writing error messages. Keeps me in flow better not having to switch to writing Human English every five lines when something fails validation.
  - jmarchello 3y ago
    I agree, in fact I used it to generate a list of colors for a very simple color picker at work. It was very helpful. That said I could also have a chat interface do that for me when I want it, so I can keep my editing experience distraction free.
deleted 3y ago
[deleted]
Toutouxc 3y ago
> while attempting to grasp new concepts, numerous questions might crop up that encourage you to delve deeper and acquire additional knowledge So far GPT-4 has proven to be much more helpful in giving me pointers or straight up answers (that I DO check!) to some of my questions than any other source, tool or entity in my life. I've explored and lightly discussed topics I never knew I was interested in.

44 points by mhdi_kr99 3y ago

i think everyone learns differently, and one person's "toggling between 30 tabs" is another's "having this explained to me personally over and over"

jebarker 3y ago

I strongly disagree with this perspective. I have found GPT4 in particular to be very helpful in learning new programming languages and improving my overall coding. I still solve problems myself and I still read books, but GPT4 is the TA that I can have a conversation with about a piece of code. It's all about how you use it. If you just ask it to tell you complete answers or generate code on a blank page then you're doing yourself a disservice.

dmarchand90 3y ago
Yeah this logic would have us writing everything in assembly and only using textbooks.
- userbinator 3y ago
  I wish more people would know Asm just so they can realise how much computing power they're wasting (multiplied by however many users...) when they've seen the limits of the machine.
- mhdi_kr99 3y ago
  I believe there's a inherent benefit in learning assembly and using textbooks because it helps you better understand the software you are writing with any other language. learning hard things the hard way is good.
  - dmarchand90 3y ago
    Oh for sure! But you wouldn't argue to do this exclusively though
dgroshev 3y ago
It's a really bad TA that has no self awareness of its limitations. You are well aware that it hallucinates, right? You probably have observed it yourself in domains you're familiar with. But how would you recognise a hallucination in something that you have no familiarity with? "Learning something new" from LLMs is like trying to learn something new from a con artist. Sure, they might actually tell you the truth, but having little knowledge of something is exactly the worst possible state to be in while trying to figure out what's true and what's hallucinated.
- jebarker 3y ago
  My experience has been that the hallucinations in GPT4 are actually pretty rare. But in any case, if I choose to use code it suggests I ask it for explanations and then I verify those myself by other means, e.g. tests. I think it's too strong to call it a really bad TA. I'd say it's an imperfect TA and you need to check it's work, but it's work still has great value.
  - dgroshev 3y ago
    It's not just things that can be caught with tests. You wouldn't know if you got recommended an obsolete API, a dated pattern, or a charged foot gun, and if you're looking up docs for every suggestion I doubt that'd save you much time compared to reading docs through to begin with.
    - jebarker 3y ago
      I can definitely see those things being problems, but they are much more likely to occur when asking an LLM to generate code for you. That's not the best coding use case for LLMs IMO, it's much better to provide it code already written and ask it for explanations, bug fixes, refactoring etc. Then it rarely introduces new API calls etc
- netruk44 3y ago
  When it comes to programming, when an LLM hallucinates something in an area I'm not familiar with, it's usually very obvious that it is not correct because it just does not work at all. For me, programming hallucinations have never fallen into the case where it appears like everything is working but actually isn't doing the correct thing. So then I just forward that back to the model, or in the very worst case use my own brain to fix the problem the LLM generated. In either case, I'm still better off than had the LLM not existed, because even an incorrect solution is closer to the correct solution than nothing at all. I now have something that's close to solving my problem that I wouldn't have been able to come up with solely on my own. It just needs a little tweaking to be correct, which I can muster my way through with a little help from the LLM or Google.
piva00 3y ago
To me GPT-4 has been very valuable when I already know the gist of what I need done, as in: I've done it before, I know the techniques and the right keywords I need, I know the tools I'll use, etc. but can't remember exactly the right incantation and would need to go read references, search Google, and so on. Instead I pop by ChatGPT and ask it what I need done, the tools I'm using and let it write me an answer that I double-check to see if it matches what I'm looking for. It's definitely saved me a lot of time, I still go read the documentation but I have a much narrower point to start from. I would never just copy-paste whatever it spits out, I don't trust it to generate code nor will I ever trust it to be fed with intellectual property in the prompt (mine or my employer's) but as a general assist for techniques and tools that I've been familiar with, and which could require some researching I do think it has pretty good value.
elesiuta 3y ago
It's basically a form of rubber duck debugging. I also use for quickly writing tiny programs to isolate and test a specific piece of functionality.
monkpit 3y ago
I find it really helpful when dabbling in a language that you haven’t used much. For example: If you’re an expert in language A, and you want to do something common in A in language B, you can ask GPT to show you the idiomatic way of doing it in B and it will usually lead you down the right path (or tell you it’s not possible).

Traubenfuchs 3y ago

What a luddite argument! AI can show you previously unknown ways, often the pragmatic and idiomatic ways to solve problems. It can also explain code in depth. I don‘t know how I‘d have quickly mastered the shitty "drool syntax" I had the displeasure to work with today. No other tool allowed one to become fluent in a new language or framework this fast, ever before. Sounds the same like people who rejected IDEs. Or even syntax highlighting. Yes, those people existed and some might still rawdog xml and java in vi today.

sharemywin 3y ago
49 20 70 72 65 66 65 72 20 74 6F 20 77 72 69 74 65 20 6D 79 20 63 6F 64 65 20 69 6E 20 48 65 78 2E
- Kim_Bruning 3y ago
  01100010 01101001 01101110 01100001 01110010 01111001 00100000 01101111 01110010 00100000 01100010 01110101 01110011 01110100 00100001 (ps. vi might not necessarily be an ide by itself (though plugins can change that) , but the unix command line in its totality definitely can count as one. Sometimes it's just faster to jump into vi, fix a problem, and then continue. )

adamsb6 3y ago

I’ve found that GPT-4 now fills the role that Google once did for me when learning new things. For whatever reason the Google results just aren’t as useful as they used to be. Highly specific queries return very generalized results, and it takes a lot of effort to find something relevant. With ChatGPT I can ask it how to do just about anything in any language and it nearly always gives me exactly what I need.

svaha1728 3y ago
Some of this has to do with Google pushing sites like geeksforgeeks above more helpful blogs.

mattpallissard 3y ago

I do so many things manually that have been successfully automated for a long time. Washing dishes, splitting firewood, grinding coffee, owning manual transmission cars, I could go on. Some things are subjectively better like splitting firewood; It's more work but quieter. Some things are objectively better; washing dished by hand are cleaner. I don't think about it too much. I guess I'll just add programming to that list and not think about it too much.

jebarker 3y ago
But you do use a car, instead of walking everywhere.
- haswell 3y ago
  This is a bit beside the point. They also use dishes instead of leaves, presumable a modern axe instead of a piece of stone, etc. To me it’s about finding the sweet spot between staying connected to the thing that you’re doing for the purpose of keeping life interesting/fulfilling and utilizing your time wisely. Driving stick is like shooting manual photos. Not necessary in 2023, but can be a lot more fun than the alternative, and enables levels of control that cannot be found with automated solutions. Choosing to walk is like choosing to forego fire altogether.
  - jebarker 3y ago
    My point is that AI is just a tool for programming and that I see it as more comparable to the difference between walking and driving than the difference between driving stick and automatic.
    - haswell 3y ago
      Eh, I can’t really agree, but I suppose this is relative to each person’s interests. To me, the analogy to walking would be like foregoing the computer entirely and using an abacus, or sticks in the sand. The end result of writing code manually or automatically is still a program made from code that must be maintained. Some people like more control over the code. The end result of using an automatic or manual transmission is still driving. Some people like more control over the operation of the transmission because of the finer grained control over the output of the car that it enables.

baz00 3y ago

I have gone past the point of giving a shit. Unless it's something I really care about the minuscule details for I'll get ChatGPT to write the main chunk of code and then fix the bugs. It's no different than having a junior dev working for you, except it's quicker and doesn't piss off to lunch or go sick when you need them.

reportgunner 3y ago
> doesn't piss off to lunch or go sick when you need them It doesn't until it does I think.
- baz00 3y ago
  I'll worry when it starts a union :)
quonn 3y ago
I never need this. Just the work to get the code out of ChatGPT correctly is usually more effort than writing it. And besides - most of the time there is a huge amount of context and ChatGPT knows nothing about it. I touch dozens if not hundreds of files every day. Doing a little tweak here or there which rarely takes more than a few seconds to do. And the rest of the time I spend thinking and designing.
michaelt 3y ago
> It's no different than having a junior dev working for you In my experience it's completely different. A junior dev who wants to produce a PDF automatically will have to spend a few hours evaluating all the library options and learning their usage. But the code will certainly be free of syntax errors and will produce not-obviously-wrong output before they send it for code review. ChatGPT will select a library and know how to use it instantaneously - and yet, might call methods that don't exist. Which any human developer would have caught on the very first test run. They do have certain things in common though - ChatGPT certainly comments code like a beginner programmer :)
- dgroshev 3y ago

koryk 3y ago

I agree with the author. The idea of "collateral knowledge" or "chain of learning" is definitely important for personal understanding and learning. However, I think depending on the person, ChatGPT actually helps the chain of learning rather than hinders.

userbinator 3y ago

Me neither. If I'm responsible for all the code I write, then "the AI did it" is not an acceptable excuse if it happens to be wrong in some subtle way.

taylodl 3y ago

You can use AI for learning. I know a lot of people who learn by looking at code and then playing with it to understand it. Generative AI is great for those people - they can quickly get code they can start with. Different strokes for different folks.

jasfi 3y ago
Agreed. Learning is one of AI's stronger points. You can get examples and explanations for code that's seemingly non-existent according to Google.

bitwize 3y ago

Some people (like, for example "Conscience of a Hacker" author Loyd Blankenship) still like to make furniture by hand. Despite the widespread availability of flatpacked IKEA style furniture, there is still the desire for the sheer joy of craftsmanship... and there is still demand for the end product.

mhdi_kr99 3y ago
That's right!

DigitalNoumena 3y ago

IMO learning is at the service of building. I learn what I need to build. If GPT can do the first 30% of any project, then I can focus my learning on the remaining 70%. Just think of GPT as another layer of abstraction. Sure, pure learning is enjoyable but also ephemeral. Building/doing = learning, yes, but no need to do everyhting.

dgroshev 3y ago

I used Copilot full time for about four months writing Rust and Python, and then stopped completely. Problem is, LLMs are really good at uninsightful, repetitive, boilerplate-y bits of code that shouldn't be there. The moment I get to anything remotely complex or subtle, it either doesn't grasp what I'm doing, needs more coaxing than it's worth, or worst of all introduces subtle bugs. It just incentivises me to write mountains of pointless code instead of stopping and thinking if it even needs to be written in the first place. What we get from code writing LLMs are exactly the same kind of "improved productivity" we got a decade ago with IDEs autogenerating Java boilerplate. It might help in the short term, but long term it just masks the pain that drives tools and skills getting better, more concise, less repetitive, more insightful. On the other hand, I'm really looking forward to tools that will be able to figure out how much "insight per line of code" our code has, some sort of "Kolmogorov complexity" but with LLMs. Anything too predictable, anything LLMs can easily fill in is a good code smell signal.

eigenhombre 3y ago

I have mixed feelings about this. I switch back and forth between two editors: Emacs for Clojure and other lisps (and Org Mode and Magit), and VSCode with Copilot for, well, most everything else. This ad hoc setup provides a little bit of "personal A/B testing" -- feeling the difference Copilot makes. When I was learning Go I had Copilot on most of the time. I was amazed at how much progress I could make despite being new to the language. And, when switching to Emacs, I'd notice myself reflexively waiting for Copilot to supply autocompletions that weren't there. Yet, strangely enough, I can actually write Go code now (with or without Copilot). All the troubleshooting, refactoring, redesigning, bug fixing, etc. that I still had to go through saw to that. (Presumably having programmed already for many years didn't hurt.) Repeated and quick exposure to the relevant idioms supplied by my tooling seems to have helped at least as much as it set me back. On the lisp (Clojure, Elisp, ...) side of things, I have found Copilot (and ChatGPT) much less helpful, presumably because of the relative paucity of training data. Simple things such as matching parentheses seem to elude these tools, and hallucinations are worse and more frequent. I'm still feeling out how this difference w.r.t VSCode/Copilot and other languages affects my experience and the quality of my output, but it definitely feels different (and, frankly, a little clumsier). Over time I've learned how helpful different kind of tools are for making software. Learning one's editor well. Writing good (and fast) tests. Using linters and code formatters. Using type systems when available. The new AI tools certainly involve trade-offs, but as I gain experience with them I am becoming convinced that putting them aside will make programming a different, and probably much slower, experience than what most programmers come to experience. (Slow can be good, but probably not what most of us want.)

jimmaswell 3y ago
> Repeated and quick exposure to the relevant idioms supplied by my tooling seems to have helped This sounds exactly like what we know about how children learn language. The recommendation now for adults focuses on a large portion of full immersion in content like tv/movies with no translation. We pick up on patterns and rules from repeated exposure. Maybe this will inform how we teach programming too. I had Copilot on learning Rust and it alternately felt distracting and like cheating so I turned it off, but I think I'll turn it back on based on your experience.

aldarisbm 3y ago

As anything in life, this isn't a binary choice. You can use an llm, while learning yourself. I agree with the take on copilot being a hindrance on using your brain. While learning a new language it was really hard to understand whatever is being put in front of me (as autocomplete) is truly best practice or something tied together very loosely. The extra time it takes you to google, and learn about strategies on how to code, and patterns, and design. This is what is actually valuable, being able to speak to these things in conversation when defending the code you wrote on a PR etc, it makes a difference.

gorjusborg 3y ago

I think that for some types of programmers, AI-assisted programming feels no different from what they already do. Those types feel that the end goal of 'working code' is all that matters. Their definition of 'working' meaning that the functional requirements of the feature are met. That mindset is an extreme. A developer's job is to solve a problem for someone, not to code, but copy/pasting code without understanding as a consistent behavior is a recipe for eventual collapse of a codebase. I see Copilot and other LLM code generation as methamphetamine for that sort of programmer. They will feel great, but the situation around them will degrade, those surrounding them will be unhappy, and eventually it will be clear the behavior was unsustainable. From a different angle, I would feel uneasy giving others access to my proprietary data (code), and also accepting LLM generated code into my companies products. The legal rules don't seem fully sorted.

jimmaswell 3y ago
As a front-end lead and long time game programmer, I've been using copilot at work and home since it was invite-only. This assessment isn't accurate, at least in my case. Copilot mostly: - Writes the same code (often verbatim) I was about to write, often multiple lines of it. "Ok, next I'll loop over the rigidbodies and apply the force to the ones in range- oh look the loop is already there" - Gives me some API calls I didn't know about, much easier than finding a stackoverflow post. There's generally not much enlightenment to be had diving deep into the formatting of java.date or some such - if I just want to format a date and move on, it's awesome being able to type a comment "//format the date as.." and get the code. The IDE will yell at me right away if it gave me something deprecated, and I'll hover over the unfamiliar bits it put there anyway to make sure everything is sensible. - Solves the small problem I'm looking at in a way I didn't think of, or even alerts me to an error in my thinking frequently enough - "Why is it doing that? Oh yeah, I really should've done that.." My codebases haven't suffered, and I'm still diligent to keep them maintainable, well-tested, and fundamentally sound. Overall, it speeds things up a lot and frees mental space for me to think about the big picture more. I generally understand the code being put out just as well as any other code I've written - be honest, you don't read a book on every API call you copy from Stack Overflow. The beauty of a good abstraction is that we generally don't have to. If anything I think it's helped me become a better programmer over the past year, elevating the mental level of abstraction I work at somewhat. I can see what you said being a problem for somebody just starting out though. And as the author said (and I found trying to learn Rust with Copilot on) it can even be an annoying distraction in that case. As an aside, copilot and gpt have certainly degraded lately with hallucinations way up (copilot chat has been extra disappointing lately), maybe to save server costs. Could be why we're seeing more negative sentiment.

jmarchello 3y ago

I used Copilot for several months and there were times when I was “wow”ed and others where it’s just got in the way. I recently decided to make my editor much more minimal and even went so far as to disable autocomplete entirely. So far I haven’t noticed a reduction in my productivity and actually find it much easier to think about what I’m writing since I’m not having to correct autocomplete and Copilot all the time. All the extra things popping up on the screen were really distracting and I find that for me, a minimal approach gives me more space to think deeply.

mwint 3y ago
The main thing I find copilot good for is things that are easy to validate, but tedious (not hard!) to write. Stuff like colors for different statuses, or error messages. I actually love it for writing error messages. Keeps me in flow better not having to switch to writing Human English every five lines when something fails validation.
- jmarchello 3y ago
  I agree, in fact I used it to generate a list of colors for a very simple color picker at work. It was very helpful. That said I could also have a chat interface do that for me when I want it, so I can keep my editing experience distraction free.

deleted 3y ago

[deleted]

Toutouxc 3y ago

> while attempting to grasp new concepts, numerous questions might crop up that encourage you to delve deeper and acquire additional knowledge So far GPT-4 has proven to be much more helpful in giving me pointers or straight up answers (that I DO check!) to some of my questions than any other source, tool or entity in my life. I've explored and lightly discussed topics I never knew I was interested in.