5 ms·

I strongly disagree with this perspective. I have found GPT4 in particular to be very helpful in learning new programming languages and improving my overall cod

I strongly disagree with this perspective. I have found GPT4 in particular to be very helpful in learning new programming languages and improving my overall coding. I still solve problems myself and I still read books, but GPT4 is the TA that I can have a conversation with about a piece of code. It's all about how you use it. If you just ask it to tell you complete answers or generate code on a blank page then you're doing yourself a disservice.

dmarchand90 3y ago
Yeah this logic would have us writing everything in assembly and only using textbooks.
- userbinator 3y ago
  I wish more people would know Asm just so they can realise how much computing power they're wasting (multiplied by however many users...) when they've seen the limits of the machine.
- mhdi_kr99 3y ago
  I believe there's a inherent benefit in learning assembly and using textbooks because it helps you better understand the software you are writing with any other language. learning hard things the hard way is good.
  - dmarchand90 3y ago
    Oh for sure! But you wouldn't argue to do this exclusively though
dgroshev 3y ago
It's a really bad TA that has no self awareness of its limitations. You are well aware that it hallucinates, right? You probably have observed it yourself in domains you're familiar with. But how would you recognise a hallucination in something that you have no familiarity with? "Learning something new" from LLMs is like trying to learn something new from a con artist. Sure, they might actually tell you the truth, but having little knowledge of something is exactly the worst possible state to be in while trying to figure out what's true and what's hallucinated.
- jebarker 3y ago
  My experience has been that the hallucinations in GPT4 are actually pretty rare. But in any case, if I choose to use code it suggests I ask it for explanations and then I verify those myself by other means, e.g. tests. I think it's too strong to call it a really bad TA. I'd say it's an imperfect TA and you need to check it's work, but it's work still has great value.
  - dgroshev 3y ago
    It's not just things that can be caught with tests. You wouldn't know if you got recommended an obsolete API, a dated pattern, or a charged foot gun, and if you're looking up docs for every suggestion I doubt that'd save you much time compared to reading docs through to begin with.
    - jebarker 3y ago
      I can definitely see those things being problems, but they are much more likely to occur when asking an LLM to generate code for you. That's not the best coding use case for LLMs IMO, it's much better to provide it code already written and ask it for explanations, bug fixes, refactoring etc. Then it rarely introduces new API calls etc
- netruk44 3y ago
  When it comes to programming, when an LLM hallucinates something in an area I'm not familiar with, it's usually very obvious that it is not correct because it just does not work at all. For me, programming hallucinations have never fallen into the case where it appears like everything is working but actually isn't doing the correct thing. So then I just forward that back to the model, or in the very worst case use my own brain to fix the problem the LLM generated. In either case, I'm still better off than had the LLM not existed, because even an incorrect solution is closer to the correct solution than nothing at all. I now have something that's close to solving my problem that I wouldn't have been able to come up with solely on my own. It just needs a little tweaking to be correct, which I can muster my way through with a little help from the LLM or Google.
piva00 3y ago
To me GPT-4 has been very valuable when I already know the gist of what I need done, as in: I've done it before, I know the techniques and the right keywords I need, I know the tools I'll use, etc. but can't remember exactly the right incantation and would need to go read references, search Google, and so on. Instead I pop by ChatGPT and ask it what I need done, the tools I'm using and let it write me an answer that I double-check to see if it matches what I'm looking for. It's definitely saved me a lot of time, I still go read the documentation but I have a much narrower point to start from. I would never just copy-paste whatever it spits out, I don't trust it to generate code nor will I ever trust it to be fed with intellectual property in the prompt (mine or my employer's) but as a general assist for techniques and tools that I've been familiar with, and which could require some researching I do think it has pretty good value.
elesiuta 3y ago
It's basically a form of rubber duck debugging. I also use for quickly writing tiny programs to isolate and test a specific piece of functionality.
monkpit 3y ago
I find it really helpful when dabbling in a language that you haven’t used much. For example: If you’re an expert in language A, and you want to do something common in A in language B, you can ask GPT to show you the idiomatic way of doing it in B and it will usually lead you down the right path (or tell you it’s not possible).

dmarchand90 3y ago

Yeah this logic would have us writing everything in assembly and only using textbooks.

userbinator 3y ago
I wish more people would know Asm just so they can realise how much computing power they're wasting (multiplied by however many users...) when they've seen the limits of the machine.
mhdi_kr99 3y ago
I believe there's a inherent benefit in learning assembly and using textbooks because it helps you better understand the software you are writing with any other language. learning hard things the hard way is good.
- dmarchand90 3y ago
  Oh for sure! But you wouldn't argue to do this exclusively though

dgroshev 3y ago

It's a really bad TA that has no self awareness of its limitations. You are well aware that it hallucinates, right? You probably have observed it yourself in domains you're familiar with. But how would you recognise a hallucination in something that you have no familiarity with? "Learning something new" from LLMs is like trying to learn something new from a con artist. Sure, they might actually tell you the truth, but having little knowledge of something is exactly the worst possible state to be in while trying to figure out what's true and what's hallucinated.

jebarker 3y ago
My experience has been that the hallucinations in GPT4 are actually pretty rare. But in any case, if I choose to use code it suggests I ask it for explanations and then I verify those myself by other means, e.g. tests. I think it's too strong to call it a really bad TA. I'd say it's an imperfect TA and you need to check it's work, but it's work still has great value.
- dgroshev 3y ago
  It's not just things that can be caught with tests. You wouldn't know if you got recommended an obsolete API, a dated pattern, or a charged foot gun, and if you're looking up docs for every suggestion I doubt that'd save you much time compared to reading docs through to begin with.
  - jebarker 3y ago
    I can definitely see those things being problems, but they are much more likely to occur when asking an LLM to generate code for you. That's not the best coding use case for LLMs IMO, it's much better to provide it code already written and ask it for explanations, bug fixes, refactoring etc. Then it rarely introduces new API calls etc
netruk44 3y ago
When it comes to programming, when an LLM hallucinates something in an area I'm not familiar with, it's usually very obvious that it is not correct because it just does not work at all. For me, programming hallucinations have never fallen into the case where it appears like everything is working but actually isn't doing the correct thing. So then I just forward that back to the model, or in the very worst case use my own brain to fix the problem the LLM generated. In either case, I'm still better off than had the LLM not existed, because even an incorrect solution is closer to the correct solution than nothing at all. I now have something that's close to solving my problem that I wouldn't have been able to come up with solely on my own. It just needs a little tweaking to be correct, which I can muster my way through with a little help from the LLM or Google.

piva00 3y ago

To me GPT-4 has been very valuable when I already know the gist of what I need done, as in: I've done it before, I know the techniques and the right keywords I need, I know the tools I'll use, etc. but can't remember exactly the right incantation and would need to go read references, search Google, and so on. Instead I pop by ChatGPT and ask it what I need done, the tools I'm using and let it write me an answer that I double-check to see if it matches what I'm looking for. It's definitely saved me a lot of time, I still go read the documentation but I have a much narrower point to start from. I would never just copy-paste whatever it spits out, I don't trust it to generate code nor will I ever trust it to be fed with intellectual property in the prompt (mine or my employer's) but as a general assist for techniques and tools that I've been familiar with, and which could require some researching I do think it has pretty good value.

elesiuta 3y ago

It's basically a form of rubber duck debugging. I also use for quickly writing tiny programs to isolate and test a specific piece of functionality.

monkpit 3y ago

I find it really helpful when dabbling in a language that you haven’t used much. For example: If you’re an expert in language A, and you want to do something common in A in language B, you can ask GPT to show you the idiomatic way of doing it in B and it will usually lead you down the right path (or tell you it’s not possible).