6 ms·

lol no If you think it's the same as using SO, I don't think you have used LLMs enough. For example, I have countless times just dumped errors from my terminal

lol no If you think it's the same as using SO, I don't think you have used LLMs enough. For example, I have countless times just dumped errors from my terminal into GPT-4 and it gave me steps to resolve the issues. A number of times, Googling the same issues didn't provide any answers. Can "copy/paste from stackoverflow" fix and update config files you paste into it? No. Can it explain to you the changes it made and answer your specific questions about said changes immediately? No. Can it give you all options for libraries in your language that do x, tell you the pros and cons, relating them to your use-case, and integrate said library into a code snippet you provide? No. Right now, GPT-4 is tutoring me in Haskell, Elasticsearch, and AWS. Can "copy/paste from stackoverflow" do that? No.

smoldesu 3y ago
> For example, I have countless times just dumped errors from my terminal into GPT-4 and it gave me steps to resolve the issues. That's fine, but a pretty poor basis of comparison for what StackOverflow can do. If you identify the part that is broken (which most linters will do for you) and paste that into Google, you will get what you want roughly as often as ChatGPT furnishes a proper answer. I disagree with the parent's pointless moralizing of the situation, but I'm also pretty disenfranchised with the state of GPT-enabled search and programming. When I ask ChatGPT to write something I've already done myself, it will often do things in roundabout or outdated ways. When you play the role of a proctor rather than a student, you can really identify it's shortcomings. That's not to say AI-enabled coding isn't helpful. But past a certain watermark of trivial problem-solving, AI literally lacks the context to assist in certain codebases where human programmers could easily grok what's happening. Maybe my answer isn't based off the state of the art, but nothing I've tried so far has really gotten everything right. Claiming that it's "malpractice" to avoid it is a plainly inflammatory claim.
- catlover76 3y ago
  > Claiming that it's "malpractice" to avoid it is a plainly inflammatory claim Well, I'll certainly accept the critique on that part, lol I think that the current frontier is the completion of tasks that require large chunks of a codebase as context/knowledge. I think one major area where it's useful is that it "knows" how to use sufficiently common libraries in your language and is simply a superior form of documentation for a lot of stuff. For example, I wanted to do stuff with PDFs in Python, both ingesting PDFs and modifying them. For the latter functionality, I asked GPT-4 what my options were for modifying PDFs in Python in xyz manner; it gave some options, I went with ReportLab; I asked it how specifically to do what I wanted in ReportLab, telling it to fill in some half-baked function I copied and pasted in. It filled it in, gave me the imports, etc. This all took about 5-10m. There is no way it would have been efficient for me to research the original question, and no way I could have derived the same functionality from ReportLab's documentation as quickly. I use it a lot as a replacement for "documentation" in that sense; for language-level libs, for stuff like Docker (whose docs are pretty bad). It's just a waste of time, IMO, to read people's (usually) poorly written documentation or a SO thread when the LLM can just cogently distill the information for you AND answer your specific questions about the issue. Similarly, it's been pretty invaluable for me in learning Haskell. Syntax and other errors in Haskell provided through VSCode Haskell extension via GHC are comically bad, IMO, often amounting to a small red squiggly that's not even where the actual error is, with the message "parse error". When I have put the code snippets into GPT-4, it has identified and corrected the issue 99% of the time, and I can ask follow up questions of my own and get explanations. It's like having a senior Haskell dev at my beck and call to badger, and I think that's the general vibe of my usage. I consider myself a pretty competent mid-level dev who can ask good questions and has a reasonable estimation of what he doesn't know, and I think that's a good Goldilocks position to make good use of the tool. Accordingly, I think if people who are working with the same stuff in the same codebase who have themselves become relatively expert in what they need to know won't find LLMs as useful here, but if you're in a situation where you have to get familiar with a lot of new stuff, they are pretty incomparable. > If you identify the part that is broken (which most linters will do for you) and paste that into Google, you will get what you want roughly as often as ChatGPT furnishes a proper answer. Well most of the time, I don't need to identify the part that is broken or devote any mental energy to combing through Google search results. I just copy and paste errors into GPT-4, often without any context, and it tells me what's wrong, and I take the fix and do it, only stopping to comprehend what actually went wrong if I judge that it's worthwhile. The UX of traditional internet search, with its multiple results of unguaranteed quality, is, I think, kind of obsolete now in many contexts. I have questions to which I want answers. I have patience for reading actual long-form content, such as good dev blog posts, books on stuff like Docker, cloud providers, "how to build an interpreter", etc. What I do not have patience for is combing a bunch of random internet Q&A stuff that is about an error that might not even be the exact error I have.
  - cratermoon 3y ago
    Everything LLMs produce just a statistically likely pattern based on inputs, which in this case happens to be a huge corpus of information from published information on the web. Everything it's "tutoring" you with can already be found, GPT-4 has just created a compressed summary of the most frequent patterns. If you want to be an autodidact, seek out the information from primary sources, study it and organize it yourself. With GPT-4, you're essentially reading a sloppy cliffs notes of the topics. You'll be much better informed and be able to think more deeply about the subject matter if you adopt some ideas from Richard Feynman, wherein you try to explain what you think you know to a child, or at least someone with no knowledge of the subject. That will highlight the gaps in your understanding, and you can return to the source material to fill in the gaps. You're right about the enshittification of search, though. Maybe ask GPT-4 for a solution to that.
    - catlover76 3y ago
      Most of the time, I have not found the distillation from GPT-4 to be "sloppy". > If you want to be an autodidact, seek out the information from primary sources, study it and organize it yourself. With GPT-4, you're essentially reading a sloppy cliffs notes of the topics. You'll be much better informed and be able to think more deeply about the subject matter if you adopt some ideas from Richard Feynman, wherein you try to explain what you think you know to a child, or at least someone with no knowledge of the subject. That will highlight the gaps in your understanding, and you can return to the source material to fill in the gaps. The most immediate and fundamental problem with what you are suggesting is that it is infeasible. Nobody has time to learn everything they wish to at the level of depth they would ideally want. Nobody has time to read all the primary sources on which GPT-4 was trained. I find this so obvious I won't belabor the point. I don't think any other tool can approximate the personalized "per-unit-of-time learning" afforded by a good LLM.
      - ipaddr 3y ago
        The topics you mentioned were picked up by others with gpt. Because they learned the pattern themselves they have a deeper knowledge then you. You might remember the prompted you used but someone else can identity the pattern and apply to other problems. You are cheating yourself out of learning.

smoldesu 3y ago

> For example, I have countless times just dumped errors from my terminal into GPT-4 and it gave me steps to resolve the issues. That's fine, but a pretty poor basis of comparison for what StackOverflow can do. If you identify the part that is broken (which most linters will do for you) and paste that into Google, you will get what you want roughly as often as ChatGPT furnishes a proper answer. I disagree with the parent's pointless moralizing of the situation, but I'm also pretty disenfranchised with the state of GPT-enabled search and programming. When I ask ChatGPT to write something I've already done myself, it will often do things in roundabout or outdated ways. When you play the role of a proctor rather than a student, you can really identify it's shortcomings. That's not to say AI-enabled coding isn't helpful. But past a certain watermark of trivial problem-solving, AI literally lacks the context to assist in certain codebases where human programmers could easily grok what's happening. Maybe my answer isn't based off the state of the art, but nothing I've tried so far has really gotten everything right. Claiming that it's "malpractice" to avoid it is a plainly inflammatory claim.

catlover76 3y ago
> Claiming that it's "malpractice" to avoid it is a plainly inflammatory claim Well, I'll certainly accept the critique on that part, lol I think that the current frontier is the completion of tasks that require large chunks of a codebase as context/knowledge. I think one major area where it's useful is that it "knows" how to use sufficiently common libraries in your language and is simply a superior form of documentation for a lot of stuff. For example, I wanted to do stuff with PDFs in Python, both ingesting PDFs and modifying them. For the latter functionality, I asked GPT-4 what my options were for modifying PDFs in Python in xyz manner; it gave some options, I went with ReportLab; I asked it how specifically to do what I wanted in ReportLab, telling it to fill in some half-baked function I copied and pasted in. It filled it in, gave me the imports, etc. This all took about 5-10m. There is no way it would have been efficient for me to research the original question, and no way I could have derived the same functionality from ReportLab's documentation as quickly. I use it a lot as a replacement for "documentation" in that sense; for language-level libs, for stuff like Docker (whose docs are pretty bad). It's just a waste of time, IMO, to read people's (usually) poorly written documentation or a SO thread when the LLM can just cogently distill the information for you AND answer your specific questions about the issue. Similarly, it's been pretty invaluable for me in learning Haskell. Syntax and other errors in Haskell provided through VSCode Haskell extension via GHC are comically bad, IMO, often amounting to a small red squiggly that's not even where the actual error is, with the message "parse error". When I have put the code snippets into GPT-4, it has identified and corrected the issue 99% of the time, and I can ask follow up questions of my own and get explanations. It's like having a senior Haskell dev at my beck and call to badger, and I think that's the general vibe of my usage. I consider myself a pretty competent mid-level dev who can ask good questions and has a reasonable estimation of what he doesn't know, and I think that's a good Goldilocks position to make good use of the tool. Accordingly, I think if people who are working with the same stuff in the same codebase who have themselves become relatively expert in what they need to know won't find LLMs as useful here, but if you're in a situation where you have to get familiar with a lot of new stuff, they are pretty incomparable. > If you identify the part that is broken (which most linters will do for you) and paste that into Google, you will get what you want roughly as often as ChatGPT furnishes a proper answer. Well most of the time, I don't need to identify the part that is broken or devote any mental energy to combing through Google search results. I just copy and paste errors into GPT-4, often without any context, and it tells me what's wrong, and I take the fix and do it, only stopping to comprehend what actually went wrong if I judge that it's worthwhile. The UX of traditional internet search, with its multiple results of unguaranteed quality, is, I think, kind of obsolete now in many contexts. I have questions to which I want answers. I have patience for reading actual long-form content, such as good dev blog posts, books on stuff like Docker, cloud providers, "how to build an interpreter", etc. What I do not have patience for is combing a bunch of random internet Q&A stuff that is about an error that might not even be the exact error I have.
- cratermoon 3y ago
  Everything LLMs produce just a statistically likely pattern based on inputs, which in this case happens to be a huge corpus of information from published information on the web. Everything it's "tutoring" you with can already be found, GPT-4 has just created a compressed summary of the most frequent patterns. If you want to be an autodidact, seek out the information from primary sources, study it and organize it yourself. With GPT-4, you're essentially reading a sloppy cliffs notes of the topics. You'll be much better informed and be able to think more deeply about the subject matter if you adopt some ideas from Richard Feynman, wherein you try to explain what you think you know to a child, or at least someone with no knowledge of the subject. That will highlight the gaps in your understanding, and you can return to the source material to fill in the gaps. You're right about the enshittification of search, though. Maybe ask GPT-4 for a solution to that.
  - catlover76 3y ago
    Most of the time, I have not found the distillation from GPT-4 to be "sloppy". > If you want to be an autodidact, seek out the information from primary sources, study it and organize it yourself. With GPT-4, you're essentially reading a sloppy cliffs notes of the topics. You'll be much better informed and be able to think more deeply about the subject matter if you adopt some ideas from Richard Feynman, wherein you try to explain what you think you know to a child, or at least someone with no knowledge of the subject. That will highlight the gaps in your understanding, and you can return to the source material to fill in the gaps. The most immediate and fundamental problem with what you are suggesting is that it is infeasible. Nobody has time to learn everything they wish to at the level of depth they would ideally want. Nobody has time to read all the primary sources on which GPT-4 was trained. I find this so obvious I won't belabor the point. I don't think any other tool can approximate the personalized "per-unit-of-time learning" afforded by a good LLM.
    - ipaddr 3y ago
      The topics you mentioned were picked up by others with gpt. Because they learned the pattern themselves they have a deeper knowledge then you. You might remember the prompted you used but someone else can identity the pattern and apply to other problems. You are cheating yourself out of learning.