2 ms·

Agreed, for me it's on multiple occasions fixed some of those errors that are syntactically correct but can't be seen by the IDE or the interpreter/compiler, li

Agreed, for me it's on multiple occasions fixed some of those errors that are syntactically correct but can't be seen by the IDE or the interpreter/compiler, like forgetting to change some variable when copy pasting a section or calling the wrong function of the correct type. Stuff that can only be spotted when thinking about code semantically. GPT 4 is also a serious CSS master, so that cuts so much time from trial and error there. Current drawbacks though: - the 2021 cutoff is very apparent, it's terrible at newer stuff since it can't pull from many examples (browsing mode helps, but it usually fails at finding the info it needs) - it really can't help with the typical workflow of editing some small thing in a huge codebase because there's no way you can give it enough context for an answer that isn't based on heavy speculation - when doing too much back and forth it eventually starts to cut tokens and no longer knows what the original question was; sometimes it's not an issue but other times it just goes off topic

lakySK 3y ago
Definitely experienced all 3, but still getting more boost than drag overall. The large codebase context might be somewhat solvable and I've seen projects that use embeddings to find the relevant bits of code to feed GPT to help it with context. No clue how well any of them work though, haven't tested them yet. I've definitely noticed times when the conversation gets cut off and it can't "remember" the previous messages. Often, it results in a loop of ChatGPT giving me a solution, me getting an error and sending it back, then ChatGPT being terribly sorry and suggesting a new solution. Repeat 3 times and often we make a full circle to the first solution in this way...
- dinvlad 3y ago
  I know right? To me it feels like cheating in an exam, where I ask a know-it-all who gives a straight answer without me understanding it, but then it turns out that the know-it-all didn’t know it very well either, but just appeared confident, so we both failed the exam.
  - lakySK 3y ago
    If you treat LLMs as know-it-alls and just copy the output, then your expectations of the current generation of LLMs are too high. That doesn’t mean they’re not useful though.
    - dinvlad 3y ago
      OK let’s say if I don’t, how can I make sure what I get is correct, even after a few iterations?
      - hxugufjfjf 3y ago
        You ask it to explain its reasoning step by step. This has also proven to yield more accurate results. Look into Chain of Thought
        dinvlad 3y ago
        It’s not very good at explaining stuff though. One example: ask it to explain a subtle joke, and it will keep failing in funny ways. This is not surprising though, as these kinds of models (LLMs) were specifically optimized for generation, not explanation.
        hxugufjfjf 3y ago
        It's extremely good at explaining technical things like solving programming problems.

lakySK 3y ago

Definitely experienced all 3, but still getting more boost than drag overall. The large codebase context might be somewhat solvable and I've seen projects that use embeddings to find the relevant bits of code to feed GPT to help it with context. No clue how well any of them work though, haven't tested them yet. I've definitely noticed times when the conversation gets cut off and it can't "remember" the previous messages. Often, it results in a loop of ChatGPT giving me a solution, me getting an error and sending it back, then ChatGPT being terribly sorry and suggesting a new solution. Repeat 3 times and often we make a full circle to the first solution in this way...

dinvlad 3y ago
I know right? To me it feels like cheating in an exam, where I ask a know-it-all who gives a straight answer without me understanding it, but then it turns out that the know-it-all didn’t know it very well either, but just appeared confident, so we both failed the exam.
- lakySK 3y ago
  If you treat LLMs as know-it-alls and just copy the output, then your expectations of the current generation of LLMs are too high. That doesn’t mean they’re not useful though.
  - dinvlad 3y ago
    OK let’s say if I don’t, how can I make sure what I get is correct, even after a few iterations?
    - hxugufjfjf 3y ago
      You ask it to explain its reasoning step by step. This has also proven to yield more accurate results. Look into Chain of Thought
      - dinvlad 3y ago
        It’s not very good at explaining stuff though. One example: ask it to explain a subtle joke, and it will keep failing in funny ways. This is not surprising though, as these kinds of models (LLMs) were specifically optimized for generation, not explanation.
        hxugufjfjf 3y ago
        It's extremely good at explaining technical things like solving programming problems.