4 ms·

If you don't trust it to write… you probably shouldn't trust it to tell you how to write either.

by Nullabillity 2y ago

constantcrying 2y ago
Why? It is clearly better at one thing than the other.
- Nullabillity 2y ago
  Is it, or are you just having a harder time spotting the errors?
  - constantcrying 2y ago
    It seems far superior to Google.
  - kragen 2y ago
    llms like gpt-4 are enormously better than google at finding the relevant functions in an api and telling you how to chain them together to accomplish a given task. sometimes there are errors, but in most cases those are easy to spot, especially if you read the documentation for those functions. for some purposes, testing is enough (and it will happily write you a test suite too) it's kind of annoying to ferret out the bugs it has carefully concealed in the code it wrote for you, though
    - consp 2y ago
      > especially if you read the documentation for those functions Shouldn't that be enough in the first place? RTFM is a real thing.
      - kragen 2y ago
        the trouble is that you don't know which functions to read the manual for. last night, for example, i was trying to find out how to get objdump to disassemble an ms-dos .com file with the correct load address of 0x100 rather than 0. google was unhelpful, but gpt-4 told me that the option i was looking for was called `--adjust-vma`. probably if i'd read the objdump manual i would have been able to find it (though the objdump documentation isn't very clear about what `--adjust-vma` will do on a raw binary file), and technically that's only about 15 pages, and spending the requisite half hour reading it would probably produce returns the next time i use objdump. so it's a reasonable idea but it's pretty common for gpt-4 to be able to instantly find the thing you need out of 150 pages or 1500 pages or 15000 pages. or the five things you need, and how to combine them. if you want to know what libraries can probably solve your problem, asking gpt-4 is a much better idea than reading the manuals of all possible libraries
  - madeofpalk 2y ago
    Stack Overflow is also full of very confident wrong answers. People saying "but what if ChatGPT lies to you!" seem to forget the rest of the internet is also filled with outdated or wrong information. You always still need to actually figure out if the result is correct or not. Heck, even when I come up with a solution myself without needing to research it, I'm still likely to get it wrong or make mistakes. That's why we rely on tests, specs, code review, etc, because humands still make mistakes.

If you don't trust it to write… you probably shouldn't trust it to tell you how to write either.

constantcrying 2y ago

Why? It is clearly better at one thing than the other.

Nullabillity 2y ago
Is it, or are you just having a harder time spotting the errors?
- constantcrying 2y ago
  It seems far superior to Google.
- kragen 2y ago
  llms like gpt-4 are enormously better than google at finding the relevant functions in an api and telling you how to chain them together to accomplish a given task. sometimes there are errors, but in most cases those are easy to spot, especially if you read the documentation for those functions. for some purposes, testing is enough (and it will happily write you a test suite too) it's kind of annoying to ferret out the bugs it has carefully concealed in the code it wrote for you, though
  - consp 2y ago
    > especially if you read the documentation for those functions Shouldn't that be enough in the first place? RTFM is a real thing.
    - kragen 2y ago
      the trouble is that you don't know which functions to read the manual for. last night, for example, i was trying to find out how to get objdump to disassemble an ms-dos .com file with the correct load address of 0x100 rather than 0. google was unhelpful, but gpt-4 told me that the option i was looking for was called `--adjust-vma`. probably if i'd read the objdump manual i would have been able to find it (though the objdump documentation isn't very clear about what `--adjust-vma` will do on a raw binary file), and technically that's only about 15 pages, and spending the requisite half hour reading it would probably produce returns the next time i use objdump. so it's a reasonable idea but it's pretty common for gpt-4 to be able to instantly find the thing you need out of 150 pages or 1500 pages or 15000 pages. or the five things you need, and how to combine them. if you want to know what libraries can probably solve your problem, asking gpt-4 is a much better idea than reading the manuals of all possible libraries
- madeofpalk 2y ago
  Stack Overflow is also full of very confident wrong answers. People saying "but what if ChatGPT lies to you!" seem to forget the rest of the internet is also filled with outdated or wrong information. You always still need to actually figure out if the result is correct or not. Heck, even when I come up with a solution myself without needing to research it, I'm still likely to get it wrong or make mistakes. That's why we rely on tests, specs, code review, etc, because humands still make mistakes.