4 ms·

There's a big difference between "build this program for me" and "explain how this line of bash code works". I use LLMs for the latter, and they often do a gre

There's a big difference between "build this program for me" and "explain how this line of bash code works". I use LLMs for the latter, and they often do a great job and save a lot of time looking up individual flags, concepts, etc.

Too 3y ago
Both could be equally wrong and hallucinated.
- thedaly 3y ago
  > Both could be equally wrong and hallucinated. As can I. I recently broke my years long streak of not deleting something by accident with an errant bash command. Just because a tool has the potential for a negative outcome doesn't mean it shouldn't be used. It just means appropriate caution should be used as well.
- williamcotton 3y ago
  This statement implies that LLM hallucinations are completely random which is objectively false. LLMs fill in the blanks when left to synthesize a response from a prompt as opposed to translating a response from a prompt. These synthesized responses, aka hallucinations, are predictable in nature. Quotes, titles of books, web page links, etc. Conversely, providing an LLM with all of the facts necessary to complete a response will result in few to no hallucinations. For example: Select name and row_id from table1 joined on table2 on table1_id. This will never return "DROP table1;". It will basically only ever return something very close to what you want.
  - williamcotton 3y ago
    Why the downvotes? The comment is not rude and it is factually correct.
  - Too 3y ago
    Not my downvotes but this is factually incorrect. A LLM will give you the highest likely suggestion. If that happens to be a DROP, it will not stop. Now that is of course going to be extremely unlikely in your example. What is more likely though is that your SELECT may include a sql injection vulnerability, even more so once your prompts get more complex. The chance of that happening or not, is completely random from a users point of view. Are we going to blame the user for not providing the requirement “without vulnerabilities”? Even if they did, it’s not sure to be fulfilled. In this parent case, the scenario was inverted. Given a sql query, will gpt explain if it has vulnerabilities or not? Will it even explain the gist of it correct? Who knows if it will hallucinate or not? As will answers from stackoverflow, always read the comments, always review yourself. Use gpt all you want. I do it it myself, it’s great for suggestions. Just think that using gpt to explain things you don’t understand and can’t verify easily, can be risky. Even more so in bash where the difference making a destructive command can be a lot more subtle than select vs drop.
    - williamcotton 3y ago
      Now that is of course going to be extremely unlikely in your example. The OpenAI API now has support for deterministic responses. There you go, the burden of proof is on the accuser. If I were to state “you can never ride your bicycle to the moon”, you could easily say, well, there is a remote possibility, and then force me to prove that there actually is no remote possibility, well, you would clearly see the problem. I’ll state it again: you will never ride your bicycle to the moon and ChatGPT will never return “DROP table1;” in response to the aforementioned request. It might not be correct, but it won’t be wildly off target like is flippantly suggested in these forums for populist appeal. My entire point was that hallucinations are not random. If you craft a query that reduces the task to mere translation then you will not get some wildly incorrect response like you would if you asked for quotes from War and Peace. I’m pretty much convinced that most of the shade against LLMs from developers is motivated more by emotion than reason because this stuff is easily verifiable. To not have realized this means approaching the tools willingly blindfolded!
      - Too 3y ago
- xcv123 3y ago
  So what? Review and test it before running in production. Humans also screw things up on first attempts. Test it, check the output, fix it, try again. In my experience with GPT-4 if you describe a bug it will correct it.

Too 3y ago

Both could be equally wrong and hallucinated.

thedaly 3y ago
> Both could be equally wrong and hallucinated. As can I. I recently broke my years long streak of not deleting something by accident with an errant bash command. Just because a tool has the potential for a negative outcome doesn't mean it shouldn't be used. It just means appropriate caution should be used as well.
williamcotton 3y ago
This statement implies that LLM hallucinations are completely random which is objectively false. LLMs fill in the blanks when left to synthesize a response from a prompt as opposed to translating a response from a prompt. These synthesized responses, aka hallucinations, are predictable in nature. Quotes, titles of books, web page links, etc. Conversely, providing an LLM with all of the facts necessary to complete a response will result in few to no hallucinations. For example: Select name and row_id from table1 joined on table2 on table1_id. This will never return "DROP table1;". It will basically only ever return something very close to what you want.
- williamcotton 3y ago
  Why the downvotes? The comment is not rude and it is factually correct.
- Too 3y ago
  Not my downvotes but this is factually incorrect. A LLM will give you the highest likely suggestion. If that happens to be a DROP, it will not stop. Now that is of course going to be extremely unlikely in your example. What is more likely though is that your SELECT may include a sql injection vulnerability, even more so once your prompts get more complex. The chance of that happening or not, is completely random from a users point of view. Are we going to blame the user for not providing the requirement “without vulnerabilities”? Even if they did, it’s not sure to be fulfilled. In this parent case, the scenario was inverted. Given a sql query, will gpt explain if it has vulnerabilities or not? Will it even explain the gist of it correct? Who knows if it will hallucinate or not? As will answers from stackoverflow, always read the comments, always review yourself. Use gpt all you want. I do it it myself, it’s great for suggestions. Just think that using gpt to explain things you don’t understand and can’t verify easily, can be risky. Even more so in bash where the difference making a destructive command can be a lot more subtle than select vs drop.
  - williamcotton 3y ago
    Now that is of course going to be extremely unlikely in your example. The OpenAI API now has support for deterministic responses. There you go, the burden of proof is on the accuser. If I were to state “you can never ride your bicycle to the moon”, you could easily say, well, there is a remote possibility, and then force me to prove that there actually is no remote possibility, well, you would clearly see the problem. I’ll state it again: you will never ride your bicycle to the moon and ChatGPT will never return “DROP table1;” in response to the aforementioned request. It might not be correct, but it won’t be wildly off target like is flippantly suggested in these forums for populist appeal. My entire point was that hallucinations are not random. If you craft a query that reduces the task to mere translation then you will not get some wildly incorrect response like you would if you asked for quotes from War and Peace. I’m pretty much convinced that most of the shade against LLMs from developers is motivated more by emotion than reason because this stuff is easily verifiable. To not have realized this means approaching the tools willingly blindfolded!
    - Too 3y ago
xcv123 3y ago
So what? Review and test it before running in production. Humans also screw things up on first attempts. Test it, check the output, fix it, try again. In my experience with GPT-4 if you describe a bug it will correct it.