3 ms·
You are getting downvoted but you are right, a typo in a variable that already exists in a file like this is not the failure mode for LLMs. The failure mode is
by brigadier132 2y ago
You are getting downvoted but you are right, a typo in a variable that already exists in a file like this is not the failure mode for LLMs. The failure mode is logic bugs, making up methods / functions.
- varjag 2y agoNo, you can get variable names "mutated" on follow up requests. The thing is like sculpting with toothpaste.
- brigadier132 2y agoI've been using copilot for as long as it has existed and what you are describing has not happened to me once. Literally on in the background 8 hours a day. Excuse me for not trusting the internet hivemind that hates everything that is hyped just a little bit.
- varjag 2y agoMy goto check of AI assistants is asking to write a function calculating the first N digits of Pi in Common Lisp. On at least two attempts when prompted to fix its code the model would change one of the variable names to T, which is a reserved symbol. So yeah pretty sure it does happen.
- deleted 2y ago[deleted]