7 ms·
If your tool regularly lies, gaslights and produces wrong results, that's a tooling issue. It's a human issue if you don't recognise that the code it's generat
by onion2k 6mo ago
If your tool regularly lies, gaslights and produces wrong results, that's a tooling issue.
It's a human issue if you don't recognise that the code it's generated is wrong. That will never change no matter how good the tooling gets.
- kiba 6mo agoThe tooling is the issue because humans designed the tooling wrong. It's a chatbot interface fined tuned to sycophancy. That's not a coincidence.
- Hamuko 6mo agoIsn't part of the problem that these tools are advertised as allowing non-coders to code? How are you gonna recognise that the code is wrong when you don't know how to code and the product is telling you that you don't even need to?
- drw85 6mo agoWould it be a human issue, if you type something into a calculator and the calculated result is wrong? Would anyone use a calculator confidently, if the result was randomly generated?
- allarm 6mo agoWhy do you think this analogy is even remotely correct? It’s well-known that LLMs produce non-deterministic results. It’s also well-known that they hallucinate. To make it even clearer, all the top LLM players make sure to remind everyone of that behavior. If calculators had similar effects and warnings, it would have been a valid analogy. Instead, you're comparing apples and oranges.
- drw85 6mo agoI think it's a valid analogy in some contexts. Like when talking to a person that is not aware of non-determinism and hallucinations. Which happens on this website very frequently. Many people here tell you to use AI like you use a calculator. With minimal or no oversight, with full access to production systems, etc. To let a non deterministic tool communicate on your behalf, or give it access to critical systems is evidence enough that a good number of people are not aware of these facts.