3 ms·
This actually gets at the heart of the problem with the current batch of LLMs. They have no concept of truth, and so will gladly generate tokens that are "like
by posnet 4y ago
This actually gets at the heart of the problem with the current batch of LLMs.
They have no concept of truth, and so will gladly generate tokens that are "likely", but are at best clearly wrong, or worse look correct to a non-expert, but are subtly wrong.
With software at least you can run it and inspect the behavior for flaws. Or have the results be validated by an actual programmer for correctness. And hopefully the developer using the LLM understands the limitations, and understands how it can hallucinate incorrect but convincing results.
However even software engineers I know, who are aware of all of the above issues, will ask ChatGPT questions about fields they aren't familiar with and view the results as authoritative. It's the same effect as when you read a news article about a topic you're familiar with and can immediately see flaws, shortcuts and exaggerations; but then you turn the page to a topic you aren't an expert it and say to yourself "what an interesting article".
The example from the article is even worse, since the author asked for an explanation of the code, and it hallucinated a clear and convincing but completely wrong explanation. Again to a programmer it is obvious, but to a layperson it's actively deceptive.
I could easily imagine a scenario where some asks ChatGPT, what is the best combination of chemicals to clean mold from a ceramic surface, and it responding with "Bleach and Vinegar are the perfect way to clean a ceramic surface." and following up with an explanation like "Vinegar de-greases the surface allowing the the disinfectant power of the bleach to penetrate into the mold". Which all sounds reasonable, unless you know beforehand that those chemicals mixed produces toxic chlorine gas.
Now that was definitely a contrived example, but unless you know and constantly question the output of an LLM you could easily be misled. Especially if you are asking about topics outside of it's training data or with minimal training examples.
Maybe with enough training data or some combination of transformers with reinforcement learning that has a "truth" metric, hallucinating completely incorrect information can be reduced to an acceptable level. But at this point it seems intractable.
- polishdude20 4y agoNext thing in the future is to ask the llm to generate code. The llm generates it, runs it, writes tests, confirms they work etc. Now will it write correct tests? Maybe?
- thephyber 4y agoYou seem to be talking about giving the LLM the responsibility to do both strategy and tactics. In my experience, it can be useful in tactics, but usually fails to understand the concepts of strategy. My personal feeling is that an LLM is insufficient for strategy and is not the only technology suitable for implementing the tactics. I think it makes more sense to treat each as a module in a system, and build different modules to compete against each other.
- mikewarot 4y ago>They have no concept of truth 1 - They aren't optimized for truth, they are optimized for best appearing answers. 2 - What things do you know that you are willing to state on HN without being fear of being contradicted?
- posnet 4y agoI know they aren't optimized for truth, that was the point of my comment. But based on my own experience of showing people ChatGPT, without explaining the flaws they tend to treat the output as if it was. I don't understand the second question.
- mikewarot 4y ago2 was about my feeling that pretty much anything that is a "fact" seems to dissolve here on HN. Restated thus: The quickest way to falsify something is to state it as a fact here on HN.
- mistermann 4y agoEven here there are epistemic issues: >> This actually gets at the heart of the problem with the current batch of LLMs. Here you are implying that the fault lies [solely] with LLMs. >> They have no concept of truth, and so will gladly generate tokens that are "likely", but are at best clearly wrong, or worse look correct to a non-expert, but are subtly wrong. Hear you are asserting that you have the means yourself to reach truth. This is extremely easy to do if you're speaking abstractly, but try executing that at the concrete level (as above) and it's pretty difficult to avoid imperfection. Regarding the difficulty of truth, is the problem here entirely with LLMs, or is the problem with reality itself? When ChatGPT (or anyone/anything) says something "is" true (and some people agree) someone else says it "is" not, how are we to decide which is correct? Where does this "is" that people "are [only] perceiving" come from? Where, and what, "is" "reality"? Is the universe equal/identical to reality? That's how a lot of people talk...but is it true? INB4 "we could all be brains in jars", which is one of the most common responses to arise "purely by coincidence" when this topic is raised.
- jondwillis 4y agoI used it to help me wire up some solar panels in series / parallel, and while mostly helpful, it messed up on some math, which with electricity could have resulted in a fire. Luckily I am skeptical and noticed, but I just as easily could have not.