4 ms·
I was hoping that LLMs being able to access strict tools, like Gemini using Python libraries, would finally give reliable results. So today I asked Gemini to s
by jansan 1y ago
I was hoping that LLMs being able to access strict tools, like Gemini using Python libraries, would finally give reliable results.
So today I asked Gemini to simplify a mathematical expression with sympy. It did and explained to me how some part of the expression could be simplified wonderfully as a product of two factors.
But it was all a lie. Even though I explicitly asked it to use sympy in order to avoid such hallucinations and get results that are actually correct, it used its own flawed reasoning on top and again gave me a completely wrong result.
You still cannot trust LLMs. And that is a problem.
- ogogmad 1y agoThe obvious point has to be made: Generating formal proofs might be a partial fix for this. By contrast, coding is too informal for this to be as effective for it.