3 ms·

No-one should have the expectation LLMs are giving correct answers 100% of the time. It's inherent to the tech for them to be confidently wrong Code needs to b

by AstroBen 10mo ago

No-one should have the expectation LLMs are giving correct answers 100% of the time. It's inherent to the tech for them to be confidently wrong Code needs to be checked References need to be checked Any facts or claims need to be checked

malfist 10mo ago
According to the benchmarks here they're claiming up to 97% accuracy. That ought to be good enough to trust them right? Or maybe these benchmarks are all wrong
- AstroBen 10mo ago
  Does code work if it's 97% correct? It's not okay if claims are totally made up 1/30 times Of course people aren't always correct either, but we're able to operate on levels of confidence. We're also able to weight others' statements as more or less likely to be correct based on what we know about them
  - fooker 10mo ago
    > Does code work if it's 97% correct? Of course it does. The vast majority of software has bugs. Yes, even critical one like compilers and operating systems.
- refactor_master 10mo ago
  Gemini routinely makes up stuff about BigQuery’s workings. “It’s poorly documented”. Well, read the open source code, reason it out. Makes you wonder what 97% is worth. Would we accept a different service with only 97% availability, and all downtime during lunch break?
  - TeMPOraL 10mo ago
    I.e. like most restaurants and food delivery? :). Though 3% problem rate is optimistic.
- JimDabell 10mo ago
  Something that is 97% accurate is wrong 3% of the time, so pointing out that it has gotten something wrong does not contradict 97% accuracy in the slightest.
- mbesto 10mo ago
  > Or maybe these benchmarks are all wrong You must be new to LLM benchmarks.
dolmen 10mo ago
"confidently" is a feature selected in the system prompt. As a user you can influence that behavior.
- malfist 10mo ago
  No it isn't. It isn't intelligent, it's a statistical engine. Telling it to be confident or less confident doesn't make it apply confidence appropriately. It's all a facade

malfist 10mo ago

According to the benchmarks here they're claiming up to 97% accuracy. That ought to be good enough to trust them right? Or maybe these benchmarks are all wrong

AstroBen 10mo ago
Does code work if it's 97% correct? It's not okay if claims are totally made up 1/30 times Of course people aren't always correct either, but we're able to operate on levels of confidence. We're also able to weight others' statements as more or less likely to be correct based on what we know about them
- fooker 10mo ago
  > Does code work if it's 97% correct? Of course it does. The vast majority of software has bugs. Yes, even critical one like compilers and operating systems.
refactor_master 10mo ago
Gemini routinely makes up stuff about BigQuery’s workings. “It’s poorly documented”. Well, read the open source code, reason it out. Makes you wonder what 97% is worth. Would we accept a different service with only 97% availability, and all downtime during lunch break?
- TeMPOraL 10mo ago
  I.e. like most restaurants and food delivery? :). Though 3% problem rate is optimistic.
JimDabell 10mo ago
Something that is 97% accurate is wrong 3% of the time, so pointing out that it has gotten something wrong does not contradict 97% accuracy in the slightest.
mbesto 10mo ago
> Or maybe these benchmarks are all wrong You must be new to LLM benchmarks.

dolmen 10mo ago

"confidently" is a feature selected in the system prompt. As a user you can influence that behavior.

malfist 10mo ago
No it isn't. It isn't intelligent, it's a statistical engine. Telling it to be confident or less confident doesn't make it apply confidence appropriately. It's all a facade