3 ms·
Funny thing is the structured output in the last example. ``` { "reasoning": "`cfg` can be nil on line 42; dereferenced without check on line 47", "finding
by elzbardico 1y ago
Funny thing is the structured output in the last example.
```
{
"reasoning": "`cfg` can be nil on line 42; dereferenced without check on line 47",
"finding": "Possible nil‑pointer dereference",
"confidence": 0.81
}
```
You know the confidence value is completely bogus, don't you?
- MattSayar 1y agoCould you have a higher-order reasoning LLM generate a better confidence rating? That's how eval frameworks generally work today
- sharkjacobs 1y agoDo you mean that there is no correlation between confidence and false positives or other errors?
- ramity 1y agoelzbardico is pointing out how the author is having the confidence value generated in the output of the response rather than it being the confidence of the output.
- ramity 1y agoI too once fell into the trap of having an LLM generate a confidence value in a response. This is a very genuine concern to raise.
- munificent 1y agoEasy fix, just have the LLM generate: { "reasoning": "`cfg` can be nil on line 42; dereferenced without check on line 47", "finding": "Possible nil‑pointer dereference", "confidence": 0.81, "confidence_in_confidence_rating": 0.54, "confidence_in_confidence_rating_in_confidence_rating": 0.12, "confidence_in_confidence_rating_in_confidence_rating_in_confidence_rating": 0.98, // Etc... }
- zengid 1y agoconfidence all the way down
- GardenLetter27 1y agoConfidence is all you need.
- lgas 1y agoTrue in many situations in life.
- paisawalla 1y agoWasteful. `confidence`'s type should be Array<number>, wherein confidence[N] gives the Nth derivative confidence rating.
- volkk 1y agoi immediately noticed the same thing, but to be fair, we don't know if it's enriched by a separate service that checks the response and uses some heuristics to compute that value. If not, yeah, that is an entirely made up and useless value
- skipants 1y agoWhen I was younger and more into music, when I went to a concert I would often judge if a drummer was "good" based on if they were better than me or not. I knew enough about drumming to tell how good someone was at the different parts of having that skill but also knew enough to know that I was not even close to having what it took to be a professional drummer. This is what I feel like with this blogpost. I've barely scratched the surface of the innards of LLMs but even I know it should be completely obvious to anyone that has a product built around it that these confidence levels are completely made up. I've never heard or used cubic before today but that part of the blog post, along with the obvious LLM generated quality of it, gives a terrible first impression.
- baby 1y agoyou know everything is made up right? And yet it just works. I too use a confidence score in an bug finder app, Github seems to use them in copilot reviews, people will use them until it is shown not to work anymore. on the other hand this post https://www.greptile.com/blog/make-llms-shut-up https://www.greptile.com/blog/make-llms-shut-up says that it didn't work in their case: > Sadly, this also failed. The LLMs judgment of its own output was nearly random. This also made the bot extremely slow because there was now a whole new inference call in the workflow.