5 ms·

Am I missing something or in this image, is the confidence score 99.23% and has the incorrect license plate? The plates clearly have a 5 but it's predicting an

by giarc 6y ago

Am I missing something or in this image, is the confidence score 99.23% and has the incorrect license plate? The plates clearly have a 5 but it's predicting an S. https://camo.githubusercontent.com/5138d6bce5c840d15e533db6f300660c80eb2619/68747470733a2f2f692e696d6775722e636f6d2f746731504531452e6a7067 https://camo.githubusercontent.com/5138d6bce5c840d15e533db6f...

hundchenkatze 6y ago
Nope, you are correct. Wow, both examples from that image are wrong.
IncRnd 6y ago
99.23% is not 100%. 1/130ish But, the wrong examples don't seem a good demonstration of that...
cptskippy 6y ago
Is that confidence for the value or the category. It comes after LP in the image which makes me think it's the confidence that it is a license plate, not necessarily the confidence of the reading of said license plate.
- giarc 6y ago
  You might be right. The gif on github, at times, shows no plate prediction but still includes a confidence score. I would suspect that if it can't even show a prediction, it's confidence score wouldn't show either (being that it would be 0%).
  - cptskippy 6y ago
    > I would suspect that if it can't even show a prediction, it's confidence score wouldn't show either (being that it would be 0%). Recognizing that something is a license place, and being able to read the plate are mutually exclusive things. People can identify license plates even if they can't read them, and people who don't know what a license plate is my still be able to read the numbers on them. The ML has to do two things; the first is reject anything that isn't a License Plate, the second is OCR License Plates.
cochne 6y ago
Typical model output is not actually a probability, but a mostly arbitrary number between 0 and 1. Models usually have to be calibrated to match probability.
blackcat201 6y ago
For neural networks unless it's calibrated otherwise the confidence score has little meaning to the accuracy. [1] see figure 4 and you will understand what I mean [1] https://arxiv.org/pdf/1912.03263.pdf https://arxiv.org/pdf/1912.03263.pdf
Aperocky 6y ago
The 5 prediction probably had a confidence score of 98.52% or something, and the model chose the one it believed to be most likely.
- lultimouomo 6y ago
  I don't know anything about neural networks, but the fact that you can have two incompatible predictions each with more than 50% confidence tells me that confidence is not what my intuition would predict (with 87% confidence)
teruakohatu 6y ago
It is really confident in its answer, but that does tell you much about how likely the answer is correct. For example, if I built a model that always returns the same plate number, it would be 100% confident, but wrong 1947791 times out of every 1947792 plates it sees. That said I would have thought confidence would be lower for plates with 5 and S in it.
implements 6y ago
ANPR Tip: Correlate between successive images to improve confidence, and use (if available eg in the UK) legal plate format information to rule out unlikely characters. Also, humans aren’t perfect at reading plates from highway footage still images - which is why ANPR systems are never 100% accurate.

by giarc 6y ago

hundchenkatze 6y ago

Nope, you are correct. Wow, both examples from that image are wrong.

IncRnd 6y ago

99.23% is not 100%. 1/130ish But, the wrong examples don't seem a good demonstration of that...

cptskippy 6y ago

Is that confidence for the value or the category. It comes after LP in the image which makes me think it's the confidence that it is a license plate, not necessarily the confidence of the reading of said license plate.

giarc 6y ago
You might be right. The gif on github, at times, shows no plate prediction but still includes a confidence score. I would suspect that if it can't even show a prediction, it's confidence score wouldn't show either (being that it would be 0%).
- cptskippy 6y ago
  > I would suspect that if it can't even show a prediction, it's confidence score wouldn't show either (being that it would be 0%). Recognizing that something is a license place, and being able to read the plate are mutually exclusive things. People can identify license plates even if they can't read them, and people who don't know what a license plate is my still be able to read the numbers on them. The ML has to do two things; the first is reject anything that isn't a License Plate, the second is OCR License Plates.

cochne 6y ago

Typical model output is not actually a probability, but a mostly arbitrary number between 0 and 1. Models usually have to be calibrated to match probability.

blackcat201 6y ago

For neural networks unless it's calibrated otherwise the confidence score has little meaning to the accuracy. [1] see figure 4 and you will understand what I mean [1] https://arxiv.org/pdf/1912.03263.pdf https://arxiv.org/pdf/1912.03263.pdf

Aperocky 6y ago

The 5 prediction probably had a confidence score of 98.52% or something, and the model chose the one it believed to be most likely.

lultimouomo 6y ago
I don't know anything about neural networks, but the fact that you can have two incompatible predictions each with more than 50% confidence tells me that confidence is not what my intuition would predict (with 87% confidence)

teruakohatu 6y ago

It is really confident in its answer, but that does tell you much about how likely the answer is correct. For example, if I built a model that always returns the same plate number, it would be 100% confident, but wrong 1947791 times out of every 1947792 plates it sees. That said I would have thought confidence would be lower for plates with 5 and S in it.

implements 6y ago

ANPR Tip: Correlate between successive images to improve confidence, and use (if available eg in the UK) legal plate format information to rule out unlikely characters. Also, humans aren’t perfect at reading plates from highway footage still images - which is why ANPR systems are never 100% accurate.