4 ms·

> "They also demonstrated a machine-learning model that helps detect tampering by identifying similar glue pattern fingerprints with more than 99 percent accura

by gcr 3y ago

> "They also demonstrated a machine-learning model that helps detect tampering by identifying similar glue pattern fingerprints with more than 99 percent accuracy." This system would probably have to be far better than 99% to be helpful. If customers test (say) a hundred thousand tags per day, we might suppose that 99% accuracy means a thousand false alarms per day. If users get suspicious and return their genuine devices upon seeing a false alarm, that's a huge cost, even without any fakes at all. Reporting "99% accuracy" allows a worrisome amount of luck into a system marketed as "tamper-proof." Let's compare to biometric verification. Apple's TouchID claims about 1 false accept among 50,000 fake fingers (FAR=0.00002) with false rejection rates (FRR) hovering around 2%-10% or so. Even though this averages to worse than 99% "overall accuracy," it makes sense for a fingerprint matcher because the genuine user gets to retry their finger a few seconds later, so high FRR isn't a problem in practice. On the other hand, it may be better for these tags to trade off a higher FAR for a lower FRR. I'm also worried by the article's claim that terahertz waves "travel much faster than radio waves." The speed of light doesn't depend on its frequency.

borispavlovic 3y ago
I read 99% percent accuracy in a given time. For 1% the algorithm could ask the user for patience and perform a "deeper" search increasing its overall accuracy.
pimterry 3y ago
> This system would probably have to be far better than 99% to be helpful. This article does talk about this specifically: > “One drawback is that we had a limited data sample for this demonstration, but we could improve the neural network in the future if a large number of these tags were deployed in a supply chain, giving us a lot more data samples,” Lee says. I.e. they're well aware that 99% would need to be improved, but it sounds like they've managed to hit that level with a handful of individually created samples and a tiny project to train a model. This is just a prototype really, and 99% accuracy when testing a prototype doesn't seem unreasonable, it's not like they're planning to ship this tomorrow. It's a demonstration that recognizing these patterns automatically is actually possible in the first place, and in a serious roll-out scenario it would _probably_ be possible to train a model to do very significantly better.
roarcher 3y ago
To be fair it says "more than 99%", which may just be popsci-journalist-speak for "very accurate". This is the same author who says terahertz waves "travel much faster than radio waves".
AceJohnny2 3y ago
Indeed. Journalistic leeway aside: I work close to manufacturing, and at our scale we measure things in DPPM (Defective Parts Per Million (manufactured)). In my line of work, defect rates in the tens to low hundred DPPM (10 - 100 / 1000000) is noise, high hundreds is cause for attention. When failures approach the Per Cent-age rate, that's equivalent to ~10k DPPM, and that's daily stand-up territory.
- user_7832 3y ago
  Could you share in terms of defective parts per day (or hour), how much would that be? Or roughly how many parts were being made daily? (Reason for asking is that this metric could be partially extrapolated to high-quality low-scale manufacturing as well.)
  - hef19898 3y ago
    DPMO lives from large numbers, extrapolaiting it to low scale manufacturing would extend measurement periods to point the KPI becomes useless.
    - user_7832 3y ago
      It might not be very helpful in many cases, but can still be an "ideal" reference/target especially from the point of view of someone manufacturing at a a small scale.
m463 3y ago
I think this could have lots of novel ways of being accurate. it might not be like biometrics, because it could have an identifier repeated 100 times, and then the glue/tampering part could be a completely different sort of algorithm that could be optimized to physical removal, not just degradation.
deleted 3y ago
[deleted]
danvoell 3y ago
The interesting part is it’s not the tag. It’s the adhesive behind the tag. I agree 99% is not good enough. But if it’s some sort of backup verification to the tag itself, at no extra cost, there might be something.

by gcr 3y ago

borispavlovic 3y ago

I read 99% percent accuracy in a given time. For 1% the algorithm could ask the user for patience and perform a "deeper" search increasing its overall accuracy.

pimterry 3y ago

> This system would probably have to be far better than 99% to be helpful. This article does talk about this specifically: > “One drawback is that we had a limited data sample for this demonstration, but we could improve the neural network in the future if a large number of these tags were deployed in a supply chain, giving us a lot more data samples,” Lee says. I.e. they're well aware that 99% would need to be improved, but it sounds like they've managed to hit that level with a handful of individually created samples and a tiny project to train a model. This is just a prototype really, and 99% accuracy when testing a prototype doesn't seem unreasonable, it's not like they're planning to ship this tomorrow. It's a demonstration that recognizing these patterns automatically is actually possible in the first place, and in a serious roll-out scenario it would _probably_ be possible to train a model to do very significantly better.

roarcher 3y ago

To be fair it says "more than 99%", which may just be popsci-journalist-speak for "very accurate". This is the same author who says terahertz waves "travel much faster than radio waves".

AceJohnny2 3y ago

Indeed. Journalistic leeway aside: I work close to manufacturing, and at our scale we measure things in DPPM (Defective Parts Per Million (manufactured)). In my line of work, defect rates in the tens to low hundred DPPM (10 - 100 / 1000000) is noise, high hundreds is cause for attention. When failures approach the Per Cent-age rate, that's equivalent to ~10k DPPM, and that's daily stand-up territory.

user_7832 3y ago
Could you share in terms of defective parts per day (or hour), how much would that be? Or roughly how many parts were being made daily? (Reason for asking is that this metric could be partially extrapolated to high-quality low-scale manufacturing as well.)
- hef19898 3y ago
  DPMO lives from large numbers, extrapolaiting it to low scale manufacturing would extend measurement periods to point the KPI becomes useless.
  - user_7832 3y ago
    It might not be very helpful in many cases, but can still be an "ideal" reference/target especially from the point of view of someone manufacturing at a a small scale.

m463 3y ago

I think this could have lots of novel ways of being accurate. it might not be like biometrics, because it could have an identifier repeated 100 times, and then the glue/tampering part could be a completely different sort of algorithm that could be optimized to physical removal, not just degradation.

deleted 3y ago

[deleted]

danvoell 3y ago

The interesting part is it’s not the tag. It’s the adhesive behind the tag. I agree 99% is not good enough. But if it’s some sort of backup verification to the tag itself, at no extra cost, there might be something.