4 ms·
Regexps are not great in general – whether something is sensitive or not is highly contextual. Regexps tend to be too narrow and produce a lot of noise = false
by Radim 6y ago
Regexps are not great in general – whether something is sensitive or not is highly contextual. Regexps tend to be too narrow and produce a lot of noise = false positives, so not terribly actionable.
And false negatives (missed info) aren't much better either: sensitive data often lives in images, as people's profile photos or passport or ID scans, where you must really look at the pixels (OCR rarely helps). Here regexps instafail because they assume input is text.
Personal war story: I encountered a bizarre category, sensitive data that's true negative which people nevertheless perceive as true positive. Think "specimen (invalid) passports" or "sample (invalid) credit cards". Not really sensitive (zero risk), but people still expect to see them detected. Creates some funny situations during testing. If you're curious, I wrote about it here [0], with some examples.
[0] https://pii-tools.com/how-to-evaluate-pii-discovery/ https://pii-tools.com/how-to-evaluate-pii-discovery/
- Radim 6y ago> Regexps are not great in general …not great for sensitive data detection in general, of course. Hopefully clear from the context (ha!).
- donbowman 6y agoThe anti-virus industry created the EICAR tame virus for this purpose. https://www.eicar.org/?page_id=3950 https://www.eicar.org/?page_id=3950 its mean to match, not be evil, and be in a separate category. so maybe we need an EICAR for PII?