3 ms·
Is a logging service/pipeline that removes* PII the only real high-level solution? (* or 'does its best' to remove PII; I feel like there's an immovable object
by adamisom 4y ago
Is a logging service/pipeline that removes* PII the only real high-level solution?
(* or 'does its best' to remove PII; I feel like there's an immovable object / unstoppable force thing if the answer is "0% PII _ever_!" because that conflicts with the need to log as many things as you can)
- deleted 4y ago[deleted]
- 0xbadcafebee 4y agoYou shouldn't log as many things as you can. You should only log useful things, when they're needed, along with context. Otherwise you have too much noise and not enough signal, and it's easier to leak PII. Applications should have a logging interface that can identify and conceal sensitive information, and data stored in the app/database/etc should have a type that can be denoted as sensitive. You will eventually also want to reduce the amount of logs you generate, so it's useful to have logging levels the same way operating systems do.
- bob1029 4y agoWe do explicit redaction of logs forwarded to us so that we don't have to see any end-customer information. The redaction process occurs automatically in the customer's secure environment, so theoretically it is clean from our perspective. This is indeed a very tricky process, but we have it close enough to make regulators happy. Most of our log information is stored in SQLite and XML, so we can do a lot of parsing magic to achieve determinism.