If you choose to handle PII you have to keep track of where it ends up. Feeding PII into a black box and pretending it isn't there anymore without taking reason

If you choose to handle PII you have to keep track of where it ends up. Feeding PII into a black box and pretending it isn't there anymore without taking reasonable precautions, especially on something like ML that is known to leak its input, doesn't seem like it should be an option. If you don't know the safe assumption should be that the ML model can leak PII and should be destroyed along with the training data.