For the experiment in section 5.3, where they try to recover private information from embeddings of clinical notes, it's interesting that the model has to also

For the experiment in section 5.3, where they try to recover private information from embeddings of clinical notes, it's interesting that the model has to also spend capacity trying to reconstruct non-private information. I wonder if you could do better at recovering names by first learning a custom distance metric that tries to assign a low distance to embeddings of texts that share a name, regardless of other content, and then using this method to minimize that distance.