DeepMind recently extracted PII from ChatGPT by prompting (e.g., telling the LLM to repeat 'poem' indefinitely will cause a long sequence of that word until pop

DeepMind recently extracted PII from ChatGPT by prompting (e.g., telling the LLM to repeat 'poem' indefinitely will cause a long sequence of that word until popping out of it and revealing by accident some PII from a person's email signature). So, yes.