I wonder whether the "data dimension" from https://transformer-circuits.pub/2023/toy-double-descent/index.html#comment-mnist https://transformer-circuits.pub/20

I wonder whether the "data dimension" from https://transformer-circuits.pub/2023/toy-double-descent/index.html#comment-mnist https://transformer-circuits.pub/2023/toy-double-descent/ind... could be used to identify the model parameters involved in memorization and remove them without having to retrain from scratch on a cleaned-up dataset.