I believe you are correct, I worked on a summer research project at NYU in 2018 based on https://arxiv.org/abs/1805.12185 https://arxiv.org/abs/1805.12185 As p

I believe you are correct, I worked on a summer research project at NYU in 2018 based on https://arxiv.org/abs/1805.12185 https://arxiv.org/abs/1805.12185 As part of that project I constructed an API that took a small dataset and a model, launched a K8s pod and ran something like this from the paper: > The pruning defense works as follows: the defender exercises the DNN received from the attacker with clean inputs from the validation dataset, D_valid, and records the average activation of each neuron. The defender then iteratively prunes neurons from the DNN in increasing order of average activations and records the accuracy of the pruned network in each iteration. The defense terminates when the accuracy on the validation dataset drops below a pre-determined threshold. We note that pruning has been proposed in prior work for n Obviously this wasn't on transformers but the idea is similar.