I doubt it. There’s no reason to suppose that any configuration of the parameters is any more or less susceptible to this type of attack. If the paper I cited i

I doubt it. There’s no reason to suppose that any configuration of the parameters is any more or less susceptible to this type of attack. If the paper I cited is true, then the “thing” transformer-based language models learn is a function to predict the gradient update via the forward pass evaluation. That means that forward evaluation is an approximation of fine-tuning. That implies that the very act of evaluating an input changes the behavior of the model, regardless of the current parameter settings. The model would have to “give up” the powers of in-context learning in order to avoid this weakness.