The number of weights(unless extremely small) is irrelevant but the general idea is not. You can't train a neural network on internet scale data and expect to c

The number of weights(unless extremely small) is irrelevant but the general idea is not. You can't train a neural network on internet scale data and expect to control what it can say. We can train but we don't teach them anything. They learn from data directly and we don't know or understand what they learn so we can't adjust what they learn directly. You can't make "always obey these types of tokens" a part of the architecture or training. It's a concept that doesn't even make sense for the vast majority of text it pre-trains on. "Solving" prompt injection is solving alignment. It's not happening.