> I'm curious how frequently the model actually uses this backspace token in practice For it to use the backspace, wouldn't it have to predict the wrong token

> I'm curious how frequently the model actually uses this backspace token in practice For it to use the backspace, wouldn't it have to predict the wrong token with greater confidence than the corrected token? I would think this would require more examples of a wrong token + correction than the correct token, which seems a bit odd. I wonder if this could be have value for when a forward and reverse word predictors collide/fight. I naively assume reverse word predictor models will have the ability to work towards goals, by working from the "solution" back, with the forward word predictor acting as some sort of "causal resolution".