There is a new way to train diffusion models to insert tokens between existing tokens rather than unmasking <mask> tokens: https://openreview.net/forum?id=VbvXj

There is a new way to train diffusion models to insert tokens between existing tokens rather than unmasking <mask> tokens: https://openreview.net/forum?id=VbvXjs5f72 https://openreview.net/forum?id=VbvXjs5f72 However, I believe this would "only" be able to insert tokens, not to delete tokens again it mistakenly produced before. (The deletion in the title refers to the reverse process during training, where tokens are progressively deleted rather than masked.)