I've experimented with building diffusion-based LLMs but didn't scale them up enough to draw any worthwhile conclusions about their strength vs transformer-base

I've experimented with building diffusion-based LLMs but didn't scale them up enough to draw any worthwhile conclusions about their strength vs transformer-based LLMs. In this case however, theoretical ability to unminify a piece of code mainly comes down to the training set and amount of relevant context baked into the system, not the model architecture itself.