Weak "chain-of-thought" abilities, high error rates and a very bad ability to recover from errors. If they produce a non-sequitur somewhere (which they are high

Weak "chain-of-thought" abilities, high error rates and a very bad ability to recover from errors. If they produce a non-sequitur somewhere (which they are highly prone to doing), that global refinement spreads it everywhere like a blood infection superhighway. Diffusion text models are cool, but they're functionally much less reliable than autoregressive transformers... and man that's really saying something. Right now most research on them is trying to figure out what complementary systems they need to be reasonably useful.