Saw this, it looks very cool. What I was wondering was how different the rest of this model is from other llms, in that it appears to have some kind of RL layer

Saw this, it looks very cool. What I was wondering was how different the rest of this model is from other llms, in that it appears to have some kind of RL layer that is performing the backspacing - does that sit seamlessly onto an otherwise standard llm or are there bigger architectural differences?