3 ms·

I like to think of the intermediate tokens as low-dimensional hidden states. Also see the Coconut paper/citation