2 ms·

I guess the question is how the mix of experts works. Do they predict one token from each model? If so you’re still doing 1.7T worth of computation.