Yes! That would be awesome. Especially since there are ~32*6 independent effort settings for every single token. I tested the most basic implementation, with a

Yes! That would be awesome. Especially since there are ~32*6 independent effort settings for every single token. I tested the most basic implementation, with a flat effort setting for all the muls, but I bet the results could be pushed even further with such an approach. Or even with just doing some ML to figure out which layer/matrix needs more and which less effort.