Haha halfwit! I’m waiting for such a fork. As for predicting the weights - not necessarily so. It seems most weights are being used, just not all the time. Kin

Haha halfwit! I’m waiting for such a fork. As for predicting the weights - not necessarily so. It seems most weights are being used, just not all the time. Kind of like that saying that humans are using just 5% of their brain - perhaps they are, but it’s various parts of the 5%. Interestingly, Effort works just as well on MoE, if not better. I did most of the development on Mixtral and I think it go even to 15-20% effort before losing quality, but there is some sort of a bug right now that prevents the inference on Mixtral. It’s on a todo to fix, but I didn’t want to delay the release because of it.