Check out Unsloths REAP models, you can outright delete a few of the lesser used experts without the model going braindead since they all can handle each token

Check out Unsloths REAP models, you can outright delete a few of the lesser used experts without the model going braindead since they all can handle each token but some are better posed to do so.