The problem always was that the improvements worsened other cases. Mixture of experts tends to lock models up. Chain of thought tends to turn the models loopy.

The problem always was that the improvements worsened other cases. Mixture of experts tends to lock models up. Chain of thought tends to turn the models loopy. As in circling the drain even when asked not to. Tree of thought has a tendency to make the models unstable by flipping between the branches, and otherwise unpredictable complicating training ...