3 ms·
What's interesting to me is how many of these advancements are just obvious next steps for these tools. Chain of thought, tree of thought, mixture of experts e
by CSMastermind 1y ago
What's interesting to me is how many of these advancements are just obvious next steps for these tools. Chain of thought, tree of thought, mixture of experts etc. are things you'd come up with in the first 10 minutes of thinking about improving LLMs.
Of course the devil's always in the details and there have been real non-obvious advancements at the same time.
- AstralStorm 1y agoThe problem always was that the improvements worsened other cases. Mixture of experts tends to lock models up. Chain of thought tends to turn the models loopy. As in circling the drain even when asked not to. Tree of thought has a tendency to make the models unstable by flipping between the branches, and otherwise unpredictable complicating training ...