> With mixture of expert systems we’re introducing dedicated subsystems into the llm responsible for specific aspect of the llm Common misconception, MoEs do h

> With mixture of expert systems we’re introducing dedicated subsystems into the llm responsible for specific aspect of the llm Common misconception, MoEs do have different "experts", but the model learns when to send input to different experts, and the model does not cleanly send coding tasks to the coding agent, physics tasks to the physics agent, etc. It's quite messy, and not nearly as intepretable as we'd want it to be.