Obviously a sensationalist title, but a lot of details that seem true. I wonder if the open source models are going to go this route, imagine some sort of small

Obviously a sensationalist title, but a lot of details that seem true. I wonder if the open source models are going to go this route, imagine some sort of smaller, but still much more scaled/capable MoE model derived from the LLaMA models.