Many MoE models like Mixtral were not created from scratch. Somebody could still upcycle that 8B into an 8x8B. It should ideally be done by Meta since they have

Many MoE models like Mixtral were not created from scratch. Somebody could still upcycle that 8B into an 8x8B. It should ideally be done by Meta since they have access to the original training data.