seems like a large undertrained model, not that exciting imo compared to mixtral it is also not the biggest model oss, switch transformer was released years ag

seems like a large undertrained model, not that exciting imo compared to mixtral it is also not the biggest model oss, switch transformer was released years ago and is larger and similarly undertrained