That makes sense especially since they're not intending to deploy this model to production. For models like GPT-3/4 it makes sense why they would train them mor

That makes sense especially since they're not intending to deploy this model to production. For models like GPT-3/4 it makes sense why they would train them more because the costs of running the inference "in production" likely dominates the compute costs. (Just like how Youtube will spend 50x more compute to compress a video an extra 2% because bandwidth costs far outstrip the compression costs.) Do you know what percentage, roughly, this model has been trained relative to something like LLaMA? Are we talking 10%? 50%? 90%? It may be possible that it is still useful if it can be trained further by the community!