4 ms·
Are there any strong LLMs trained without CUDA?
by robrenaud 2y ago
Are there any strong LLMs trained without CUDA?
- blackeyeblitzar 2y agoYes, there are several. See this blog post from Databricks describing the landscape of LLMs trained on AMD hardware for example: https://www.databricks.com/blog/training-llms-scale-amd-mi250-gpus https://www.databricks.com/blog/training-llms-scale-amd-mi25... The most interesting one IMO is OLMo from AI2, which is truly open. You can read their blog post about it (https://blog.allenai.org/hello-olmo-a-truly-open-llm-43f7e7359222 https://blog.allenai.org/hello-olmo-a-truly-open-llm-43f7e73...) but basically it is open everything - they released everything you need to reproduce their weights (training data, training code, evaluation code, and weights) with a friendly (Apache) license.
- ZoomerCretin 2y agoGemini and Gemma were trained on Google's TPUs.