3 ms·
A rant on LLaMA: please stop training giant language models
- whywhywhywhy 4y ago> I don’t believe, for instance, that OpenAI is really using a 175B parameter model for ChatGPT or for their other language products (here is an interesting analysis I saw recently that makes the same point, with some caveats), but they have an incentive for making people believe that they’re using a 175B parameter model and that it’s actually critical to use a giant model like that. We saw this with Dall-E2, there was an incentive to make it seem impossible for this tech to be ran or even safe to be ran outside of the OpenAI walls. This is in part the claims of size and power required but also clever use of safety theatre to make it seem too dangerous for you to be allowed to use it while they claima business advantage. I believe the same will happen with language models, they will have their stable diffusion moment too when someone is incentivized to make something that would work for the masses rather than incentivized to lock up the value.