3 ms·
They have the scale in terms of hardware, but they likely don't have the infrastructure to experiment efficiently on this scale on LLMs specifically. OpenAI has
by loveparade 3y ago
They have the scale in terms of hardware, but they likely don't have the infrastructure to experiment efficiently on this scale on LLMs specifically. OpenAI has been building LLM-specific infrastructure for years. Why do you think Google Bard sucks so much? It's not because there's some special magic secret that only OAI knows. It's because you need a huge amount of purpose-built infrastructure and trial-and-error iteration from experiments each of which take months. You can't easily catch up to that in a few months. Even if you throw 10x more staff at it, you are not going to make progress faster.
The only way you could catch up faster is by having orders of magnitude more hardware. But due the global shortage on hardware right now, nobody has that, not even Google.
- sangnoir 3y ago> They have the scale in terms of hardware, but they likely don't have the infrastructure to experiment efficiently on this scale on LLMs specifically This reads like wishful thinking, in the absence of a real moat. Do you think the hyperscalers who are buiding data centers as quickly as they can get permits do not have the infra scale to match OpenAI? Who do you suppose has been providing ever-increasing revenue to Nvidia these past few years? They didn't start doing ML experimentation when ChatGPT was announced. I agree that they aren't specifically giving LLMs all of their attention - their infrastructure still has to support existing, profitable products that bring in real revenue. I disagree that they can't experiment to the same scale.
- __loam 3y ago> But due the global shortage on hardware right now, nobody has that, not even Google. Google has nearly unlimited resources to pursue this shit.