4 ms·

Does anyone think that there will be enough GPUs in data centers soon? I honestly don't know. But it seems like there should be. Like I (kinda) get the scali

by timmg 2y ago

Does anyone think that there will be enough GPUs in data centers soon? I honestly don't know. But it seems like there should be. Like I (kinda) get the scaling laws with LLMs. But at what point do we run out of training data? Is there that much "meat on the bone" that will carry Nvidia a few more years? And at what point does spending $x million more on training the base models no longer provide that much more value?

TuringNYC 2y ago
>> Does anyone think that there will be enough GPUs in data centers soon? I honestly don't know. But it seems like there should be. Not sure about that, but given the TAM, associated market value, and obvious juicy opportunity -- what prevents driven alternatives (even Apple Silicon) from biting off portions of this TAM?
pton_xd 2y ago
> But at what point do we run out of training data? Multi-modal LLMs trained on video / YouTube won't run out of data any time soon. And LLMs can always train on an endless supply of synthetic data like code. > And at what point does spending $x million more on training the base models no longer provide that much more value? My understanding is we're still waiting to see where on the S-curve LLMs are in regards to scaling. So far it seems like scaling hasn't hit any diminishing returns. I haven't read many sources talking about the practicality of that though. Even if LLMs continue scaling performance as we throw more compute + data at them, at some point it may just become uneconomical to continue. Capital expenditures also have their own utility curve. How far away are we from that point?
- jacobr1 2y ago
  > Capital expenditures also have their own utility curve This a function of the use-cases. For simple use cases like document summarization, or speech-to-text, we might already be there. There are already applications where using something like gpt-3-turbo or a local-llm vs gpt-4 make more economic sense. The marginal improvement in reasoning or extra knowledge don't provide benefits relative to the inference costs. And you can see this playing out with Mistral, Anthropic and Google also providing models at different cost/complexity tradeoffs. But .... if we get _new_ use cases, emergent use cases, from bigger models, there will be continued value in chasing that frontier. Reasoning is still poor in many areas. The ability to learn outside of application specific memory patterns that store state is largely unexplored. We are still far from AGI and there is a ton of value to be minded between current systems and a superhuman capabilities
jacobr1 2y ago
> But at what point do we run out of training data? We have at least another order of magnitude of additional data via audio-video. There also is synthetic data, such as was likely done for SORA. And we likely could increase that another few orders of magnitude if we had more universal sensors. Windows Recall is crazy for privacy reasons, but gathering feeds of data from everyone as they go about their lives, physical as well as digital provides much more room for training. You also need to take into account data-growth. How much more text data is generated daily, relative to how much has ever existed? There also is probably another 1-2 orders of magnitude more data in private locations and also growing fast. Some of that can be captured via RAG. But even if a small fraction is used for fine-tuning, in aggregate that is a lot of GPU.
creer 2y ago
> But at what point do we run out of training data? Isn't the training itself very dependent on each algorithm. Even if you somehow gathered all the plausible training data, you'd have to run it through again for most improvements of your system. But there are trends to look at in this direction: changes in the training that make it much, much faster; And processors which become more effective (fewer of them needed). So anyway, very long term, sure. For now, doesn't seem to be a problem. For that matter, for now, there is not only building new data centers, but also upgrading existing ones to the more efficient processors.

by timmg 2y ago

TuringNYC 2y ago

>> Does anyone think that there will be enough GPUs in data centers soon? I honestly don't know. But it seems like there should be. Not sure about that, but given the TAM, associated market value, and obvious juicy opportunity -- what prevents driven alternatives (even Apple Silicon) from biting off portions of this TAM?

pton_xd 2y ago

> But at what point do we run out of training data? Multi-modal LLMs trained on video / YouTube won't run out of data any time soon. And LLMs can always train on an endless supply of synthetic data like code. > And at what point does spending $x million more on training the base models no longer provide that much more value? My understanding is we're still waiting to see where on the S-curve LLMs are in regards to scaling. So far it seems like scaling hasn't hit any diminishing returns. I haven't read many sources talking about the practicality of that though. Even if LLMs continue scaling performance as we throw more compute + data at them, at some point it may just become uneconomical to continue. Capital expenditures also have their own utility curve. How far away are we from that point?

jacobr1 2y ago
> Capital expenditures also have their own utility curve This a function of the use-cases. For simple use cases like document summarization, or speech-to-text, we might already be there. There are already applications where using something like gpt-3-turbo or a local-llm vs gpt-4 make more economic sense. The marginal improvement in reasoning or extra knowledge don't provide benefits relative to the inference costs. And you can see this playing out with Mistral, Anthropic and Google also providing models at different cost/complexity tradeoffs. But .... if we get _new_ use cases, emergent use cases, from bigger models, there will be continued value in chasing that frontier. Reasoning is still poor in many areas. The ability to learn outside of application specific memory patterns that store state is largely unexplored. We are still far from AGI and there is a ton of value to be minded between current systems and a superhuman capabilities

jacobr1 2y ago

> But at what point do we run out of training data? We have at least another order of magnitude of additional data via audio-video. There also is synthetic data, such as was likely done for SORA. And we likely could increase that another few orders of magnitude if we had more universal sensors. Windows Recall is crazy for privacy reasons, but gathering feeds of data from everyone as they go about their lives, physical as well as digital provides much more room for training. You also need to take into account data-growth. How much more text data is generated daily, relative to how much has ever existed? There also is probably another 1-2 orders of magnitude more data in private locations and also growing fast. Some of that can be captured via RAG. But even if a small fraction is used for fine-tuning, in aggregate that is a lot of GPU.

creer 2y ago

> But at what point do we run out of training data? Isn't the training itself very dependent on each algorithm. Even if you somehow gathered all the plausible training data, you'd have to run it through again for most improvements of your system. But there are trends to look at in this direction: changes in the training that make it much, much faster; And processors which become more effective (fewer of them needed). So anyway, very long term, sure. For now, doesn't seem to be a problem. For that matter, for now, there is not only building new data centers, but also upgrading existing ones to the more efficient processors.