I don’t think people fully appreciate yet how much of LLMs’ value comes from their underlying dataset (I.e. the entire internet - probably .. quadrillions..? of

I don’t think people fully appreciate yet how much of LLMs’ value comes from their underlying dataset (I.e. the entire internet - probably .. quadrillions..? of tokens of text) rather than the model + compute itself. If you’re trying to predict something within the manifold of data on the internet (which is incredibly vast, but not infinite), you will do very well with today’s LLMs. Building an internet-scale dataset for another problem domain is a monumental task, still with significant uncertainty about “how much is enough”. People have been searching for the right analogy for “what type of company is Open AI most like?” I’ll suggest they’re like an oil company, but without the right to own oil fields. The internet is the field, the model is the refining process (which mostly yield the same output but with some variations - not dissimilar from petroleum products).. and the process / model is a significant asset. And today, Nvidia is the only manufacturer of refining equipment.