5 ms·

The shovels and labour used to make those things where not depreciated. The GPUs are the shovels, not the project. AI at any capability will retain that capbi

• jiggawatts 6mo ago
  > retain that capbibilty forever Not really. The base training data cutoff will quickly render models useless as they fail to keep up with developments. Translating some Farsi news articles about the war was hilarious, Gemini Pro got into a panic. ChatGPT either accused me of spreading fake news, or assumed this was some sort of fantasy scenario.
  • m00x 6mo ago
    That's GPT4 thinking. New models use tools to look at current events or latest versions, and rely very little on weight knowledge.
    • zozbot234 6mo ago
      You can pull new information into the context via RAG, but that is expensive and only gives very shallow understanding compared to retraining.
  • nl 6mo ago
    Not really. For coding I care mostly about reasoning ability which is uncorrelated with cut off
  • jeremyjh 6mo ago
    Karpathy - and others - consider the pre-training knowledge as much a liability as an asset. If we could just retain the emergent reasoning and language capability without the hazy recollections the models would likely be stronger.
• loandbehold 6mo ago
  You need to separate training and inference usage of GPUs for this analysis.
  • Mathnerd314 6mo ago
    "Inference consumes 60–90% of total AI lifecycle costs." So shovel is not the right analogy, more like GPU = coal burning engine. And yes, coal was a big railroad expense, more so than financing construction debt.
• kennywinker 6mo ago
  Calling the GPUs the shovels is bonkers because a) shovels are cheap, GPUs are not. And b) when you build a bridge the bridge doesn’t need shovels to be passable. Without GPUs, the datacenter is useless, the model is useless, etc. If anything, the GPUs are the steel that the bridge is made of. Each beam can be replaced, but if too many fail the bridge is impassible. A bridge with a 6 year lifespan for each beam is insane.
  • throwup238 6mo ago
    You’re taking the metaphor way too literally. The people who made the most profit weren’t literally selling shovels, they were the ones providing logistics and support services to the gold miners, like hauling tons of equipment over tens of miles of mountain or providing the sales channel for the gold. They siphoned off most of the profit from the ventures that depended on them (like LLMs depend on GPUs) because the miners had no other choice, to the point where even the most productive mines often weren’t profitable at all. A less literal example is the conquistadors: their shovels were ships, horses, gunpowder, and steel. You can look at Spanish records from the Council of the Indies archive and any time treasures were discovered, the price of each skyrocketed to the point where only the wealthiest hidalgos and their patrons could afford to go on such adventures. I.e. the cost of a ship capable of a cross Atlantic voyage going from 100k pieces of eight to over a million in the span of only a few years (predating the treasure fleet inflation!) Gold rushes create demand shocks, and anyone who is a supplier to that demand makes bank, regardless of whether its GPUs or “shovels”.
    • TeMPOraL 6mo ago
      > You can look at Spanish records from the Council of the Indies archive and any time treasures were discovered, the price of each skyrocketed to the point where only the wealthiest hidalgos and their patrons could afford to go on such adventures. Today this is real estate. And it's something people keep forgetting when arguing that ${whatever breakthrough or just more competition} will make ${some good or service} cheaper for consumers: prices of other things elsewhere will raise to compensate and consume any average surplus. Money left on the table doesn't stay there for long.
• deleted 6mo ago
  [deleted]
• deleted 6mo ago
  [deleted]