4 ms·
Even if DeepSeek has figured out how to do more (or at least as much) with less, doesn't the Jevons Paradox come into play? GPU sales would actually increase be
by UncleOxidant 2y ago
Even if DeepSeek has figured out how to do more (or at least as much) with less, doesn't the Jevons Paradox come into play? GPU sales would actually increase because even smaller companies would get the idea that they can compete in a space that only 6 months ago we assumed would be the realm of the large mega tech companies (the Metas, Googles, OpenAIs) since the small players couldn't afford to compete. Now that story is in question since DeepSeek only has ~200 employees and claims to be able to train a competitive model for about 20X less than the big boys spend.
- samvher 2y agoMy interpretation is that yes in the long haul, lower energy/hardware requirements might increase demand rather than decrease it. But right now, DeepSeek has demonstrated that the current bottleneck to progress is _not_ compute, which decreases the near term pressure on buying GPUs at any cost, which decreases NVIDIA's stock price.
- kemiller 2y agoShort term, I 100% agree, but remains to be seen what "short" means. According to at least some benchmarks, Deepseek is two full orders of magnitude cheaper for comparable performance. Massive. But that opens the door for much more elaborate "architectures" (chain of thought, architect/editor, multiple choice) etc, since it's possible to run it over and over to get better results, so raw speed & latency will still matter.
- groby_b 2y agoI think it's worth carefully pulling apart _what_ DeepSeek is cheaper at. It's somewhat cheaper at inference (0.3 OOM), and about 1-1.5 OOM cheaper for training (Inference costs: https://www.latent.space/p/reasoning-price-war https://www.latent.space/p/reasoning-price-war) It's also worth keeping in mind that depending on benchmark, these values change (and can shrink quite a bit) And it's also worth keeping in mind that the drastic drop in training cost(if reproducible) will mean that training is suddenly affordable for a much larger number of organizations. I'm not sure the impact on GPU demand will be as big as people assume.
- yifanl 2y agoIt does, but proving that it can be done with cheaper (and more importantly for NVidia), lower margin chips breaks the spell that NVidia will just be eating everybody's lunch until the end of time.
- aurareturn 2y agoIf demand for AI chips will increase due to Jevon’s paradox, why would Nvidia’s chips become cheaper? In the long run, yes, they will be cheaper due to more competition and better tech. But next month? It will be more expensive.
- yifanl 2y agoThe usage of existing but cheaper nvidia chips to make models of similar quality is the main takeaway. It'll be much harder to convince people to buy the latest and greatest with this out there.
- UncleOxidant 2y agoThe sweet spot for running local LLMs (from what I'm seeing on forums like r/localLlama) is 2 to 4 3090s each with 24GB of VRAM. NVidia (or AMD or Intel) would clean up if they offered a card with 3090 level performance but with 64GB of VRAM. Doesn't have to be the leading edge GPU, just a decent GPU with lots of VRAM. This is kind of what Digits will be (though the memory bandwidth is going to be slower with because it'll be DDR5) and kind of what AMD's Strix Halo is aiming for - unified memory systems where the CPU & GPU have access to the same large pool of memory.
- redlock 2y agoThe issue here is that, even with a lot of VRAM, you may be able to run the model, but with a large context, it will still be too slow. (For example, running LLaMA 70B with a 30k+ context prompt takes minutes to process.)
- aurareturn 2y ago
- hodder 2y agoJevons paradox isn't some iron law like gravity.
- trgn 2y agofeels like it is in tech. any gains in hardware or algorithm advance, immediately get consumed by increase in data retention and software bloat.
- gamblor956 2y agoImportant to note: the $5 million alleged cost is just the cpu compute cost for the final version of the model; it's not the cumulative cost of the research to date. The analogous costs would be what OpenAI spent to go from GPT 4 to GPT 4o (i.e., to develop the reasoning model from the most up-to-date LLM model). $5 million is still less than what OpenAI spent but it's not a magnitude lower. (OpenAI spent up to $100 million on GPT4 but a fraction of that to get GPT 4o. Will update comment if I can find numbers for 4o before edit window closes)
- fspeech 2y agoIt doesn't make sense to compare individual models. A better way is to look at total compute consumed, normalized by the output. In the end what counts is the cost of providing tokens.
- fspeech 2y agoBut why would the customers accept the high prices and high gross margin of Nvidia if they no longer fear missing out with insufficient hardware?
- tedunangst 2y agoSelling 100 chips for $1 profit is less profitable than selling 20 chips for $10 profit.
- HDThoreaun 2y agoMargin only goes down if a competitor shows up. Getting more "performance" per chip will actually let nvidia raise prices even more if they want.
- deadbabe 2y agoSince you no longer need CUDA, AMD becomes a new viable option.
- HDThoreaun 2y agoDeepseek uses cuda.