5 ms·
No. In the semiconductor industry, the "catch-up" player isn't normally spending less in absolute R&D terms. Comparing the R&D costs of creating GPT-4o vs. Dee
by saithound 29d ago
No. In the semiconductor industry, the "catch-up" player isn't normally spending less in absolute R&D terms.
Comparing the R&D costs of creating GPT-4o vs. DeepSeek V3 (the latest gen for which we already have good accurate numbers) it looks like the latter cost 1/20th as much to create.
If Samsung could catch up with TSMC for 1/20th of the cost, people definitely would say that TSMC has no moat.
- aurareturn 29d agoWhy do you think Chinese models cost 1/20th to train?
- saithound 29d agoThat's the ratio the widely published numbers give [1]. One does not have to believe the numbers [2], but those who do believe them are then justified to conclude that there's no moat. Which numbers you believe is of course going to affect whether you think there's a moat or not. That's largely orthogonal to your TSMC/Samsung analogy I responded to. If you think the "moatists" are wrong because they believe the wrong numbers, that's fine, but then there's no need for the analogy. [1] https://galileo.ai/blog/llm-model-training-cost https://galileo.ai/blog/llm-model-training-cost [2] https://medium.com/@theiand/how-can-deepseek-a-5-6-million-llm-outperform-openai-and-meta-38e995b35140 https://medium.com/@theiand/how-can-deepseek-a-5-6-million-l...
- aurareturn 29d agoBut fundamentally, why is their cost 1/20 and is it sustainable in the next 10 years of competition?
- saithound 29d agoNow that is a good and interesting question! Hopefully a "no-moatist" will share their reasoning.
- aurareturn 29d agoUltimately, that's what I need to be convinced. No one has put forth a good argument yet. Clever architecture --> Ok but OpenAI/Anthropic can use these as well and they also have very smart people with their secret clever architectures Distilling --> Ok but distilling means you will never be smarter than the original. Furthermore, reasoning is now hidden by private labs and they have poison pill answers for distilling if they can detect it. They will be able to detect distilling better and better. Cheaper electricity --> Ok this is cancelled out by their chips being much less efficient due to not having ASML EUV machine access. So I don't see why fundamentally their training costs are cheaper over the long term. I'm looking for a no-moatist to convince me.
- dartharva 29d agoLabor. Smart labor would be much cheaper I'd reckon in China than in the US.
- aurareturn 29d agoHow much advantage in costs? What % of labor is training cost?
- celrod 28d agoMercor, Tacit Labs, Handshake AI... I suspect companies like these play a big part in model improvements, generating high quality benchmark/task-focused data for training. However, these do require educated, white collar, workers.
- dartharva 28d agoConsidering that frontier scientists and engineers in the US are currently taking home seven (or even eight, in some cases) figure salaries - pretty high, I'd reckon.
- aurareturn 28d ago
- gtirloni 29d agoPlease just say what you want to say.