3 ms·
How long would take them to get a new chip into production and then used for training/inference?
by fuddle 2y ago
How long would take them to get a new chip into production and then used for training/inference?
- wmf 2y agoThe first generation chip won't be good enough to use in production so then you have the second generation... maybe 3-4 years.
- KaiserPro 2y agotwoish years, if you're good. Then the software to make it work properly. its not a quick thing to do. You can throw more money at it to make it go faster, but it also might fuck it up and take longer.
- disqard 2y agoCorrect! AMD has been trying to fix "the software to make it work properly" part for years -- more than two years.
- latchkey 2y agoAMD hardware is good. The software is getting better daily. Training still sucks (mostly due to unoptimized libraries), but inference is looking pretty decent with recent advances in tuning... https://shisa.ai/blog/posts/tuning-vllm-mi300x/ https://shisa.ai/blog/posts/tuning-vllm-mi300x/ https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html
- KaiserPro 2y ago> AMD hardware is good. The software is getting better daily. Which is the story of AMD for the last ~15 years. getting support so we could develop apps for their graphics card was dispiriting. at the time they were faster and very much cheaper than nvidia (but no cuda) But every time we found an issue, the poor devs who were contracted to fix it were left struggling. Its the same where I am now. We were trying to qualify some motherboard/TPM/other issue, but they didn't have enough bandwidth to help us in time. Its better now, we do have some AMD stuff in the fleet.
- latchkey 2y agoI agree, the history is pretty bad. At least from someone who is on the edge of the business and seeing what is going on. It is a lot better now, especially at the enterprise level. They've been hiring and buying companies too. It won't be fixed over night, but it is definitely a renewed focus.
- Oras 2y agoI don’t think time is quite important here, look at Apple silicon chips and how they transformed the performance and removed dependency on Intel. Once they get it right, they will increase the gap with other LLM providers.
- hmottestad 2y agoThey could also end up running out of money before they get it right, or another company might come out with a chip that is just a bit better and cheaper a year before them. Apple didn't build their M1 chip from scratch. By the time the M1 chip was released they had already been building chips for the iPhone and iPad for several years!