5 ms·
Tesla is very vertically integrated. This is just how they operate. You can make the argument that they shouldnt be so vertically integrated but it has worked
by jml78 4y ago
Tesla is very vertically integrated. This is just how they operate. You can make the argument that they shouldnt be so vertically integrated but it has worked for them thus far.
- dragontamer 4y agoExcept Tesla has 7,360 A100 GPUs. https://www.tomshardware.com/news/tesla-brags-about-in-house-supercomputer-now-with-7360-a100-gpus https://www.tomshardware.com/news/tesla-brags-about-in-house... So... no? They are clearly leveraging the NVidia ecosystem right now. Now maybe they have ambitions to get off of NVidia, but they're doing so in a rather asinine fashion. There's probably half-a-dozen groups trying to make a faster systolic matrix multiplication unit for the deep learning crowd. Tesla probably should have either worked with those groups and/or bought one out, for example.
- kjksf 4y agoThis gives me flashbacks of "advice" that Tesla should outsource manufacturing of Model 3 and focus on design. Tesla is (also) an AI company. Musk is thinking 10 years ahead. If you read the resumes of people leading FSD chip and DOJO: those are chip design industry veterans that Musk hired away from AMD, Apple and other. He built a stronger team than whatever startup you think he could buy. Those people have already been working on Dojo for over 6 years and they'll be working for next 20. https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ is the lead for Dojo and has been at Tesla for 6.7 years. Tesla can hire they best people and give them practically unlimited funding, certainly much bigger than any startup could afford. This is a big, bold, risky bet. Kind of like reusable rockets or going to mars.
- dragontamer 4y agoExcept... they already did outsource training chips and bought thousands of NVidia A100 GPUs. So now they'll need porting effort to move their GPU-based current design to this new, very non-GPU, instruction set.
- speedgoose 4y agoIt doesn’t sound like an insurmontable task for such a company.
- baq 4y agoif 99% of your compute is hidden behind pytorch/tensorflow... there isn't _that_ much work
- dragontamer 4y agoSure. You only have to design a chip, design an assembly language, design a compiler, design the kernels, design a parallelization framework, design a server system to load-balance tasks, and then rework the pytorch/tensorflow code to use your new faster custom primitives that no one else has. ----- Except step 1: "design a chip", is already something on the order of hundreds-of-megabucks of investment
- baq 4y agooh absolutely. but they're already there, looks like.
- dragontamer 4y agoIf they were already there, they'd have ResNet 50 benchmarks to share. And the other tidbit is that it has to be better than the off the shelf NVidia chip (A100 today, and GH100 in a few months). I think Dojo might be faster than A100, but GH100 will be 1.5 nodes ahead, that's a big process advantage... ------- If GH100 is faster in practice, then all the work has been somewhat wasted.