7 ms·

It's not about saving money. It's about accelerating nn training and winning robotaxi market, which could make them the most profitable company ever.

by kjksf 4y ago

dragontamer 4y ago
NVidia chips can do this job. (EDIT: The job of training a neural network I should say. I don't think that's enough to get self driving, but Dojo ain't anything but a faster NN training chip anyway) The question is if they are saving money compared to buying an off the shelf DGX system. I really doubt it.
- jml78 4y ago
  Tesla is very vertically integrated. This is just how they operate. You can make the argument that they shouldnt be so vertically integrated but it has worked for them thus far.
  - dragontamer 4y ago
    Except Tesla has 7,360 A100 GPUs. https://www.tomshardware.com/news/tesla-brags-about-in-house-supercomputer-now-with-7360-a100-gpus https://www.tomshardware.com/news/tesla-brags-about-in-house... So... no? They are clearly leveraging the NVidia ecosystem right now. Now maybe they have ambitions to get off of NVidia, but they're doing so in a rather asinine fashion. There's probably half-a-dozen groups trying to make a faster systolic matrix multiplication unit for the deep learning crowd. Tesla probably should have either worked with those groups and/or bought one out, for example.
    - kjksf 4y ago
      This gives me flashbacks of "advice" that Tesla should outsource manufacturing of Model 3 and focus on design. Tesla is (also) an AI company. Musk is thinking 10 years ahead. If you read the resumes of people leading FSD chip and DOJO: those are chip design industry veterans that Musk hired away from AMD, Apple and other. He built a stronger team than whatever startup you think he could buy. Those people have already been working on Dojo for over 6 years and they'll be working for next 20. https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ is the lead for Dojo and has been at Tesla for 6.7 years. Tesla can hire they best people and give them practically unlimited funding, certainly much bigger than any startup could afford. This is a big, bold, risky bet. Kind of like reusable rockets or going to mars.
      - dragontamer 4y ago
        Except... they already did outsource training chips and bought thousands of NVidia A100 GPUs. So now they'll need porting effort to move their GPU-based current design to this new, very non-GPU, instruction set.
        speedgoose 4y ago
        It doesn’t sound like an insurmontable task for such a company.
        baq 4y ago
        if 99% of your compute is hidden behind pytorch/tensorflow... there isn't _that_ much work
        dragontamer 4y ago
        Sure. You only have to design a chip, design an assembly language, design a compiler, design the kernels, design a parallelization framework, design a server system to load-balance tasks, and then rework the pytorch/tensorflow code to use your new faster custom primitives that no one else has. ----- Except step 1: "design a chip", is already something on the order of hundreds-of-megabucks of investment
        baq 4y ago
        oh absolutely. but they're already there, looks like.
        dragontamer 4y ago
        If they were already there, they'd have ResNet 50 benchmarks to share. And the other tidbit is that it has to be better than the off the shelf NVidia chip (A100 today, and GH100 in a few months). I think Dojo might be faster than A100, but GH100 will be 1.5 nodes ahead, that's a big process advantage... ------- If GH100 is faster in practice, then all the work has been somewhat wasted.
- pengaru 4y ago
  > NVidia chips can do this job. Presumably Tesla, at the time they decided to pursue this option, thought they'd potentially have a competitive advantage with in-house designs. It's entirely possible that's just their hubris showing, time will tell if this was the right decision or not. After seeing the NVidia presentation announcing their latest datacenter-scale AI hardware, I'd be surprised if Tesla's in-house design is more than just a massive cost center vs. buying something from NVidia. But sometimes you do things that appear irrational in part to keep your talented engineers from seeking work elsewhere. Just look at NASA's SLS, how much of that is a job program in part to prevent hoards of talented folks building rockets for competing nations?
  - dragontamer 4y ago
    > After seeing the NVidia presentation announcing their latest datacenter-scale AI hardware Did we watch the same presentation? NVidia knocked it out of the park. Thread block cluster is obviously amazing. Routing between SMs / compute units will be far faster with this level of software abstraction, and it will be exceptionally easy to write code for. NVidia always impresses me with their advanced software techniques and clear understanding of the fundamental model of SIMD compute. ------ Ignoring those software details... the important thing is GH100 will be TSMC 4nm, which is 1.5 nodes ahead of the 7nm Dojo. A significant process advantage, representing 60+% less power usage and 300% the transistor density of the older 7nm tech. Even if NVidia's GPU had issues, there's something to be said about just being a raw process node (or 1.5 nodes) ahead.
    - pengaru 4y ago
      > Did we watch the same presentation? NVidia knocked it out of the park. Perhaps I worded it poorly, I agree with you. My meaning was vs. NVidia's latest tech it seems like Tesla's in-house datacenter NN could be nothing more than a huge cost center without even offering an advantage over what NVidia could sell them. But like I said, if you have a staff of folks capable of building such things you have to keep them satisfied with practicing their craft or they leave.
      - 4y ago
- zeristor 4y ago
  If I recall from the Dojo chip announcement previously they were utilising SpaceX know how to greatly increase the cooling of the chips. Although the article doesn’t look to address thermal issues from my summary skim of it.
- rbanffy 4y ago
  > NVidia chips can do this job Yes, but anyone can get them - they aren’t a competitive advantage.
SneakyTornado29 4y ago
No, the data is the valuable part. Not the model weights.
FireBeyond 4y ago
The same robotaxi market that Elon promised in 2019, that would make "buying anything but a Tesla in 2020 and beyond financially irresponsible"?

It's not about saving money. It's about accelerating nn training and winning robotaxi market, which could make them the most profitable company ever.

by kjksf 4y ago

It's not about saving money. It's about accelerating nn training and winning robotaxi market, which could make them the most profitable company ever.

dragontamer 4y ago

NVidia chips can do this job. (EDIT: The job of training a neural network I should say. I don't think that's enough to get self driving, but Dojo ain't anything but a faster NN training chip anyway) The question is if they are saving money compared to buying an off the shelf DGX system. I really doubt it.

jml78 4y ago
Tesla is very vertically integrated. This is just how they operate. You can make the argument that they shouldnt be so vertically integrated but it has worked for them thus far.
- dragontamer 4y ago
  Except Tesla has 7,360 A100 GPUs. https://www.tomshardware.com/news/tesla-brags-about-in-house-supercomputer-now-with-7360-a100-gpus https://www.tomshardware.com/news/tesla-brags-about-in-house... So... no? They are clearly leveraging the NVidia ecosystem right now. Now maybe they have ambitions to get off of NVidia, but they're doing so in a rather asinine fashion. There's probably half-a-dozen groups trying to make a faster systolic matrix multiplication unit for the deep learning crowd. Tesla probably should have either worked with those groups and/or bought one out, for example.
  - kjksf 4y ago
    This gives me flashbacks of "advice" that Tesla should outsource manufacturing of Model 3 and focus on design. Tesla is (also) an AI company. Musk is thinking 10 years ahead. If you read the resumes of people leading FSD chip and DOJO: those are chip design industry veterans that Musk hired away from AMD, Apple and other. He built a stronger team than whatever startup you think he could buy. Those people have already been working on Dojo for over 6 years and they'll be working for next 20. https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ is the lead for Dojo and has been at Tesla for 6.7 years. Tesla can hire they best people and give them practically unlimited funding, certainly much bigger than any startup could afford. This is a big, bold, risky bet. Kind of like reusable rockets or going to mars.
    - dragontamer 4y ago
      Except... they already did outsource training chips and bought thousands of NVidia A100 GPUs. So now they'll need porting effort to move their GPU-based current design to this new, very non-GPU, instruction set.
      - speedgoose 4y ago
        It doesn’t sound like an insurmontable task for such a company.
      - baq 4y ago
        if 99% of your compute is hidden behind pytorch/tensorflow... there isn't _that_ much work
        dragontamer 4y ago
        Sure. You only have to design a chip, design an assembly language, design a compiler, design the kernels, design a parallelization framework, design a server system to load-balance tasks, and then rework the pytorch/tensorflow code to use your new faster custom primitives that no one else has. ----- Except step 1: "design a chip", is already something on the order of hundreds-of-megabucks of investment
        baq 4y ago
        oh absolutely. but they're already there, looks like.
        dragontamer 4y ago
        If they were already there, they'd have ResNet 50 benchmarks to share. And the other tidbit is that it has to be better than the off the shelf NVidia chip (A100 today, and GH100 in a few months). I think Dojo might be faster than A100, but GH100 will be 1.5 nodes ahead, that's a big process advantage... ------- If GH100 is faster in practice, then all the work has been somewhat wasted.
pengaru 4y ago
> NVidia chips can do this job. Presumably Tesla, at the time they decided to pursue this option, thought they'd potentially have a competitive advantage with in-house designs. It's entirely possible that's just their hubris showing, time will tell if this was the right decision or not. After seeing the NVidia presentation announcing their latest datacenter-scale AI hardware, I'd be surprised if Tesla's in-house design is more than just a massive cost center vs. buying something from NVidia. But sometimes you do things that appear irrational in part to keep your talented engineers from seeking work elsewhere. Just look at NASA's SLS, how much of that is a job program in part to prevent hoards of talented folks building rockets for competing nations?
- dragontamer 4y ago
  > After seeing the NVidia presentation announcing their latest datacenter-scale AI hardware Did we watch the same presentation? NVidia knocked it out of the park. Thread block cluster is obviously amazing. Routing between SMs / compute units will be far faster with this level of software abstraction, and it will be exceptionally easy to write code for. NVidia always impresses me with their advanced software techniques and clear understanding of the fundamental model of SIMD compute. ------ Ignoring those software details... the important thing is GH100 will be TSMC 4nm, which is 1.5 nodes ahead of the 7nm Dojo. A significant process advantage, representing 60+% less power usage and 300% the transistor density of the older 7nm tech. Even if NVidia's GPU had issues, there's something to be said about just being a raw process node (or 1.5 nodes) ahead.
  - pengaru 4y ago
    > Did we watch the same presentation? NVidia knocked it out of the park. Perhaps I worded it poorly, I agree with you. My meaning was vs. NVidia's latest tech it seems like Tesla's in-house datacenter NN could be nothing more than a huge cost center without even offering an advantage over what NVidia could sell them. But like I said, if you have a staff of folks capable of building such things you have to keep them satisfied with practicing their craft or they leave.
    - 4y ago
zeristor 4y ago
If I recall from the Dojo chip announcement previously they were utilising SpaceX know how to greatly increase the cooling of the chips. Although the article doesn’t look to address thermal issues from my summary skim of it.
rbanffy 4y ago
> NVidia chips can do this job Yes, but anyone can get them - they aren’t a competitive advantage.

SneakyTornado29 4y ago

No, the data is the valuable part. Not the model weights.

FireBeyond 4y ago

The same robotaxi market that Elon promised in 2019, that would make "buying anything but a Tesla in 2020 and beyond financially irresponsible"?