9 ms·

It is incredibly odd tesla decided to invest in this level of engineering, even perfect hardware is like, 1/5th the battle. Now you get to stand up the entire s

It is incredibly odd tesla decided to invest in this level of engineering, even perfect hardware is like, 1/5th the battle. Now you get to stand up the entire software stack, make it performant, and hope you can compete with whatever Nvidia has in the pipeline.

etaioinshrdlu 4y ago
As far as I can tell Tesla doesn't even use Dojo seriously yet. The real work is still done on NVIDIA hardware.
- deleted 4y ago
  [deleted]
- cjbgkagh 4y ago
  It seems like to me they took a big bet and it didn’t pay off. They may not have felt like they could rely on Nvidia to deliver the promised Tensor Core performance.
  - kjksf 4y ago
    It's a bit premature to call this a failure given they are barely at a stage of making first chips and validating them. You don't seem to understand the timelines of designing chips like that from scratch. Dojo has been in works for almost 7 years. Tesla will continue to work on this for the next 20 years. There will be Dojo v2, Dojo v3 etc. just like at SpaceX there was Falcon 1, Falcon 9, Falcon Heavy. This still might end up a failure but they clearly feel confident enough to talk about Dojo publicly, which wasn't the case for the first 5 years of its development.
    - cjbgkagh 4y ago
      Thanks for explaining to me what I don’t know. Clearly you’re a Tesla fanboy. There is no doubt that it is an amazing piece of tech but I’m not confident Tesla will be able to pull off beating NVidia. Especially when compared to NVidias Tensor Cores and economics of scale. I don’t like their whole approach to self driving ML. I know a lot of people disagree with me so I would rather not get into it. I think a lot of the talk of future performance is posturing in order to get NVidia chips at cheaper prices.
      - minhazm 4y ago
        I'm sure that it's at least partially for them to have a better negotiating position with NVIDIA. But the reality is that Tesla actually has some very good expertise in chip design. Jim Keller worked there for several years and along with Pete Bannon designed their Autopilot HW3 computer which is in 1+ million cars right now. At the time that HW3 was released it outperformed what NVIDIA had to offer. That said it's not likely they'll be able to beat NVIDIA, but they may be able to beat them for their hyper domain specific use cases. Additionally NVIDIA chips are difficult to get and they're extremely expensive. Even if Tesla can get something that performs 80-90% as good as NVIDIA but at significantly lower cost then it may still be worth it.
        cjbgkagh 4y ago
        I know these things. I think some of the dojo architecture was a reaction to their FSD chips being over optimized for resnet style models. They’re targeting extremely large models which is a new frontier for ML and in my view not the panacea they hoped it would be. I think with better ML they wouldn’t need so many chips, be it NVidia or Dojo.
    - mupuff1234 4y ago
      It's even more premature to predict what will happen 20 years in the future.
- kjksf 4y ago
  Yes, per the talk Dojo is at the stage of having made first chips. But your comment is neither here not there. What do you think is a timeline for designing a chip like Dojo from scratch? Tesla has been working on this for almost 7 years (https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/). They might not work fast enough for your satisfaction but can you point any other car company designing chips like Dojo? Hell, can you point to any company at all that can come out of the gate with a chip design competitive with the best design of NVIDIA, a company that makes nothing but chips? After 7 years of work they are clearly confident that this will work or else they would scarp the project instead of giving talks at conferences.
  - panick21_ 4y ago
    Also, even if the first version didn't make financial sense, building the team and the infrastructure required to make this work will likely translate into a next generation chip. Part of Tesla vertical integration mantra is that you are building internal capacity.
  - FireBeyond 4y ago
    > After 7 years of work they are clearly confident that this will work or else they would scarp the project instead of giving talks at conferences. They are clearly confident that FSD will work, to the point of promising it "this year"... coincidentally, they've been promising "this year" for the last 7 years. And FSD still fails in spectacular/hilarious/horrifying ways, and isn't _remotely_ close.
  - aunty_helen 4y ago
    >After 7 years of work they are clearly confident that this will work or else they would scarp the project instead of giving talks at conferences. Being real for a second, Tesla don’t have an amazing track record of delivery on their ai software side of things.
  - p1esk 4y ago
    Hell, can you point to any company at all that can come out of the gate with a chip design competitive with the best design of NVIDIA, a company that makes nothing but chips? Google.
SuperscalarMeme 4y ago
Not to mention that in-house silicon is all about economies of scale, this is even more of a puzzling move
- kjksf 4y ago
  Tesla has a path to economies of scale: they already announced that if Dojo works as expected they'll make it available to others as an AWS-style service. Which is brilliant: they might end up making money on this. AI is clearly here to stay. The demand for AI training will clearly explode in the future. Running training in-house is not easy or cheap. You don't just plug in 1000 NVIDIA GPUs. You need massive up-front payment for GPUs and you're basically running your own extremely energy hungry datacenter . Tesla might built and operate massive datacenters. They'll use as much as they need for internal needs and sell the remaining capacity to others. This might take 5 years but the path to do it is clear.
  - scoopertrooper 4y ago
    I don’t see how they’re going to commercialise this as a cloud compute service. For one, they’ve built a chip that operates in a fundamentally different way to other chips. So any other company that wanted to use it would have to invest a considerable amount of resources in building up the institutional knowledge to use it effectively. Additionally, the lack of virtual memory and multi-tasking support renders it pretty much impossible to divide up compute between multiple customers. So, commercialising this would require customers renting out the whole unit, which is contrary to how cloud computing usually works. Are there companies out there that have the capital and use cases necessary to fit into Dojo Cloud? Maybe, though not one I’ve worked for. Would they trust the stable genius currently heading up Tesla enough to make such an investment? Perhaps, but I wouldn’t, but what do I know?
    - kjksf 4y ago
      Part of the plan is to have PyTorch compatibility. Dojo has it's own IR but they also have a PyTorch to Dojo compiler. People's opinion of Musk wont matter: either Dojo will be a capable service at a good price or it won't. People will use it based on merits.
dragontamer 4y ago
Dojo is rumored 7nm to boot. They'll be competing with a process node disadvantage. TSMC is down to 3nm and 5nm already. AMD chips like MI250x are 5nm with tensor matrix multiplication units. NVidia Hopper will be like 4nm IIRC?
- kjksf 4y ago
  During first AI day Tesla already said that they have plans and ideas for 10x improvements for Dojo v2. And I'm sure one of the improvements is to use 5nm or 3nm process. They've already been working on this for almost 7 years. This is not some side projects but a serious operation. They have to ship something sometime and they decided to ship this now. I'm confident that they know about 5nm process and chose 7nm for good reasons. There will be next version and next process.
  - sbierwagen 4y ago
    >And I'm sure one of the improvements is to use 5nm or 3nm process. Implementing a process shrink is not just scaling the masks by the appropriate percentage. It often is a completely different optical train, at a different wavelength, different pellicle, changing the refractive index of the immersion fluid, different multiple patterning. It takes months (years?) of work. For many applications it's worth it, but it's not at all a slam dunk. The vast majority of ASICs are designed for a particular node and never upgraded. It's the kind of crazy long-term speculative capital investment TSLA might have indulged in when the stock was at $414 a share, but it's nowhere remotely near that today. They're already engaging in layoffs, today. Why spend the money?
    - rbanffy 4y ago
      > vast majority of ASICs are designed for a particular node and never upgraded They are, if the new node is cheap enough to justify the investment in shrinking the design. In most ASICs, being faster won’t make people rip out their embedded electronics for new models.
- panick21_ 4y ago
pookeh 4y ago
Perhaps they do this so they can carry over this tech into their cars. If you are making millions of cars per year, with tens or hundreds of chips per car, it is easy to reason doing this on your own instead of outsourcing it.
jillesvangurp 4y ago
I think they developed this during the years where all the nvidia hardware was gobbled up by crypto miners. The resulting pricing probably did not help either. Tesla is big enough that they can afford to experiment a little. This sounds like an experiment that actually worked. Once they had it working, doing more of it and iterating on it is just business as usual for them.
- cyber_kinetist 4y ago
  > This sounds like an experiment that actually worked. Are you sure? We haven’t even seen the chip in any physical form yet. We haven’t seen it running any code.
  - seunosewa 4y ago
    The self-driving AI they use this chip for appears to be less than impressive from my perspective. Its mistakes seem to come up in the news quite frequently.
    - johnsimer 4y ago
      Dojo’s D1 chip for training AFAIK is not being used yet. Tesla still trains with their ~7000 cluster of A100s And the D1 is for their training machine anyways. Not their inference chip in the cars
    - jillesvangurp 4y ago
      Are you aware of a better implementation though? It might not be perfect; it's still miles ahead of competing systems. Yes, it makes mistakes. But that's a rather poor argument to dismiss the whole strategy and technology. From my point of view, Tesla is converging on good enough more rapidly than anyone else in the industry. Courtesy of their AI strategy and technology; including the in house developed chips.
      - friendzis 4y ago
        > Tesla is converging on good enough This is a big problem, though. Any ADAS is a safety critical system and while "good enough" can help push volumes, it is extremely bad from safety perspective. If you really want to read on this u/adamjosephcook/ has some awesome writing going into far deeper detail than I could explaining this particular issue.
torginus 4y ago
Let me play the devil's advocate - when you have an $1T company, investing a couple million into something like this signals to investors that you have big plans for the future and have a leg up on the competition in the self-driving space. Even if that increases the stock price by 1%, this investment pays for itself a hundred times over.
- lclarkmichalek 4y ago
  Setting up a hardware platform of this kind that is useable in production costs significantly more than a couple of million
  - ricardobeat 4y ago
    Still a blip for a company that was toying with $1B in Bitcoin.
- baq 4y ago
  this is more like couple hundred million TBH
  - dragontamer 4y ago
    Hundred million for the chip alone, a few hundred million for software (compiler, assembler, router, protocols, communications, load balancing, server, supercomputer utilization, power management). The $100-million chip might very well be the easy part of this project...
    - sinenomine 4y ago
      As a software person it baffles me how we tend to overestimate our value compared to hardware. The chip design field is really demanding, especially with deep submicron processes and with rigorous testing & verification culture that blows out of the water almost everything software organizations do. I doubt developing a compiler for this chip will be as costly as your estimate, especially given mature opensource compiler frameworks like LLVM (already adapted to similar TPU-like architectures) and upcoming pytorch glow, apache TVM ML-specific compiler frameworks. And especially given in our industry there are people with advanced degrees who are willing to jump through additional hoops and even accept pay cuts, just to work on a cool compiler. I respect the hardware for what it is and what it could enable.

etaioinshrdlu 4y ago

As far as I can tell Tesla doesn't even use Dojo seriously yet. The real work is still done on NVIDIA hardware.

deleted 4y ago
[deleted]
cjbgkagh 4y ago
It seems like to me they took a big bet and it didn’t pay off. They may not have felt like they could rely on Nvidia to deliver the promised Tensor Core performance.
- kjksf 4y ago
  It's a bit premature to call this a failure given they are barely at a stage of making first chips and validating them. You don't seem to understand the timelines of designing chips like that from scratch. Dojo has been in works for almost 7 years. Tesla will continue to work on this for the next 20 years. There will be Dojo v2, Dojo v3 etc. just like at SpaceX there was Falcon 1, Falcon 9, Falcon Heavy. This still might end up a failure but they clearly feel confident enough to talk about Dojo publicly, which wasn't the case for the first 5 years of its development.
  - cjbgkagh 4y ago
    Thanks for explaining to me what I don’t know. Clearly you’re a Tesla fanboy. There is no doubt that it is an amazing piece of tech but I’m not confident Tesla will be able to pull off beating NVidia. Especially when compared to NVidias Tensor Cores and economics of scale. I don’t like their whole approach to self driving ML. I know a lot of people disagree with me so I would rather not get into it. I think a lot of the talk of future performance is posturing in order to get NVidia chips at cheaper prices.
    - minhazm 4y ago
      I'm sure that it's at least partially for them to have a better negotiating position with NVIDIA. But the reality is that Tesla actually has some very good expertise in chip design. Jim Keller worked there for several years and along with Pete Bannon designed their Autopilot HW3 computer which is in 1+ million cars right now. At the time that HW3 was released it outperformed what NVIDIA had to offer. That said it's not likely they'll be able to beat NVIDIA, but they may be able to beat them for their hyper domain specific use cases. Additionally NVIDIA chips are difficult to get and they're extremely expensive. Even if Tesla can get something that performs 80-90% as good as NVIDIA but at significantly lower cost then it may still be worth it.
      - cjbgkagh 4y ago
        I know these things. I think some of the dojo architecture was a reaction to their FSD chips being over optimized for resnet style models. They’re targeting extremely large models which is a new frontier for ML and in my view not the panacea they hoped it would be. I think with better ML they wouldn’t need so many chips, be it NVidia or Dojo.
  - mupuff1234 4y ago
    It's even more premature to predict what will happen 20 years in the future.
kjksf 4y ago
Yes, per the talk Dojo is at the stage of having made first chips. But your comment is neither here not there. What do you think is a timeline for designing a chip like Dojo from scratch? Tesla has been working on this for almost 7 years (https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/ https://www.linkedin.com/in/ganesh-venkataramanan-99272a3/). They might not work fast enough for your satisfaction but can you point any other car company designing chips like Dojo? Hell, can you point to any company at all that can come out of the gate with a chip design competitive with the best design of NVIDIA, a company that makes nothing but chips? After 7 years of work they are clearly confident that this will work or else they would scarp the project instead of giving talks at conferences.
- panick21_ 4y ago
  Also, even if the first version didn't make financial sense, building the team and the infrastructure required to make this work will likely translate into a next generation chip. Part of Tesla vertical integration mantra is that you are building internal capacity.
- FireBeyond 4y ago
  > After 7 years of work they are clearly confident that this will work or else they would scarp the project instead of giving talks at conferences. They are clearly confident that FSD will work, to the point of promising it "this year"... coincidentally, they've been promising "this year" for the last 7 years. And FSD still fails in spectacular/hilarious/horrifying ways, and isn't _remotely_ close.
- aunty_helen 4y ago
  >After 7 years of work they are clearly confident that this will work or else they would scarp the project instead of giving talks at conferences. Being real for a second, Tesla don’t have an amazing track record of delivery on their ai software side of things.
- p1esk 4y ago
  Hell, can you point to any company at all that can come out of the gate with a chip design competitive with the best design of NVIDIA, a company that makes nothing but chips? Google.

SuperscalarMeme 4y ago

Not to mention that in-house silicon is all about economies of scale, this is even more of a puzzling move

kjksf 4y ago
Tesla has a path to economies of scale: they already announced that if Dojo works as expected they'll make it available to others as an AWS-style service. Which is brilliant: they might end up making money on this. AI is clearly here to stay. The demand for AI training will clearly explode in the future. Running training in-house is not easy or cheap. You don't just plug in 1000 NVIDIA GPUs. You need massive up-front payment for GPUs and you're basically running your own extremely energy hungry datacenter . Tesla might built and operate massive datacenters. They'll use as much as they need for internal needs and sell the remaining capacity to others. This might take 5 years but the path to do it is clear.
- scoopertrooper 4y ago
  I don’t see how they’re going to commercialise this as a cloud compute service. For one, they’ve built a chip that operates in a fundamentally different way to other chips. So any other company that wanted to use it would have to invest a considerable amount of resources in building up the institutional knowledge to use it effectively. Additionally, the lack of virtual memory and multi-tasking support renders it pretty much impossible to divide up compute between multiple customers. So, commercialising this would require customers renting out the whole unit, which is contrary to how cloud computing usually works. Are there companies out there that have the capital and use cases necessary to fit into Dojo Cloud? Maybe, though not one I’ve worked for. Would they trust the stable genius currently heading up Tesla enough to make such an investment? Perhaps, but I wouldn’t, but what do I know?
  - kjksf 4y ago
    Part of the plan is to have PyTorch compatibility. Dojo has it's own IR but they also have a PyTorch to Dojo compiler. People's opinion of Musk wont matter: either Dojo will be a capable service at a good price or it won't. People will use it based on merits.

dragontamer 4y ago

Dojo is rumored 7nm to boot. They'll be competing with a process node disadvantage. TSMC is down to 3nm and 5nm already. AMD chips like MI250x are 5nm with tensor matrix multiplication units. NVidia Hopper will be like 4nm IIRC?

kjksf 4y ago
During first AI day Tesla already said that they have plans and ideas for 10x improvements for Dojo v2. And I'm sure one of the improvements is to use 5nm or 3nm process. They've already been working on this for almost 7 years. This is not some side projects but a serious operation. They have to ship something sometime and they decided to ship this now. I'm confident that they know about 5nm process and chose 7nm for good reasons. There will be next version and next process.
- sbierwagen 4y ago
  >And I'm sure one of the improvements is to use 5nm or 3nm process. Implementing a process shrink is not just scaling the masks by the appropriate percentage. It often is a completely different optical train, at a different wavelength, different pellicle, changing the refractive index of the immersion fluid, different multiple patterning. It takes months (years?) of work. For many applications it's worth it, but it's not at all a slam dunk. The vast majority of ASICs are designed for a particular node and never upgraded. It's the kind of crazy long-term speculative capital investment TSLA might have indulged in when the stock was at $414 a share, but it's nowhere remotely near that today. They're already engaging in layoffs, today. Why spend the money?
  - rbanffy 4y ago
    > vast majority of ASICs are designed for a particular node and never upgraded They are, if the new node is cheap enough to justify the investment in shrinking the design. In most ASICs, being faster won’t make people rip out their embedded electronics for new models.
panick21_ 4y ago

pookeh 4y ago

Perhaps they do this so they can carry over this tech into their cars. If you are making millions of cars per year, with tens or hundreds of chips per car, it is easy to reason doing this on your own instead of outsourcing it.

jillesvangurp 4y ago

I think they developed this during the years where all the nvidia hardware was gobbled up by crypto miners. The resulting pricing probably did not help either. Tesla is big enough that they can afford to experiment a little. This sounds like an experiment that actually worked. Once they had it working, doing more of it and iterating on it is just business as usual for them.

cyber_kinetist 4y ago
> This sounds like an experiment that actually worked. Are you sure? We haven’t even seen the chip in any physical form yet. We haven’t seen it running any code.
- seunosewa 4y ago
  The self-driving AI they use this chip for appears to be less than impressive from my perspective. Its mistakes seem to come up in the news quite frequently.
  - johnsimer 4y ago
    Dojo’s D1 chip for training AFAIK is not being used yet. Tesla still trains with their ~7000 cluster of A100s And the D1 is for their training machine anyways. Not their inference chip in the cars
  - jillesvangurp 4y ago
    Are you aware of a better implementation though? It might not be perfect; it's still miles ahead of competing systems. Yes, it makes mistakes. But that's a rather poor argument to dismiss the whole strategy and technology. From my point of view, Tesla is converging on good enough more rapidly than anyone else in the industry. Courtesy of their AI strategy and technology; including the in house developed chips.
    - friendzis 4y ago
      > Tesla is converging on good enough This is a big problem, though. Any ADAS is a safety critical system and while "good enough" can help push volumes, it is extremely bad from safety perspective. If you really want to read on this u/adamjosephcook/ has some awesome writing going into far deeper detail than I could explaining this particular issue.

torginus 4y ago

Let me play the devil's advocate - when you have an $1T company, investing a couple million into something like this signals to investors that you have big plans for the future and have a leg up on the competition in the self-driving space. Even if that increases the stock price by 1%, this investment pays for itself a hundred times over.

lclarkmichalek 4y ago
Setting up a hardware platform of this kind that is useable in production costs significantly more than a couple of million
- ricardobeat 4y ago
  Still a blip for a company that was toying with $1B in Bitcoin.
baq 4y ago
this is more like couple hundred million TBH
- dragontamer 4y ago
  Hundred million for the chip alone, a few hundred million for software (compiler, assembler, router, protocols, communications, load balancing, server, supercomputer utilization, power management). The $100-million chip might very well be the easy part of this project...
  - sinenomine 4y ago
    As a software person it baffles me how we tend to overestimate our value compared to hardware. The chip design field is really demanding, especially with deep submicron processes and with rigorous testing & verification culture that blows out of the water almost everything software organizations do. I doubt developing a compiler for this chip will be as costly as your estimate, especially given mature opensource compiler frameworks like LLVM (already adapted to similar TPU-like architectures) and upcoming pytorch glow, apache TVM ML-specific compiler frameworks. And especially given in our industry there are people with advanced degrees who are willing to jump through additional hoops and even accept pay cuts, just to work on a cool compiler. I respect the hardware for what it is and what it could enable.