5 ms·

A lot of people want to talk about the Tesla Bot. That is clearly in the early stages. Can we talk about Dojo? This seems like a pretty crazy projects. A fully

A lot of people want to talk about the Tesla Bot. That is clearly in the early stages. Can we talk about Dojo? This seems like a pretty crazy projects. A fully integrated super computer designed from core to multi-rack design. I have never really seen a system integrated like that? Are there other examples of people doing this? They seem to have designed some kind of sled that connect to power and liquid cooling system. Can somebody with some expertise in large scale training compare this to state of the art?

refulgentis 5y ago
The same robot is supposed to be out sometime in 2022, unbelievable. In Q+A they said the Dojo stuff was an aspirational vision, not a production-ready design Elon is a self parody at this point.
- bhauer 5y ago
  > The same robot is supposed to be out sometime in 2022, unbelievable. How did you get that interpretation from Elon saying they're aiming to have a prototype next year? It's a recruiting event. Presumably in part to recruit people to work on this new project.
- panick21 5y ago
  That not at all the case. The robot was not supposed to be 'out' next year. He said they might have the first prototype built next year. Those are very different things. And Dojo is clearly advanced state, as they show the hardware and showed how far they are. They haven't built the full rack of course. > Elon is a self parody at this point. I think people who act like everything he says is some made up nonsense are a parody of themselves if anything.
  - refulgentis 5y ago
    I think you understand why people might think extending FSD to robots is a bit...loaded...considering Musk has been promising and selling FSD to consumers for years. Remember 1 million robotaxis? There's a much higher chance there's a consumer fraud case on FSD than ten of these robots are ever sold
    - pbreit 5y ago
      The 1m fsd-capable cars are on the road. The features of "FSD" are listed clearly in the buying process. The robots as stated are to be used internally. You seem knee-jerk negative?
ipsum2 5y ago
Taking everything at face value: Dojo is overall a very impressive project! - Communication speed is one of the biggest bottlenecks to large models, so their bandwidth of 4TBps is very smart. - They claim a 1.3x perf/watt improvement, which is not really that great for ASICs compared to GPUs. Perf/watt is probably the most important number in datacenters. - They only use SRAM, no DRAM. This is a huge mistake, which limits their model size. You can only fit a ~10GB model inside a single tile, versus 80GB models for a single A100 GPU. - Software / compiler stack is as or more important than the hardware itself, because it dictates how much real performance you can squeeze out of the chips. I think Tesla will need to heavily focus on this area before getting anywhere close to real-world GPU performance.
- panick21 5y ago
  > - They only use SRAM, no DRAM. This is a huge mistake This seems pretty interesting. They must have had a reason for this approach. Maybe it doesn't matter as much for their usecase. They clearly had the option of adding DRAM if they wanted to. Why not do it?
- rajnathani 5y ago
  This other comment here mentions that the chips on the sides (i.e. sides of the grid composite of chips) have access to DRAM: https://news.ycombinator.com/item?id=28254892 https://news.ycombinator.com/item?id=28254892

refulgentis 5y ago

The same robot is supposed to be out sometime in 2022, unbelievable. In Q+A they said the Dojo stuff was an aspirational vision, not a production-ready design Elon is a self parody at this point.

bhauer 5y ago
> The same robot is supposed to be out sometime in 2022, unbelievable. How did you get that interpretation from Elon saying they're aiming to have a prototype next year? It's a recruiting event. Presumably in part to recruit people to work on this new project.
panick21 5y ago
That not at all the case. The robot was not supposed to be 'out' next year. He said they might have the first prototype built next year. Those are very different things. And Dojo is clearly advanced state, as they show the hardware and showed how far they are. They haven't built the full rack of course. > Elon is a self parody at this point. I think people who act like everything he says is some made up nonsense are a parody of themselves if anything.
- refulgentis 5y ago
  I think you understand why people might think extending FSD to robots is a bit...loaded...considering Musk has been promising and selling FSD to consumers for years. Remember 1 million robotaxis? There's a much higher chance there's a consumer fraud case on FSD than ten of these robots are ever sold
  - pbreit 5y ago
    The 1m fsd-capable cars are on the road. The features of "FSD" are listed clearly in the buying process. The robots as stated are to be used internally. You seem knee-jerk negative?

ipsum2 5y ago

Taking everything at face value: Dojo is overall a very impressive project! - Communication speed is one of the biggest bottlenecks to large models, so their bandwidth of 4TBps is very smart. - They claim a 1.3x perf/watt improvement, which is not really that great for ASICs compared to GPUs. Perf/watt is probably the most important number in datacenters. - They only use SRAM, no DRAM. This is a huge mistake, which limits their model size. You can only fit a ~10GB model inside a single tile, versus 80GB models for a single A100 GPU. - Software / compiler stack is as or more important than the hardware itself, because it dictates how much real performance you can squeeze out of the chips. I think Tesla will need to heavily focus on this area before getting anywhere close to real-world GPU performance.

panick21 5y ago
> - They only use SRAM, no DRAM. This is a huge mistake This seems pretty interesting. They must have had a reason for this approach. Maybe it doesn't matter as much for their usecase. They clearly had the option of adding DRAM if they wanted to. Why not do it?
rajnathani 5y ago
This other comment here mentions that the chips on the sides (i.e. sides of the grid composite of chips) have access to DRAM: https://news.ycombinator.com/item?id=28254892 https://news.ycombinator.com/item?id=28254892