5 ms·

I'm always curious about the decision-making progress when someone decides to make their own ASIC when there are somewhat reasonable commercial alternatives. Wh

I'm always curious about the decision-making progress when someone decides to make their own ASIC when there are somewhat reasonable commercial alternatives. What was the advantage here for Tesla?

mchusma 5y ago
I think I'm this case, the main advantage is controlling their own destiny when it comes to building the types of models they need. I think in 25% of cases it will not get them significantly more performance vs Nvidia. There is a 50% chance that they can outperform off the shelf chips by a significant amount to make it maybe worth it. (This is pretty likely because dedicated hardware tends to outperform general hardware). However, there is maybe a 25% risk buying Nvidia doesn't get them there soon. So building their own chips de-risks the worst case, and it's probably not that much more expensive (at Tesla scale). So seems like a pretty good bet to me.
- dragontamer 5y ago
  There are plenty of other, innovative companies specializing on FP16 matrix multiplication systolic arrays. For one, Google TPU. Another: Cerebras wafer scale AI. AMD MI100. Etc etc. Even if they screwed the pooch with Nvidia, there are plenty of competitors in this space. Now Tesla has to build its own software stack for large scale distributed learning, which might be harder than the chip design. Is Tesla really the kind of company that wants to carry the expensive loadstone of training and inference software + hardware? It's not like PyTorch is gonna run on this thing unless they create a fork. And a huge advantage of things like NVidia are NVlink / NVswitch. Both hardware, and software, that efficiently distributes data at 600GBps across your GPU clusters.
  - panick21 5y ago
    > Is Tesla really the kind of company that wants to carry the expensive loadstone of training and inference software + hardware? Yes. They are very much that kind of company. Tesla has been pushing vertical integration and that is very much Elon Musk whole approach for most of his companies. Doing your own battery manufacturing and even supply chain is considerably more expensive and complex compared to making a chip and getting some software developers.
    - dragontamer 5y ago
      Battery manufacturing isn't vertically integrated at all. They use Panasonic cells and are dependent on Panasonic's half of the gigafactory.
      - panick21 5y ago
        Yes, so far they are doing that. In fact they use a mix of Panasonic, CATL and LG. However they are working on having their own battery and battery factory design. They are even working on their own cathode manufacturing plants. They have a very large 'pilot' plant in California to test a completely redesigned battery manufacturing system. They are actively building additionally battery factories in Austin and Berlin and have a equipment on order for these factories. For the Battery factory in Berlin they have received funding from the European Commission for this. Tesla will not be fully vertically integrated but they will provide a large parts of their own cells and likely a increasing share over the coming years.
        dragontamer 5y ago
        > However they are working on having their own battery and battery factory design. There's no realistic plan for them to get off of Panasonic's cells. They have a purchase agreement, and Panasonic owns half the Nevada Gigafactory. They have to wait for Panasonic to sell their half of the Gigafactory back to them, during which they'd not have any cell production going on. They basically need to build another US-factory before they even have a chance to become vertically integrated in the USA.
        panick21 5y ago
        They don't want to stop working with suppliers. Batteries are massively constraint, they will buy everything they can from LG, CATL and Panasonic. That is still not enough, that is why they are also building their own. Panasonic in-fact just added another line in Nevada. These cells are required for Model Y/3. The new cells that they produce themselves will be for some Model Y, Cybertruck and Semi. I'm not sure why you insist on disagreeing with a simple fact, Tesla is building its own battery plants. They have one battery factory in the US and are already building a second one. This is literally a fact.
  - londons_explore 5y ago
    I see only two reasons to do it yourself with multiple very capable third parties out there which pretty much exactly match the requirements: 1) You hope to make it available to external customers and turn it into a scale business. Your internal use is just the first customer. You need your own hardware so you can keep costs down at large scales. 2). Someone's pet project was to design their own ML ASIC and they thought it would look very good on their CV, and the CEO took the bait. Hopefully it's the first case!
    - mschuster91 5y ago
      There's a third and fourth reason: the need to keep the secret sauce secret and keep others from replicating it, and that existing vendors aren't flexible enough to offer what they want. For all that Elon Musk values openness in some areas (e.g. putting Hyperloop into public domain), he prefers keeping stuff as vertically integrated as possible for everything he deems to be essential to the business, for maximum control. Tesla has stakes in lithium mining operations, SpaceX has their own metallurgy team and IIRC also a foundry, and while they are using a Liebherr crane at the moment they are thinking about building their own. And for that, it makes sense - SpaceX is only one of Liebherr's customers while SpaceX depends on a crane that fits their needs - so either they get Liebherr to customize their crane or they build their own.
taylorportman 5y ago
With the recent buzz about AI semiconductor design and drive for domestic semiconductor manufacture they are likely positioning themselves to be capable of investing in the next generation of fabrication. Leverage with the government and potential funding.
- option 5y ago
  they aren’t manufactured domestically. TSMC is in Taiwan. I doubt very much that Tesla will want to own fabs.
  - systemvoltage 5y ago
    I wonder if Elon has plans to tackle semiconductor manufacturing and re-thinking the entire industry from bottom-up? I've worked in semiconductor industry and it is super old school stuck in the cold-war era management, procedures and culture.
  - tantony 5y ago
    Tesla has previously used Samsung's fab in Texas for their HW3 chip. Is there any source saying D1 is built by TSMC instead?
ttul 5y ago
Rolling your own ASIC makes sense if you need to churn out enormous quantities for your own use. The actual cost of fabrication is largely weighted toward non-recurring engineering costs. Once the printing press fires up, chips are very inexpensive. Does Tesla need tens of thousands of these things?
- greesil 5y ago
  Only if they're going to put it in their cars, right?
- minhazm 5y ago
  In the demo they said their ExaPOD is 3000 of these chips. They have 10 cabinets with 6 tiles each, each tile has 25 of these D1 chips in it. If they’re successful with this they’ll likely build out multiple of these clusters.
mirker 5y ago
There are basically two options: Nvidia (A100) and Google (TPU). Both are pretty closed and TPU also has restrictions on competitors (e.g., like Tesla is to Waymo). Factor in cost over the ~2 year depreciation curve and I’d imagine it gets kind of easy to convince management. Given how there are also dozens of similar attempts in the space, I’d wager it’s not as hard as designing a general purpose CPU. George Hotz has a few interviews with a similar stance, and his company is in the space as well. For example, each A100 GPU is ~$10000. You probably want 8 for a research workstation, so (with CPUs and other overhead) that’s over $100k for a researcher to use for 2-4 years. I don’t know how many GPUs Tesla needs in aggregate, but if it’s 1000, then that’s $10M in GPUs. After capital expenses, you have operating expenses, and custom accelerators like TPU have higher performance per watt on the order of 2x. Combining expenses, TPU is 2-3x cheaper than GPUs (https://m-cacm.acm.org/magazines/2020/7/245702-a-domain-specific-supercomputer-for-training-deep-neural-networks/fulltext https://m-cacm.acm.org/magazines/2020/7/245702-a-domain-spec...).

I'm always curious about the decision-making progress when someone decides to make their own ASIC when there are somewhat reasonable commercial alternatives. What was the advantage here for Tesla?

mchusma 5y ago

I think I'm this case, the main advantage is controlling their own destiny when it comes to building the types of models they need. I think in 25% of cases it will not get them significantly more performance vs Nvidia. There is a 50% chance that they can outperform off the shelf chips by a significant amount to make it maybe worth it. (This is pretty likely because dedicated hardware tends to outperform general hardware). However, there is maybe a 25% risk buying Nvidia doesn't get them there soon. So building their own chips de-risks the worst case, and it's probably not that much more expensive (at Tesla scale). So seems like a pretty good bet to me.

dragontamer 5y ago
There are plenty of other, innovative companies specializing on FP16 matrix multiplication systolic arrays. For one, Google TPU. Another: Cerebras wafer scale AI. AMD MI100. Etc etc. Even if they screwed the pooch with Nvidia, there are plenty of competitors in this space. Now Tesla has to build its own software stack for large scale distributed learning, which might be harder than the chip design. Is Tesla really the kind of company that wants to carry the expensive loadstone of training and inference software + hardware? It's not like PyTorch is gonna run on this thing unless they create a fork. And a huge advantage of things like NVidia are NVlink / NVswitch. Both hardware, and software, that efficiently distributes data at 600GBps across your GPU clusters.
- panick21 5y ago
  > Is Tesla really the kind of company that wants to carry the expensive loadstone of training and inference software + hardware? Yes. They are very much that kind of company. Tesla has been pushing vertical integration and that is very much Elon Musk whole approach for most of his companies. Doing your own battery manufacturing and even supply chain is considerably more expensive and complex compared to making a chip and getting some software developers.
  - dragontamer 5y ago
    Battery manufacturing isn't vertically integrated at all. They use Panasonic cells and are dependent on Panasonic's half of the gigafactory.
    - panick21 5y ago
      Yes, so far they are doing that. In fact they use a mix of Panasonic, CATL and LG. However they are working on having their own battery and battery factory design. They are even working on their own cathode manufacturing plants. They have a very large 'pilot' plant in California to test a completely redesigned battery manufacturing system. They are actively building additionally battery factories in Austin and Berlin and have a equipment on order for these factories. For the Battery factory in Berlin they have received funding from the European Commission for this. Tesla will not be fully vertically integrated but they will provide a large parts of their own cells and likely a increasing share over the coming years.
      - dragontamer 5y ago
        > However they are working on having their own battery and battery factory design. There's no realistic plan for them to get off of Panasonic's cells. They have a purchase agreement, and Panasonic owns half the Nevada Gigafactory. They have to wait for Panasonic to sell their half of the Gigafactory back to them, during which they'd not have any cell production going on. They basically need to build another US-factory before they even have a chance to become vertically integrated in the USA.
        panick21 5y ago
        They don't want to stop working with suppliers. Batteries are massively constraint, they will buy everything they can from LG, CATL and Panasonic. That is still not enough, that is why they are also building their own. Panasonic in-fact just added another line in Nevada. These cells are required for Model Y/3. The new cells that they produce themselves will be for some Model Y, Cybertruck and Semi. I'm not sure why you insist on disagreeing with a simple fact, Tesla is building its own battery plants. They have one battery factory in the US and are already building a second one. This is literally a fact.
- londons_explore 5y ago
  I see only two reasons to do it yourself with multiple very capable third parties out there which pretty much exactly match the requirements: 1) You hope to make it available to external customers and turn it into a scale business. Your internal use is just the first customer. You need your own hardware so you can keep costs down at large scales. 2). Someone's pet project was to design their own ML ASIC and they thought it would look very good on their CV, and the CEO took the bait. Hopefully it's the first case!
  - mschuster91 5y ago
    There's a third and fourth reason: the need to keep the secret sauce secret and keep others from replicating it, and that existing vendors aren't flexible enough to offer what they want. For all that Elon Musk values openness in some areas (e.g. putting Hyperloop into public domain), he prefers keeping stuff as vertically integrated as possible for everything he deems to be essential to the business, for maximum control. Tesla has stakes in lithium mining operations, SpaceX has their own metallurgy team and IIRC also a foundry, and while they are using a Liebherr crane at the moment they are thinking about building their own. And for that, it makes sense - SpaceX is only one of Liebherr's customers while SpaceX depends on a crane that fits their needs - so either they get Liebherr to customize their crane or they build their own.

taylorportman 5y ago

With the recent buzz about AI semiconductor design and drive for domestic semiconductor manufacture they are likely positioning themselves to be capable of investing in the next generation of fabrication. Leverage with the government and potential funding.

option 5y ago
they aren’t manufactured domestically. TSMC is in Taiwan. I doubt very much that Tesla will want to own fabs.
- systemvoltage 5y ago
  I wonder if Elon has plans to tackle semiconductor manufacturing and re-thinking the entire industry from bottom-up? I've worked in semiconductor industry and it is super old school stuck in the cold-war era management, procedures and culture.
- tantony 5y ago
  Tesla has previously used Samsung's fab in Texas for their HW3 chip. Is there any source saying D1 is built by TSMC instead?

ttul 5y ago

Rolling your own ASIC makes sense if you need to churn out enormous quantities for your own use. The actual cost of fabrication is largely weighted toward non-recurring engineering costs. Once the printing press fires up, chips are very inexpensive. Does Tesla need tens of thousands of these things?

greesil 5y ago
Only if they're going to put it in their cars, right?
minhazm 5y ago
In the demo they said their ExaPOD is 3000 of these chips. They have 10 cabinets with 6 tiles each, each tile has 25 of these D1 chips in it. If they’re successful with this they’ll likely build out multiple of these clusters.

mirker 5y ago

There are basically two options: Nvidia (A100) and Google (TPU). Both are pretty closed and TPU also has restrictions on competitors (e.g., like Tesla is to Waymo). Factor in cost over the ~2 year depreciation curve and I’d imagine it gets kind of easy to convince management. Given how there are also dozens of similar attempts in the space, I’d wager it’s not as hard as designing a general purpose CPU. George Hotz has a few interviews with a similar stance, and his company is in the space as well. For example, each A100 GPU is ~$10000. You probably want 8 for a research workstation, so (with CPUs and other overhead) that’s over $100k for a researcher to use for 2-4 years. I don’t know how many GPUs Tesla needs in aggregate, but if it’s 1000, then that’s $10M in GPUs. After capital expenses, you have operating expenses, and custom accelerators like TPU have higher performance per watt on the order of 2x. Combining expenses, TPU is 2-3x cheaper than GPUs (https://m-cacm.acm.org/magazines/2020/7/245702-a-domain-specific-supercomputer-for-training-deep-neural-networks/fulltext https://m-cacm.acm.org/magazines/2020/7/245702-a-domain-spec...).