6 ms·
Tesla Packs 50B Transistors onto D1 Dojo Chip
- ksec 5y agoBig Numbers are good for headline. But it doesn't put anything in context. Die Size is 645mm^2 on a 7nm. This is important because we know the reticle limit which is around ~800mm^2. The Nvidia AI Chip has 54 billion transistors with a die size of 826 mm2 on 7nm. I recently saw a Ted Talk, If Content is King, then Context is God. I think it capture everything that is wrong in today's society.
- zamadatix 5y agoThe maximum die size is interesting but not really the point. The context is more the complexity and capability of the chip, for which transistor count is about as good a measure as you're going to fit in the headline. The immediate subheading jumps to telling you FLOPs which is another attempt at summarizing the capabilities of the chip quickly. Once you have the info that it's large and fast the body serves to provide the detailed context. From that view the title serves to identify the primary context well - a very complex chip, come read more about it. One basic thing I didn't see in the body was power consumption though, anyone know more details on that?
- adrian_b 5y ago400 W See e.g. https://semiwiki.com/artificial-intelligence/302502-tesla-dojo-unique-packaging-and-chip-design-allow-an-order-magnitude-advantage-over-competing-ai-hardware/ https://semiwiki.com/artificial-intelligence/302502-tesla-do...
- abc_lisper 5y agoWhere is this Ted talk? I couldn’t find it
- chacham15 5y agoI think theyre referencing this: https://www.youtube.com/watch?v=ZJ4GmZflpPI https://www.youtube.com/watch?v=ZJ4GmZflpPI
- jeffbee 5y agoIs that a lot?
- throwaway4good 5y agoThe Apple M1 chip, which is much smaller and has lower power consumption, has 16B.
- senectus1 5y agoworth pointing out that Dojo is meant to do one thing and one thing only... ML
- jeffbee 5y agoIt can multiply and add!
- jeffbee 5y agoThe nvidia A100 is larger and has 56B.
- coronadisaster 5y agoIs that a lot?
- throwaway4good 5y agoYes.
- shadilay 5y agoM1 is on 5nm.
- Someone 5y agoWhen comparing it to other large designs, I think it’s not exceptional, but also not in the back of the pack. This die is 645mm², or a square inch. We could create a wafer that size in the 1960s (https://en.wikipedia.org/wiki/Wafer_(electronics)#Standard_wafer_sizes https://en.wikipedia.org/wiki/Wafer_(electronics)#Standard_w.... Note these are for circular wafers, so a 1 inch wafer is about ¾ square inch), so in that sense, it isn’t a surprise that we can make such a chip. We couldn’t put 50B transistors on a square inch in the 1960s, though. We can now. https://en.wikipedia.org/wiki/Transistor_count https://en.wikipedia.org/wiki/Transistor_count lists several larger designs. So, the engineering is impressive, but not spectacular. Also, this being a grid of interconnected CPUs means the design is simpler than a single design filling the entire die would be. It’s ‘just’ repeating the same design over and over (possibly with some small variations near the edge) Of course looking at it without knowledge of the state of the art it is astounding that we can even think of constructing machines with 50 billion working parts
- Lio 5y agoThis is made using TSMC's 7nm fab process so surly the number of transistors in this chip is either enabled or limited by that process, isn't it? Honest question, how much is chip design a factor separate to fab process?
- tlb 5y agoDensity is partly a function of the type of circuit. Memory is denser than random logic, for instance. Interconnect eats a lot of area and reduces density. This chip is largely memory and multipliers, both of which are pretty dense. Fab processes improve over time to have higher density and lower defect rate (which allows bigger chips while getting acceptable yield). So it's not surprising to see a chip on the same node but shipping a year or 2 later (than Ampere) having more transistors.
- Lio 5y agoThank you, that's really interesting. So good design will reduces things like interconnect to improve density.
- throwaway4good 5y agoWhat process are these chips made with? It says TSMC 7nm - is that DUV or EUVL?
- tromp 5y agoAn extremely deep question...
- ttul 5y agoI don’t believe 7nm used EUVL, which would keep the cost down, relatively speaking.
- Dunedan 5y agoTSMC offers both DUV (N7, N7P) and EUV (N7+) for 7nm [1]. [1]: https://en.wikichip.org/wiki/7_nm_lithography_process#TSMC https://en.wikichip.org/wiki/7_nm_lithography_process#TSMC
- throwaway4good 5y agoI am curious if euvl (and associated high transistor density) makes sense for this type of processor or it simply would run too hot?
- greesil 5y agoI'm always curious about the decision-making progress when someone decides to make their own ASIC when there are somewhat reasonable commercial alternatives. What was the advantage here for Tesla?
- mchusma 5y agoI think I'm this case, the main advantage is controlling their own destiny when it comes to building the types of models they need. I think in 25% of cases it will not get them significantly more performance vs Nvidia. There is a 50% chance that they can outperform off the shelf chips by a significant amount to make it maybe worth it. (This is pretty likely because dedicated hardware tends to outperform general hardware). However, there is maybe a 25% risk buying Nvidia doesn't get them there soon. So building their own chips de-risks the worst case, and it's probably not that much more expensive (at Tesla scale). So seems like a pretty good bet to me.
- dragontamer 5y agoThere are plenty of other, innovative companies specializing on FP16 matrix multiplication systolic arrays. For one, Google TPU. Another: Cerebras wafer scale AI. AMD MI100. Etc etc. Even if they screwed the pooch with Nvidia, there are plenty of competitors in this space. Now Tesla has to build its own software stack for large scale distributed learning, which might be harder than the chip design. Is Tesla really the kind of company that wants to carry the expensive loadstone of training and inference software + hardware? It's not like PyTorch is gonna run on this thing unless they create a fork. And a huge advantage of things like NVidia are NVlink / NVswitch. Both hardware, and software, that efficiently distributes data at 600GBps across your GPU clusters.
- panick21 5y ago> Is Tesla really the kind of company that wants to carry the expensive loadstone of training and inference software + hardware? Yes. They are very much that kind of company. Tesla has been pushing vertical integration and that is very much Elon Musk whole approach for most of his companies. Doing your own battery manufacturing and even supply chain is considerably more expensive and complex compared to making a chip and getting some software developers.
- mrtnmcc 5y agoI wonder how their neural network structures informed the hardware design, such as the dimensions of tensor products. Or is Dojo trying for as general purpose ML as possible? I imagine there is a tension between software and hardware teams where Karpathy's team is always changing things while the hardware team wants specs/reqs. The "tiles of tiles" chip architecture seems like an Elon-obvious, let's just scale what we have approach. Do their neural networks map to that multiscale tiling well?
- gautamcgoel 5y agoQuestion: how many chips does Tesla need to buy in order to get a reasonable unit price per chip? Obviously <10k is too small, but is 100k reasonable? 1M?
- ehsankia 5y agoIs that only considering the price per cheap deal they get from TSMC, or also including the cost of d&d?
- gautamcgoel 5y agoI'd be interested in both numbers. D&D = design and ?
- adventured 5y agoDesign and development.
- sonium 5y agoThe whole point of the die on silicone seems to be that this maximizes the interface bandwidth and minimize latency between the dies. If this true the next step would be to bring the multi die modules as close as possible in three dimensions to ultimately build a borg-cube like structure in zero-g with a power source at its core.
- m3kw9 5y agoBlack box numbers would be better in terms of physical size, power usage and comparable training/inference times. Everything else is hype.
- TekMol 5y agoThe article starts with this statement: Artificial intelligence (AI) has seen a broad adoption over the past couple of years. And continues: At Tesla, who as many know is a company that works on electric and autonomous vehicles, AI has a massive value to every aspect of the company's work. Who is writing like this? And why? What would Tom's Hardware lose if they left out this type of cheap fillwords? Should I also start writing like this? Is this type of "reader hostile writing" a new thing or have newspapers always written like this? These are not rhetorical questions. I am honestly confused.
- CharlesW 5y agoIMO that first paragraph is great, especially for readers who may not have your level of industry knowledge and technical acumen. It efficiently contextualizes the article and addresses a common complaint that I often see even on HN — the failure to clearly answer "What is this and why does this matter?"
- TekMol 5y agoSo you tell me the readers of a hardware site who click on a title "Tesla Packs 50 Billion Transistors Onto D1 Dojo Chip" hear for the first time about the term Artificial Intelligence?
- CharlesW 5y agoThe article doesn't explain or even define "AI", so I'm going to respectfully disagree with your premise. I understand that you consider the writing objectively "hostile", but the simpler explanation is that you're just not the audience.
- SmellTheGlove 5y agoIt reads like Bart Simpson’s report on Libya.
- Dunedan 5y ago> […] AI has a massive value to every aspect of the company's work. That's also just wrong. During the recent "Tesla AI Day", when asked during Q/A, Elon Musk specifically mentioned that they intentionally use machine learning only for very few cases: Q: "Is Tesla using machine learning within its manufacturing, design or any other engineering processes?" Elon: "I discourage use of machine learning, because it's really difficult. Unless you have to use machine learning, don't do it. It's usually a red flag when somebody is saying 'We wanna use machine learning to solve this task'. I'm like: That sounds like bullshit. 99.9% of the time you don't need it." https://www.youtube.com/watch?v=j0z4FweCy4M&t=9307s https://www.youtube.com/watch?v=j0z4FweCy4M&t=9307s
- minhazm 5y agoTesla actually has a lot of expertise in chip design in Pete Bannon and formerly Jim Keller. I think most people know who Jim Keller is, but if not you can read his wikipedia[1]. Pete Bannon is also an industry giant and worked with Jim Keller at PA Semi and subsequently Apple on their A series chips. These two have decades of experience designing chips that went into tens of millions of devices. Tesla’s FSD computer is in hundreds of thousands of cars. They know what they’re doing. https://en.wikipedia.org/wiki/Jim_Keller_(engineer) https://en.wikipedia.org/wiki/Jim_Keller_(engineer)
- millerm 5y agoFSD computer is in over a million cars, btw. It’s still hundreds of thousands, but more like one thousand thousand.
- mupuff1234 5y agoThe same Wikipedia page also states that Jim Keller left Tesla awhile ago.
- jstandard 5y agoNon-hardware person here. How does the D1 compare to Cerebras WSE-2 wafer chip with 2.6 trillion transistors? The WSE2 is much larger obviously, but I would also think it can result in a large performance boost given everything is on a single chip.