2 ms·

Random thoughts: I thought the fact they have 1,000 data labelers was interesting. I'm unsure about their dojo efforts. Seems cool, seems better than Nvidia b

Random thoughts: I thought the fact they have 1,000 data labelers was interesting. I'm unsure about their dojo efforts. Seems cool, seems better than Nvidia but not 10x better (hard to tell it's not apples to apples). For example their dojo cluster us supposed to have over an exaflop vs a A100 superpod at 700pb (seems like I/O is over 2x better which I understand conceptually why it matters but not real world implications). By the time this gets production ready (next year) seems possible Nvidia will have a replacement that is about as good or better. Network architecture was not using video before, it was using 2-3 frames, which seemed obviously not going to work so seems better now. The updated network architecture plus the fact they need to train with a supercomputer to me means they will almost certainly need a new inference chip. They didn't announce a v3 of their FSD computer, but that is probably the physical component blocking FSD rollout. My take on timeline for actual compete everywhere self driving (robotaxi) seems to be: -dojo isn't even in production until next year, so no earlier than that. -FSD chip v3 probably not going to be design finalized until they use some dojo models in it. That plus chip shortages means it's unlikely to happen at scale until 2023. So I think this presentation was to make it clear the large about of progress they are making (very impressive), but IMO this was also a subtle way of making 2023 the new target FSD date. The alternative is they don't think they need dojo to get FSD, and if that is the case why bother building a fully integrated chip manufacturing process?

dawnerd 5y ago
V3 is what’s currently shipping right now. Elon called it FSD 1 and said FSD 2 might come with Cybertruck.
_coveredInBees 5y ago
They are already deploying the kinds of models talked about here in their FSD beta software (along with regular FSD/AP software) and they run just fine on current FSD hardware in the cars with acceptable latency. Just because you need a monstrous supercomputer to train these models doesn't mean you cannot run inference on them on much more light-weight devices. Some reasons being: 1. Training requires them to make use of ridiculous amount of data which requires large GPU compute + RAM and super high bandwidth to keep the GPUs fed at all times. Backpropagation requires a LOT of bookkeeping that adds on a lot of RAM requirement at train-time. 2. Inference is a lot easier. You just need to worry about keeping up with your 8 cameras and running them through your network. You also aren't calculating any gradients and dealing with needing to do any of the bookkeeping for backpropagation so you need a LOT less RAM 3. There are a ton of tricks you can do to make a trained network a lot more efficient for inference such as lower precision models (int8 or in their case, CFP8), network pruning, compiler tricks, etc. It is quite impressive that they've managed to fit all this into their current V3 hardware. That being said, they are definitely starting to really hit the limits of V3 and in the long-run will likely need to upgrade the FSD hardware on newer cars. They are probably trying to avoid having to provide a retrofit for all cars in the fleet as that would be prohibitively expensive.

dawnerd 5y ago

V3 is what’s currently shipping right now. Elon called it FSD 1 and said FSD 2 might come with Cybertruck.

_coveredInBees 5y ago

They are already deploying the kinds of models talked about here in their FSD beta software (along with regular FSD/AP software) and they run just fine on current FSD hardware in the cars with acceptable latency. Just because you need a monstrous supercomputer to train these models doesn't mean you cannot run inference on them on much more light-weight devices. Some reasons being: 1. Training requires them to make use of ridiculous amount of data which requires large GPU compute + RAM and super high bandwidth to keep the GPUs fed at all times. Backpropagation requires a LOT of bookkeeping that adds on a lot of RAM requirement at train-time. 2. Inference is a lot easier. You just need to worry about keeping up with your 8 cameras and running them through your network. You also aren't calculating any gradients and dealing with needing to do any of the bookkeeping for backpropagation so you need a LOT less RAM 3. There are a ton of tricks you can do to make a trained network a lot more efficient for inference such as lower precision models (int8 or in their case, CFP8), network pruning, compiler tricks, etc. It is quite impressive that they've managed to fit all this into their current V3 hardware. That being said, they are definitely starting to really hit the limits of V3 and in the long-run will likely need to upgrade the FSD hardware on newer cars. They are probably trying to avoid having to provide a retrofit for all cars in the fleet as that would be prohibitively expensive.