4 ms·

Watch the presentation from 6 months ago, where they explain the decision to build their own hardware for inferring : https://youtu.be/Ucp0TTmvqOE?t=4309 https:

by antpls 7y ago

Watch the presentation from 6 months ago, where they explain the decision to build their own hardware for inferring : https://youtu.be/Ucp0TTmvqOE?t=4309 https://youtu.be/Ucp0TTmvqOE?t=4309 It's not surprising that they also build the hardware for training. Correct me if I'm wrong, but Google use the same TPUs for training and inference, because the underlying operations are the same : multiply then add numbers. Once Tesla built the hardware for inferring, the design of the hardware for training is probably similar. Unlike Google's TPUs, Tesla have a specific use case for the hardware (computer vision for automotive), and maybe than means they can further optimize the computation pipeline with their own specialized hardware.

modeless 7y ago
Google has Edge TPUs for use outside datacenters, and they don't support training. Neither do the chips Tesla made for their cars. It's a pretty different problem.
- antpls 7y ago
  I wouldn't be so sure. Edge TPUs could be the exact same architecture than Google Cloud TPUs, but as you need less computation power for inferring than training, they have simply less transistors on the die and could be underclocked. In other words, Cloud TPUs could be the same architecture than Edge TPUs but scaled to an higher frequency and more packed. I guess we need sources to confirm.
  - modeless 7y ago
    Edge TPUs support 8 bit integer math only. Training is floating point. That's not a small change. https://coral.withgoogle.com/docs/edgetpu/models-intro/#model-requirements https://coral.withgoogle.com/docs/edgetpu/models-intro/#mode...
  - londons_explore 7y ago
    Training is currently done in floating point math, whereas inference can be done fixed point without much loss of performance. Fixed point is ~10x cheaper in terms of power and silicon area for equal performance. Also, training requires a lot more RAM per unit of compute, since it needs to store all past layer activations, whereas for inference, that is unnecessary. As far as I know, no player who has developed dedicated ML hardware (as opposed to using GPU's) uses the same hardware for both inference and training.
spyder 7y ago
Very good video, it contains answers to many of the questions that people are speculating here about and other interesting things about Tesla's custom chip. - It's under 100W so they can retrofit into old cars - lower part cost, so they can do full redundancy with doubling the parts - they estimated that 50 TOPS is needed for self-driving - lower latency with batch size of 1 compared to TPU's 256 + GPU for post-processing - security: only code signed by Tesla can run on the chip - at the time (2016) there was no neural net accelerator chips - some part's are built from bought IP (so not reinventing them) Probably things like the 12 ARM CPUs, LP DDR4 memory, video encoder, maybe the separate post-processing GPU too... - physical size of the board is small - performance example: on CPU 1.5 FPS, on GPU (600 gflop) 17 FPS, on Tesla's NN accelerator 2100 FPS - Besides the convolution even the ReLU and Pooling is implemented in hardware - Paying attention to the energy efficiency down to the arithmetic and data type usage. - The silicon cost is less than their previous hardware (HW 2.5) - old hardware 110 FPS new one 2300 FPS - 144 TOPS compared to NVidia's Drive Xavier 21 TOPS

by antpls 7y ago

modeless 7y ago

Google has Edge TPUs for use outside datacenters, and they don't support training. Neither do the chips Tesla made for their cars. It's a pretty different problem.

antpls 7y ago
I wouldn't be so sure. Edge TPUs could be the exact same architecture than Google Cloud TPUs, but as you need less computation power for inferring than training, they have simply less transistors on the die and could be underclocked. In other words, Cloud TPUs could be the same architecture than Edge TPUs but scaled to an higher frequency and more packed. I guess we need sources to confirm.
- modeless 7y ago
  Edge TPUs support 8 bit integer math only. Training is floating point. That's not a small change. https://coral.withgoogle.com/docs/edgetpu/models-intro/#model-requirements https://coral.withgoogle.com/docs/edgetpu/models-intro/#mode...
- londons_explore 7y ago
  Training is currently done in floating point math, whereas inference can be done fixed point without much loss of performance. Fixed point is ~10x cheaper in terms of power and silicon area for equal performance. Also, training requires a lot more RAM per unit of compute, since it needs to store all past layer activations, whereas for inference, that is unnecessary. As far as I know, no player who has developed dedicated ML hardware (as opposed to using GPU's) uses the same hardware for both inference and training.

spyder 7y ago

Very good video, it contains answers to many of the questions that people are speculating here about and other interesting things about Tesla's custom chip. - It's under 100W so they can retrofit into old cars - lower part cost, so they can do full redundancy with doubling the parts - they estimated that 50 TOPS is needed for self-driving - lower latency with batch size of 1 compared to TPU's 256 + GPU for post-processing - security: only code signed by Tesla can run on the chip - at the time (2016) there was no neural net accelerator chips - some part's are built from bought IP (so not reinventing them) Probably things like the 12 ARM CPUs, LP DDR4 memory, video encoder, maybe the separate post-processing GPU too... - physical size of the board is small - performance example: on CPU 1.5 FPS, on GPU (600 gflop) 17 FPS, on Tesla's NN accelerator 2100 FPS - Besides the convolution even the ReLU and Pooling is implemented in hardware - Paying attention to the energy efficiency down to the arithmetic and data type usage. - The silicon cost is less than their previous hardware (HW 2.5) - old hardware 110 FPS new one 2300 FPS - 144 TOPS compared to NVidia's Drive Xavier 21 TOPS