6 ms·
Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead
by modeless 7y ago
Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud?
I can see the case for doing their own edge hardware for the cars (barely), but I really don't think doing training hardware will pay off for them. If they're serious about it, they should spin it out as a separate business to spread the development cost over a larger customer base.
Also, I'm really curious whether the custom hardware in the cars is benefiting them at all yet. Every feature they've released so far works fine on the previous generation hardware with 1/10 the compute power. At some point won't they need to start training radically larger networks to take advantage of all that untapped compute power?
- sdan 7y agoDid they say they were building their own training hardware? I thought it was just their inference hardware (the boards on the teslas)?
- m0zg 7y agoNothing crazy about it. TPU-like stuff is ~10x the energy efficiency of GPUs and several times the speed. When you're spending megawatt-hours and days to train a single model, it adds up in both real and opportunity costs. Also, Google TPU TOS prohibits the use of TPUs for stuff that competes with Google (and I'm assuming with other companies under Alphabet umbrella), at Google's sole determination. Not that it would be a good idea to upload Tesla's proprietary data into Google Cloud even if it did not. Cloud, after all, is just somebody else's computer.
- sdan 7y agoStuff like their TPU and Waymo's Honeycomb Laserbear (something along those lines... their lidar naming system is pretty long) shows that Google is making good products for a limited reach of people. TPU? Seems like it has a lot of potential, but not for people directly competing with them. Waymo's Laserbear lidar? Seems like it has a lot of potential, but not for AV companies directly competing with them. Google's playing this game pretty fiercely... which given their size is pretty bad/daunting.
- deleted 7y ago[deleted]
- modeless 7y ago> Google TPU TOS prohibits the use of TPUs for stuff that competes with Google I don't think this is true. If you're talking about https://news.ycombinator.com/item?id=19855099 https://news.ycombinator.com/item?id=19855099, it doesn't apply to TPU hardware, as is explained in the comments there. > TPU-like stuff is ~10x the energy efficiency of GPUs 10x is probably overstating it when talking about newer GPUs because they have ML hardware in them now. Also, that still doesn't make it a good idea to build your own chips because there will soon be many third party options to choose from. Doing your own chips is a bet that you will out execute dozens of companies ranging from startups to industry giants. Simply taking your pick of the best commercially available options is likely to be a better choice in the near future.
- m0zg 7y agoYep that's the clause. The clause itself is not that problematic for Tesla. What's problematic is that it can be changed over time, and it'd be foolish to single-source something as important as deep learning compute without the option to go elsewhere. Not to mention the rather extravagant Cloud pricing. So Tesla is taking a page out of Steve Jobs' playbook and it will control its own core tech. That's smart, especially considering that they already have bits and pieces of the IP that they'll need.
- thebruce87m 7y agoStay ahead? Are they actually ahead?
- jeffshek 7y agoAt Tesla's scale and priorities, they'd probably be less keen on using external cloud providers. Using TPUs at their scale would certainly require Google's AI consultants to supervise which isn't ideal for Tesla. Not agreeing or disagreeing with their decisions, but if you have the resources, you can certainly design a custom chip that performs a specific type of task very well that beats other competitors. Nvidia's GPUs are have to be reasonably good at training across different NNs. You could have a chip that's exceptional good at training one/two specific types of tasks. For most companies, this would be a bad idea. However, Tesla knows how to produce hardware.
- chronic739i 7y ago> At Tesla's scale and priorities, they'd probably be less keen on using external cloud providers. By hardware-hours, Tesla is hardly one of the top companies training deep networks. Planet Labs (satellite imaging), Netflix, Pornhub, to name a few. What's the info they'd leak to the Google consultants? How much data or TPUs they're using? This is practically public information.
- Udik 7y agoWhat does Netflix do with NNs?
- SloopJon 7y agoTheir own page says, "our recommendation algorithms ... learning characteristics that make content successful ... optimize the production of original movies and TV shows ... optimize video and audio encoding, adaptive bitrate selection, and our in-house Content Delivery Network ... and advertising". https://research.netflix.com/research-area/machine-learning https://research.netflix.com/research-area/machine-learning Here's another article on the subject: https://becominghuman.ai/how-netflix-uses-ai-and-machine-learning-a087614630fe https://becominghuman.ai/how-netflix-uses-ai-and-machine-lea...
- Udik 7y ago
- joenathanone 7y ago>Also, I'm really curious whether the custom hardware in the cars is benefiting them at all yet. Every feature they've released so far works fine on the previous generation hardware with 1/10 the compute power. The latest OTA finally brings a hardware v3 only feature, traffic cone visualization, and traffic cone automatic lane change.
- navigatesol 7y agoTraffic cones today,1 million self driving robo taxis in 6 months, right?
- londons_explore 7y agoI would guess that while the new hardware has the same features, the accuracy might be lower on the old GPU's because they are forced to use smaller networks or to run them at lower frame rates.
- roystonvassey 7y agoAlso, the software part of it (NNs and their algorithms) have been so widely researched and published that competitive advantages here are harder to come by than in hardware RD. Also, vendor lock-in is a huge challenge in the cloud space. I don’t think Tesla would be comfortable with the fact that all their training data sits on a potential competitor’s datacenter.
- jacquesm 7y agoA car is a hardware device as well, and an electric car does not have the kind of power budget that allows you to throw oodles of standard pieces at it without paying a severe penalty in range.
- codeulike 7y agoCompared to the energy needed to move the car, everything else is pretty irrelevant. Power hungry features like Heating/AC only makes a few % difference to range.
- mrep 7y agoFrom autonomy day hacker news comment: "Pegasus consumes about 500Watts, compared to under 100 Watts for Tesla's FSD computer. Elon in particular emphasized the performance per watt (as it's always possible to cram more chips to increase performance if you ignore cost and power consumption). The comparison made in the video: 500Watts for an hour consumes about 2-3 miles of range. In a city in slow traffic, going 12mph, that's a significant range reduction. So you might have a 10% improvement in range for the Tesla ASIC in low speed conditions" [0]. Also, Tesla is/was planning on running these chips even while not in autonomous mode in order to spot new scenarios which it can then record what the car sees and what the human does in order to collect a lot more unique road scenarios to train their models with. [0] Credit Robotbeat: https://news.ycombinator.com/item?id=19729743 https://news.ycombinator.com/item?id=19729743
- maxerickson 7y ago
- antpls 7y agoWatch the presentation from 6 months ago, where they explain the decision to build their own hardware for inferring : https://youtu.be/Ucp0TTmvqOE?t=4309 https://youtu.be/Ucp0TTmvqOE?t=4309 It's not surprising that they also build the hardware for training. Correct me if I'm wrong, but Google use the same TPUs for training and inference, because the underlying operations are the same : multiply then add numbers. Once Tesla built the hardware for inferring, the design of the hardware for training is probably similar. Unlike Google's TPUs, Tesla have a specific use case for the hardware (computer vision for automotive), and maybe than means they can further optimize the computation pipeline with their own specialized hardware.
- modeless 7y agoGoogle has Edge TPUs for use outside datacenters, and they don't support training. Neither do the chips Tesla made for their cars. It's a pretty different problem.
- antpls 7y agoI wouldn't be so sure. Edge TPUs could be the exact same architecture than Google Cloud TPUs, but as you need less computation power for inferring than training, they have simply less transistors on the die and could be underclocked. In other words, Cloud TPUs could be the same architecture than Edge TPUs but scaled to an higher frequency and more packed. I guess we need sources to confirm.
- modeless 7y agoEdge TPUs support 8 bit integer math only. Training is floating point. That's not a small change. https://coral.withgoogle.com/docs/edgetpu/models-intro/#model-requirements https://coral.withgoogle.com/docs/edgetpu/models-intro/#mode...
- londons_explore 7y agoTraining is currently done in floating point math, whereas inference can be done fixed point without much loss of performance. Fixed point is ~10x cheaper in terms of power and silicon area for equal performance. Also, training requires a lot more RAM per unit of compute, since it needs to store all past layer activations, whereas for inference, that is unnecessary. As far as I know, no player who has developed dedicated ML hardware (as opposed to using GPU's) uses the same hardware for both inference and training.
- dna_polymerase 7y agoSo would you trust the company that owns one of your biggest competitors in this field (Waymo) with the stuff that decides over success: data?
- breatheoften 7y agoI think the size of the networks they are training might already be good motivation for developing custom hardware for training. I would expect their training hardware to be something specifically aimed at optimizing memory bandwidth to support distributing training of their “shared” hydra feature. It’s interesting that the shared hydra feature extractor is able to converge as they keep adding more and more output predictions under a training regime of interleaving asynchronous updates to the model from different predictor networks ... Seems to me the formula they are pursuing with custom hardware might be to support a strategy of 1. keep adding more predictions based on same feature 2. Increase the span of time represented by batches used to train the recurrent networks Both pursuits seem very data efficient in terms of the amount of training data they could conceivably collect per unit time of observation ... Custom hardware with a problem specific memory architecture aimed at efficiently supporting training with very large rnn time slices could be developed that’s more about “make it possible to train this proposed model at all” rather than “make it faster/cheaper to train existing common model architectures”. When custom hardware is required to make it possible to train the model they want, the validity of the hardware development cost bet might end up being more about the effectiveness of the model they think they want than it is about maintaining general purpose performance parity vs any off the shelf hardware options ...