6 ms·

> As a point of reference - Google’s TPU V1, which is the one that Google uses to actually run neural networks (the other versions are optimized for training) i

> As a point of reference - Google’s TPU V1, which is the one that Google uses to actually run neural networks (the other versions are optimized for training) is very similar to the specs I’ve outlined above. The TPU V1 is a 2015 chip (publicly announced in 2016, and that Google had been using internally for over a year). 4 years is a big lag in terms of technology. Additionally, the comment about "actually run neural networks " seems to be, AFAIK, plainly wrong. The first version was limited to inference because it could only perform integer operations, while V2 and onward support both training and inference, as they can perform float operations. TPU V1 was never available to the public, but both V2 and V3 are available on Google Cloud. I don't have any info on it, but at this point, I'd expect all V1 chips to have been deprecated and likely removed from production, given the savings in space and power from V2 and V3.

danjayh 8y ago
4 years is a big lag if you're not doing safety critical work. If you are doing safety critical, 4 years isn't particularly fast, but it's not unreasonable. If you've never worked in aerospace or automotive, you would be shocked at the requirements for software and hardware that can kill people if it breaks (assuming that Tesla is following the automotive equivalent of DO-178/DO-254, ISO26262)... I guess the automotive guys do have an advantage because they're not required to have independence for the various stages of verification, but it's still a big chunk of work (more than the actual development).
- ucaetano 8y ago
  > If you are doing safety critical Well, Tesla clearly isn't doing safety critical work on autonomous systems (it wouldn't have even launched "autopilot" if it was), so 4 years is indeed a big lag.
  - torpfactory 8y ago
    Tesla claims [1] that autopilot driving is safer than regular driving in terms of risk of accident per mile. Not sure if I agree with the way they've constructed their statistics, but for a moment take them at face value. Isn't then the safer thing to release and encourage autopilot to the largest extent possible? [1] https://www.tesla.com/blog/q3-2018-vehicle-safety-report?redirect=no https://www.tesla.com/blog/q3-2018-vehicle-safety-report?red...
    - evv 8y ago
      That statistic is wildly and purposefully misleading. It compares the safety of Autopilot (which only drives on the freeway) with human drivers on any road. Accidents-per-mile are already much lower on the freeway. Plus, the statistic ignores the fact that responsibility is actually shared between humans and the car. If autopilot gets into an unsafe situation and gives control to the human at the last moment, the car company might claim that the accident happened under human control. We have a long way to go before we can truly understand the safety of semi-autonomous vehicles.
      - sandworm101 8y ago
        >> If autopilot gets into an unsafe situation and gives control to the human at the last moment We have lots of experience with the reverse, where the human drives but the autopilot rips control away when things go wrong. Traction control, radar-triggered braking, even basic ABS is a machine taking over from the human once things get hairy. Several AirForce fighters have systems that will sometimes ignore the pilot's inputs and save the aircraft from a crash. That's the real path for semi-autonomous cars imho: you do all the driving until the robot steps in to save your ass. (I cannot wait for the day someone tries to sell a car that will automatically obey speed limits regardless of driver input. That's a stock to short.)
      - Faark 8y ago
        I'd even expect their human driven statistics not actually representative or useful for comparisons. Their cars are expensive, mostly excluding young and thus often unexperienced drivers. They also don't particularly target older people, the other group with high accident rates. Controlling for all those variables is hard and not to their benefit, thus I'd never expect their pr team to do so. But I wonder if that might even creep into their automated driving stats... e.g. by drivers taking control in dangerous situations.
    - ucaetano 8y ago
      > for a moment take them at face value Why would you do so? This is like saying "Let's assume the sky is purple. Then, we can conclude that the sky is purple."
  - danjayh 8y ago
    Even if it's not now, their intention is to provide higher levels of SAE automation on the same software as time progresses, which means that the current hardware probably has to hit a higher design assurance level than the current software.
    - ucaetano 8y ago
      Sure, but intentions are irrelevant. What matters is what happens.
- stfwn 8y ago
  Can you elaborate a bit on how such testing is done, or share a good article on the topic? It sounds like a hard problem to need to get things right this bad, or else.
  - miketery 8y ago
    it's been a while, do-178 is concerned with software while do-254 is hardware (both for aviation, FAA standards). Depending on the criticality of the component (e.g. engine control vs radios vs entertainment system) they will abide by different levels (ie DO-178A ... DO-178E). For both software and hardware there will be requirements for documentation, design, verification, and testing. Some even go as far as to requir the implementation of certain functional logic in multiple ways and having concensus logic (eg hardware logic to interpret GPS accuracy/confidence That ends up being broadcast externally). There's also DO-160 that's concerned with environmental requirements (eg temperature, humidity, lighting). This is what makes a radio that should cost $300, cost $5000, and why FAA certified aviation components are expensive compared to uncertified components (eg hobby airplane builds).
    - kejaed 8y ago
      Parent comment really gives a great overview of the process. One slight nitpick is that DO-178B or DO-178C are revisions of the standard and the associated criticality is associated with the Design Assurance Level (DAL) from A to E. One would say software is developed to DO-178C DAL B for example. The DAL is determined before the software is started at the Systems level according to a process described in ARP4754A (Guidelines For Development Of Civil Aircraft and Systems) looking at the system architecture, hazards, and potential mitigations of those hazards, one of which is developing software to a certain DAL.
  - godelski 8y ago
    Not that long ago there was a few HN post about running on the metal that started off with this [0] (also called "Space Shuttle Style). Also speaking of NASA, they - and many other government departments - use a system called Technology Readiness Level (TRL)[1 -image] [2 - 1pg pdf]. This is used enough that you'll see it in HN comments. With humans on board, you are basically aiming for TRL 8. Look at the steps there and you'll quickly see that 4 years is pretty freaking fast. This not only includes code, but hardware. Everything has to be thoroughly vetted. In a typically contract you can go from TRL 1-3 in 6mo. 3-4 in 6mo-1yr. 4-5 in 1-2yrs. And so on. My guess is that the Tesla stuff is closer to TRL 5 or 6, since there is a driver involved. You'd need TRL 8 at level 4/5 to get fully autonomous driving approved. As it should. As software people I think many will laugh at the low TRL of their own work. It isn't too bad or anything since other sectors need to move fast (probably security people will disagree). But other sectors need to move slow and ensure that things don't break. Because things breaking means people dying. [0] https://github.com/kubernetes/kubernetes/blob/ec2e767e59395376fa191d7c56a74f53936b7653/pkg/controller/volume/persistentvolume/pv_controller.go?utm_source=hackernewsletter&utm_medium=email&utm_term=fav&mc_cid=258334395b&mc_eid=da3810ffd1#L54 https://github.com/kubernetes/kubernetes/blob/ec2e767e593953... [1] https://steveblank.files.wordpress.com/2013/11/nasa-trl.jpg https://steveblank.files.wordpress.com/2013/11/nasa-trl.jpg [2] https://esto.nasa.gov/files/trl_definitions.pdf https://esto.nasa.gov/files/trl_definitions.pdf
- argonaut 8y ago
  Everything we know about how Tesla operates as a company indicates that they do not freeze development of a technology or component and then spend four years testing and verifying it before deployment.
  - danjayh 8y ago
    That's not exactly how it works. Development doesn't freeze - change requests, PRs, etc. can all be rolled in right up to the end and even after release if the appropriate process is followed, and in software, even in a safety critical world, you can pipeline updates out in a reasonably fast manner (we can get it down to about two weeks for minor updates). Hardware is an entirely different beast. You have to do FPGA development and prove-out, gate-level sims, timing accurate sims, thousands and thousands and thousands of simulated testbenches (hardware sim is orders of magnitude slower than software sim - hours per second, even on the most advanced equipment). Therefore, each testbench must be developed to test a very specific thing - in general, you can't just run your high-level software unit tests. Anyway, even if Tesla is the kind of company to roll software (doable) or boards (doable, but harder) right up to the last minute ... they do not (and neither does anybody else) spin a chip over and over again during the course of the development cycle. Speaking as the lead of a SW team that proved out a safety-critical custom SOC, it is colossally expensive. Every single iteration takes many months and millions of dollars.
    - argonaut 8y ago
      You're agreeing with me. Tesla in all likelihood does not freeze development of a chip model for 4 years before releasing it. So you cannot compare Tesla's current chip with a Google chip that is 4 years old.
      - danjayh 8y ago
        please read to the end nobody iterates chips right up to release. That's a software thing.
- Abishek_Muthian 8y ago
  I agree, areas where reliability > power; outdated hardware is of less concern. Same reason why New Horizon used MIPS R3000 CPU in 2006, where as it was also used in Playstation 1 in 1994.
sabareesh 8y ago
Similar thoughts but we dont have the spec for the hardware yet, this is merely what we think it supports
gok 8y ago
Merely being able to do math in floating point doesn't make a chip appropriate for NN training.
- ucaetano 8y ago
  Indeed, but it seems that the V2 and V3 TPUs are quite appropriate for NN training.
- make3 8y ago
  there's no training going on in this case
Joky 8y ago
But you don't need to train in a car, so you want to optimize cost/power for pure inference. Similarly if all you want to do is cheap and efficient inference, it isn't clear to me that TPU V1 are obsolete today.
make3 8y ago
you don't need to train in the car or to do massive scale inference, so you likely don't need anything close to the performance of recent TPUs
illumin8 8y ago
All Teslas need to do is run inference. I'm fairly certain training is all done either in the cloud or on datacenter GPUs. It would be very interesting if Google was selling Tesla EOL TPU v1 units. The specs are very good for what Tesla is trying to do. 4 years is not a long time considering this is a proprietary technology that even GCP customers haven't had access to for very long. It will likely be a few more years before any TPU-like technology is available in the mass market.

danjayh 8y ago

4 years is a big lag if you're not doing safety critical work. If you are doing safety critical, 4 years isn't particularly fast, but it's not unreasonable. If you've never worked in aerospace or automotive, you would be shocked at the requirements for software and hardware that can kill people if it breaks (assuming that Tesla is following the automotive equivalent of DO-178/DO-254, ISO26262)... I guess the automotive guys do have an advantage because they're not required to have independence for the various stages of verification, but it's still a big chunk of work (more than the actual development).

ucaetano 8y ago
> If you are doing safety critical Well, Tesla clearly isn't doing safety critical work on autonomous systems (it wouldn't have even launched "autopilot" if it was), so 4 years is indeed a big lag.
- torpfactory 8y ago
  Tesla claims [1] that autopilot driving is safer than regular driving in terms of risk of accident per mile. Not sure if I agree with the way they've constructed their statistics, but for a moment take them at face value. Isn't then the safer thing to release and encourage autopilot to the largest extent possible? [1] https://www.tesla.com/blog/q3-2018-vehicle-safety-report?redirect=no https://www.tesla.com/blog/q3-2018-vehicle-safety-report?red...
  - evv 8y ago
    That statistic is wildly and purposefully misleading. It compares the safety of Autopilot (which only drives on the freeway) with human drivers on any road. Accidents-per-mile are already much lower on the freeway. Plus, the statistic ignores the fact that responsibility is actually shared between humans and the car. If autopilot gets into an unsafe situation and gives control to the human at the last moment, the car company might claim that the accident happened under human control. We have a long way to go before we can truly understand the safety of semi-autonomous vehicles.
    - sandworm101 8y ago
      >> If autopilot gets into an unsafe situation and gives control to the human at the last moment We have lots of experience with the reverse, where the human drives but the autopilot rips control away when things go wrong. Traction control, radar-triggered braking, even basic ABS is a machine taking over from the human once things get hairy. Several AirForce fighters have systems that will sometimes ignore the pilot's inputs and save the aircraft from a crash. That's the real path for semi-autonomous cars imho: you do all the driving until the robot steps in to save your ass. (I cannot wait for the day someone tries to sell a car that will automatically obey speed limits regardless of driver input. That's a stock to short.)
    - Faark 8y ago
      I'd even expect their human driven statistics not actually representative or useful for comparisons. Their cars are expensive, mostly excluding young and thus often unexperienced drivers. They also don't particularly target older people, the other group with high accident rates. Controlling for all those variables is hard and not to their benefit, thus I'd never expect their pr team to do so. But I wonder if that might even creep into their automated driving stats... e.g. by drivers taking control in dangerous situations.
  - ucaetano 8y ago
    > for a moment take them at face value Why would you do so? This is like saying "Let's assume the sky is purple. Then, we can conclude that the sky is purple."
- danjayh 8y ago
  Even if it's not now, their intention is to provide higher levels of SAE automation on the same software as time progresses, which means that the current hardware probably has to hit a higher design assurance level than the current software.
  - ucaetano 8y ago
    Sure, but intentions are irrelevant. What matters is what happens.
stfwn 8y ago
Can you elaborate a bit on how such testing is done, or share a good article on the topic? It sounds like a hard problem to need to get things right this bad, or else.
- miketery 8y ago
  it's been a while, do-178 is concerned with software while do-254 is hardware (both for aviation, FAA standards). Depending on the criticality of the component (e.g. engine control vs radios vs entertainment system) they will abide by different levels (ie DO-178A ... DO-178E). For both software and hardware there will be requirements for documentation, design, verification, and testing. Some even go as far as to requir the implementation of certain functional logic in multiple ways and having concensus logic (eg hardware logic to interpret GPS accuracy/confidence That ends up being broadcast externally). There's also DO-160 that's concerned with environmental requirements (eg temperature, humidity, lighting). This is what makes a radio that should cost $300, cost $5000, and why FAA certified aviation components are expensive compared to uncertified components (eg hobby airplane builds).
  - kejaed 8y ago
    Parent comment really gives a great overview of the process. One slight nitpick is that DO-178B or DO-178C are revisions of the standard and the associated criticality is associated with the Design Assurance Level (DAL) from A to E. One would say software is developed to DO-178C DAL B for example. The DAL is determined before the software is started at the Systems level according to a process described in ARP4754A (Guidelines For Development Of Civil Aircraft and Systems) looking at the system architecture, hazards, and potential mitigations of those hazards, one of which is developing software to a certain DAL.
- godelski 8y ago
  Not that long ago there was a few HN post about running on the metal that started off with this [0] (also called "Space Shuttle Style). Also speaking of NASA, they - and many other government departments - use a system called Technology Readiness Level (TRL)[1 -image] [2 - 1pg pdf]. This is used enough that you'll see it in HN comments. With humans on board, you are basically aiming for TRL 8. Look at the steps there and you'll quickly see that 4 years is pretty freaking fast. This not only includes code, but hardware. Everything has to be thoroughly vetted. In a typically contract you can go from TRL 1-3 in 6mo. 3-4 in 6mo-1yr. 4-5 in 1-2yrs. And so on. My guess is that the Tesla stuff is closer to TRL 5 or 6, since there is a driver involved. You'd need TRL 8 at level 4/5 to get fully autonomous driving approved. As it should. As software people I think many will laugh at the low TRL of their own work. It isn't too bad or anything since other sectors need to move fast (probably security people will disagree). But other sectors need to move slow and ensure that things don't break. Because things breaking means people dying. [0] https://github.com/kubernetes/kubernetes/blob/ec2e767e59395376fa191d7c56a74f53936b7653/pkg/controller/volume/persistentvolume/pv_controller.go?utm_source=hackernewsletter&utm_medium=email&utm_term=fav&mc_cid=258334395b&mc_eid=da3810ffd1#L54 https://github.com/kubernetes/kubernetes/blob/ec2e767e593953... [1] https://steveblank.files.wordpress.com/2013/11/nasa-trl.jpg https://steveblank.files.wordpress.com/2013/11/nasa-trl.jpg [2] https://esto.nasa.gov/files/trl_definitions.pdf https://esto.nasa.gov/files/trl_definitions.pdf
argonaut 8y ago
Everything we know about how Tesla operates as a company indicates that they do not freeze development of a technology or component and then spend four years testing and verifying it before deployment.
- danjayh 8y ago
  That's not exactly how it works. Development doesn't freeze - change requests, PRs, etc. can all be rolled in right up to the end and even after release if the appropriate process is followed, and in software, even in a safety critical world, you can pipeline updates out in a reasonably fast manner (we can get it down to about two weeks for minor updates). Hardware is an entirely different beast. You have to do FPGA development and prove-out, gate-level sims, timing accurate sims, thousands and thousands and thousands of simulated testbenches (hardware sim is orders of magnitude slower than software sim - hours per second, even on the most advanced equipment). Therefore, each testbench must be developed to test a very specific thing - in general, you can't just run your high-level software unit tests. Anyway, even if Tesla is the kind of company to roll software (doable) or boards (doable, but harder) right up to the last minute ... they do not (and neither does anybody else) spin a chip over and over again during the course of the development cycle. Speaking as the lead of a SW team that proved out a safety-critical custom SOC, it is colossally expensive. Every single iteration takes many months and millions of dollars.
  - argonaut 8y ago
    You're agreeing with me. Tesla in all likelihood does not freeze development of a chip model for 4 years before releasing it. So you cannot compare Tesla's current chip with a Google chip that is 4 years old.
    - danjayh 8y ago
      please read to the end nobody iterates chips right up to release. That's a software thing.
Abishek_Muthian 8y ago
I agree, areas where reliability > power; outdated hardware is of less concern. Same reason why New Horizon used MIPS R3000 CPU in 2006, where as it was also used in Playstation 1 in 1994.

sabareesh 8y ago

Similar thoughts but we dont have the spec for the hardware yet, this is merely what we think it supports

gok 8y ago

Merely being able to do math in floating point doesn't make a chip appropriate for NN training.

ucaetano 8y ago
Indeed, but it seems that the V2 and V3 TPUs are quite appropriate for NN training.
make3 8y ago
there's no training going on in this case

Joky 8y ago

But you don't need to train in a car, so you want to optimize cost/power for pure inference. Similarly if all you want to do is cheap and efficient inference, it isn't clear to me that TPU V1 are obsolete today.

make3 8y ago

you don't need to train in the car or to do massive scale inference, so you likely don't need anything close to the performance of recent TPUs

illumin8 8y ago

All Teslas need to do is run inference. I'm fairly certain training is all done either in the cloud or on datacenter GPUs. It would be very interesting if Google was selling Tesla EOL TPU v1 units. The specs are very good for what Tesla is trying to do. 4 years is not a long time considering this is a proprietary technology that even GCP customers haven't had access to for very long. It will likely be a few more years before any TPU-like technology is available in the mass market.