7 ms·

> Reliance on HD Mapping gets you to "robotaxis" quicker and easier, but it doesn't and likely cannot scale. If you can make the unit economics work for a larg

> Reliance on HD Mapping gets you to "robotaxis" quicker and easier, but it doesn't and likely cannot scale. If you can make the unit economics work for a large quantity of individual cars, mapping is a small fixed cost. I agree that it's not economical to map every city and road in the US, since you need to generate revenue from every mapped road and city. So you can think of HD maps as amounting to building roads. They will be built in lucrative places. Cruise and Waymo won't make money from putting taxis in nowhere Arkansas, so they don't need to map it. > the current limiting factor is not what the car sees or knows but what it does with that information. It is unclear how or why HD mapping would help them at that point. That's simply untrue. All the hard stuff continues to be reliability and sensor gated. Cruise and Waymo have amazing sensors and even they struggle with sensor range, sensor reliability, model performance on tail cases, etc. For example, at night these cars typically do not have IR or Thermal sensing. They are relying on the limited dynamic range of their cameras + active illumination + hoping laser gets enough points / your object is reflective enough. Laser perception also hits limits when lasers shine on small objects (think: skinny railroad arm). Cars also have limits with regard to interpreting written signs, which is a big part of driving. Occlusions are still public enemy #1. Waymo killed a dog. Cruise crashed into a fire truck coming out of a blind intersection even though their sensors saw the truck within 100ms. LiDAR and HD mapping together are supremely useful, even if you don't drive with it, for enabling you to simulate accurately. You cannot simulate reliably while guessing at distances and locations. HD maps let you use visual odometry to localize, and distance measurements grounded in physics backstop the realism of your simulation at least in terms of the world's shape. Tesla lacks the ability to resim counterfactuals with confidence since they don't have HD ground truth. There are believers at the company that maybe you could make "good enough" ground truth from imagery alone but that in and of itself is a huge risk, and it's what skipping steps looks like. Most in the industry agree that barring a major change in strategy they just have no way to regression test their software to the level of reliability required for L4 / no human supervision.

Workaccount2 2y ago
Google already has a fleet of vehicles driving around continuously taking new street view photos.
- bangaladore 2y ago
  Google Maps seems to update at a frequency of 2-8 years. Maybe longer in some areas and we don't know what they do other than stitch it together. HD mapping, on the other hand, likely needs to be updated frequently and immediately when any construction occurs. It seems pretty clear that what they are doing today is nowhere near what they need to do. And again, I don't think that is possible.
  - TulliusCicero 2y ago
    The obvious thing to do is to just have every Waymo robotaxi or car with licensed Waymo tech report in its daily mapping/obstacle data to the mothership, so you can get new changes almost immediately. I dunno if said data would be as high quality as dedicated HD mapping cars, but it's probably at least decent, given the variety of cameras and lidars every Waymo car has.
    - singleshot_ 2y ago
      Further, it seems to me that if you brake hard to avoid a dog, your car should warn me as I’m approaching. I’m not sure why we are trying to teach each car to drive when we could be teaching all the cars and the road to drive.
      - c22 2y ago
        This is a good point. Do Waymo cars ever use their horn?
        nolist_policy 2y ago
        They do: https://youtu.be/YxVihk4c-h8?t=3m5s https://youtu.be/YxVihk4c-h8?t=3m5s
      - TulliusCicero 2y ago
        > Further, it seems to me that if you brake hard to avoid a dog, your car should warn me as I’m approaching. What does this mean? Electric cars are already required to emit a sound as they drive. I guess if it has to brake hard for something, honking might be a good idea, but I wouldn't want cars to constantly be beeping at everything in their vicinity if there's no imminent crash. > I’m not sure why we are trying to teach each car to drive when we could be teaching all the cars and the road to drive. I'm not sure what you mean. Presumably Waymo's software is the same across its fleet. They're not training one car's model at a time.
bangaladore 2y ago
> That's simply untrue. All the hard stuff continues to be reliability and sensor gated. IR and thermal sensing are unnecessary if the bar is human level and neither is the lidar. The point is overused, but humans rely on two eyes in the driver seat. I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5. Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve.
- kajecounterhack 2y ago
  > I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5 For one, frame rate and processing rate on human eyes is way higher than cameras. Dynamic range is another. Also, Cruise and Waymo are some of the only companies that have hard internal data / ability to simulate how well their safety drivers do, and in the very same scenario what their software driver will do. Without LiDAR you can't build that simulation, and once you have that data if you continue to use HD Maps and LiDAR there's probably a good reason. > Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve. Robotaxis need to avoid any accident that a human would be able to avoid. > IR and thermal sensing are unnecessary if the bar is human level See, you could say this if you had some data that showed that incidents per X miles (when the vehicle is driving at night) is sufficiently low, + if the software passes some contrived scenarios to gut-check its ability to see in the dark with the necessary reliability. But you don't have that data, do you? Someone has it though :) and I'd argue regulators should have it too.
  - bangaladore 2y ago
    > For one, frame rate and processing rate on human eyes is way higher than cameras. I don't think it's exciting to say that you must have theoretical parity with something to use it for this use case. Tesla's solution monitors ~6? cameras at once with accurate depth in each. That's 6x more views than a human can see. I wish people would stop comparing apples to oranges. > Robotaxis need to avoid any accident that a human would be able to avoid. I never said anything to the contrary. Animals get hit all the time, not just because a human wasn't paying attention.
FireBeyond 2y ago
> For example, at night these cars typically do not have IR or Thermal sensing. This one in itself continues to kinda surprise me. High end Audis have full-blown thermal cameras, and will literally flag heat producing objects on the console: https://www.audi-technology-portal.de/en/electrics-electronics/driver-assistant-systems/night-vision-assistant https://www.audi-technology-portal.de/en/electrics-electroni...
- kajecounterhack 2y ago
  It's likely because thermal is expensive for the BOM and also wrt energy usage. Also low res.
Dunedan 2y ago
> Tesla lacks the ability to resim counterfactuals with confidence since they don't have HD ground truth. Tesla does have HD ground truth data for verification generated by their own LIDAR-equipped vehicles. However, according to a recent tweet by Elon Musk [1], they don't need LIDAR for that anymore. [1]: https://nitter.poast.org/elonmusk/status/1788064109633875999 https://nitter.poast.org/elonmusk/status/1788064109633875999

Workaccount2 2y ago

Google already has a fleet of vehicles driving around continuously taking new street view photos.

bangaladore 2y ago
Google Maps seems to update at a frequency of 2-8 years. Maybe longer in some areas and we don't know what they do other than stitch it together. HD mapping, on the other hand, likely needs to be updated frequently and immediately when any construction occurs. It seems pretty clear that what they are doing today is nowhere near what they need to do. And again, I don't think that is possible.
- TulliusCicero 2y ago
  The obvious thing to do is to just have every Waymo robotaxi or car with licensed Waymo tech report in its daily mapping/obstacle data to the mothership, so you can get new changes almost immediately. I dunno if said data would be as high quality as dedicated HD mapping cars, but it's probably at least decent, given the variety of cameras and lidars every Waymo car has.
  - singleshot_ 2y ago
    Further, it seems to me that if you brake hard to avoid a dog, your car should warn me as I’m approaching. I’m not sure why we are trying to teach each car to drive when we could be teaching all the cars and the road to drive.
    - c22 2y ago
      This is a good point. Do Waymo cars ever use their horn?
      - nolist_policy 2y ago
        They do: https://youtu.be/YxVihk4c-h8?t=3m5s https://youtu.be/YxVihk4c-h8?t=3m5s
    - TulliusCicero 2y ago
      > Further, it seems to me that if you brake hard to avoid a dog, your car should warn me as I’m approaching. What does this mean? Electric cars are already required to emit a sound as they drive. I guess if it has to brake hard for something, honking might be a good idea, but I wouldn't want cars to constantly be beeping at everything in their vicinity if there's no imminent crash. > I’m not sure why we are trying to teach each car to drive when we could be teaching all the cars and the road to drive. I'm not sure what you mean. Presumably Waymo's software is the same across its fleet. They're not training one car's model at a time.

bangaladore 2y ago

> That's simply untrue. All the hard stuff continues to be reliability and sensor gated. IR and thermal sensing are unnecessary if the bar is human level and neither is the lidar. The point is overused, but humans rely on two eyes in the driver seat. I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5. Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve.

kajecounterhack 2y ago
> I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5 For one, frame rate and processing rate on human eyes is way higher than cameras. Dynamic range is another. Also, Cruise and Waymo are some of the only companies that have hard internal data / ability to simulate how well their safety drivers do, and in the very same scenario what their software driver will do. Without LiDAR you can't build that simulation, and once you have that data if you continue to use HD Maps and LiDAR there's probably a good reason. > Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve. Robotaxis need to avoid any accident that a human would be able to avoid. > IR and thermal sensing are unnecessary if the bar is human level See, you could say this if you had some data that showed that incidents per X miles (when the vehicle is driving at night) is sufficiently low, + if the software passes some contrived scenarios to gut-check its ability to see in the dark with the necessary reliability. But you don't have that data, do you? Someone has it though :) and I'd argue regulators should have it too.
- bangaladore 2y ago
  > For one, frame rate and processing rate on human eyes is way higher than cameras. I don't think it's exciting to say that you must have theoretical parity with something to use it for this use case. Tesla's solution monitors ~6? cameras at once with accurate depth in each. That's 6x more views than a human can see. I wish people would stop comparing apples to oranges. > Robotaxis need to avoid any accident that a human would be able to avoid. I never said anything to the contrary. Animals get hit all the time, not just because a human wasn't paying attention.

FireBeyond 2y ago

> For example, at night these cars typically do not have IR or Thermal sensing. This one in itself continues to kinda surprise me. High end Audis have full-blown thermal cameras, and will literally flag heat producing objects on the console: https://www.audi-technology-portal.de/en/electrics-electronics/driver-assistant-systems/night-vision-assistant https://www.audi-technology-portal.de/en/electrics-electroni...

kajecounterhack 2y ago
It's likely because thermal is expensive for the BOM and also wrt energy usage. Also low res.

Dunedan 2y ago

> Tesla lacks the ability to resim counterfactuals with confidence since they don't have HD ground truth. Tesla does have HD ground truth data for verification generated by their own LIDAR-equipped vehicles. However, according to a recent tweet by Elon Musk [1], they don't need LIDAR for that anymore. [1]: https://nitter.poast.org/elonmusk/status/1788064109633875999 https://nitter.poast.org/elonmusk/status/1788064109633875999