8 ms·

"Camera crushes lidar" is marketing spin, because different types of sensors "win" for different scenarios. You really want a sensor fusion strategy for device

• mensetmanusman 3y ago
  You really want what works best given the ms time budget in the context of your on-board compute.
  • mynameisvlad 3y ago
    If it’s a life or death decision, you should opt for the best strategy, not the cheapest.
    • IntToDouble 3y ago
      Cheapest being humans.
      • geysersam 3y ago
        Not necessarily. Maybe currently, but perhaps not in the future.
    • spdif899 3y ago
      Person you're replying to is talking about time budget offered by your computer, not monetary budget. Regardless everything is about optimization - the "best" option will almost always be so costly that no consumer will be able to afford it.
      • NovemberWhiskey 3y ago
        If the technology (i.e. self-driving) cannot be delivered safely at an affordable cost, then probably it shouldn't be delivered at all then, no?
        • mensetmanusman 3y ago
          Define safely given the current road hazards of human drivers.
        • burnished 3y ago
          This isn't germane to the conversation at all, the budget under discussion is compute time.
          • mrguyorama 3y ago
            If we can't fit the compute required to do it right in a car, we shouldn't be doing it at all. Their point is perfectly fine, regardless of WHAT KIND of "budget" is being discussed. That being said, even automobiles make safety tradeoffs for cheapness or feasibility. However, we really shouldn't allow any tradeoffs for a completely unnecessary feature like "self driving". Imagine if wanting your car to have android auto or similar meant it couldn't use the lights, because a tradeoff was made.
        • jjk166 3y ago
          Safely only means you're above a threshold value that is safe enough, but you can always be safer with sufficient further expense. Presumably your budget allows you to go beyond the absolute bare minimum, but it obviously won't get you to infinity either, so you optimize for the best option within your constraints.
      • mynameisvlad 3y ago
        “Time budget” is ultimately the same thing as the monetary budget. Spend more, get more ms time. In a life or death situation, you should opt for the system which will keep you alive more, not the one that costs less.
    • vigilans 3y ago
      The best strategy is to hide under your mattress.
      • Vecr 3y ago
        No it's not, have you heard of bed sores and muscle atrophy? You need to exercise a reasonable amount to minimize cardiovascular risk as well.
    • hedora 3y ago
      The best strategy is the one that reduces global annual traffic injuries the most. That’s probably also the cheapest viable strategy. If a sensor-fusion car cuts accidents per mile (vs human) by 100x, but can only be deployed on 100,000 cars a year, and a camera-only car kills 10x more than that per mile, but can be put on 10,000,000 cars a year for the same cost, the camera-only car will end up saving 10x more people than the “better” system. (I exaggerated both the improvement ratio and cost ratio because I like multiplying by powers of ten)
      • xnx 3y ago
        If we pick different made up numbers, the answer is different.
        • mensetmanusman 3y ago
          It’s still a useful thought exercise in the context of ‘the perfect’ being the enemy of improving the death rate.
          • mrguyorama 3y ago
            "Thought exercises" do not belong in a safety discussion. This isn't an 8th grade debate, it's a company putting people at risk to increase their valuation while claiming "it's for the greater good".
            • hedora 3y ago
              I explained how to reason about the trade off using 3rd grade math. If the outcome of your safety discussion ends up suggesting a “safer”, “more expensive” solution that will definitely leave more people dead and injured then that analysis, then something is seriously wrong.
            • mardifoufs 3y ago
              What? Thought exercises always belong in safety discussions. Where is this weird notion that "safety Uber Alles" is actually how safety works or even should work? Going for absurd safety standards or expectations is absurd and self defeating. Again, as the other anon said, a practical solution that helps improve safety without handwaving material realities (cost, feasibility, adoption rates) is always better than a "safer" option that won't actually be used. Obviously corporations try to make more money, but people also dont like buying more expensive cars.
• lgrebe 3y ago
  How do you manage disagreements in fusion?
  • bipson 3y ago
    That's the essence of fusion.
  • batmansmk 3y ago
    Kalman filters.
    • reedf1 3y ago
      Atleast someone knows what they are talking about.
      • mrguyorama 3y ago
        Kalman filters are like sensor fusion 101, and anyone who has attached more than one distance sensor to an arduino has attempted it. It's not that unreasonable that the average person has no idea what sensor fusion is, what IS unreasonable is the damn head of self driving at Tesla claiming that "what do you do when the sensors disagree" is even a valid question.
        • archgoon 3y ago
          [dead]
  • unregistereddev 3y ago
    The old school way is to add logic based on the strengths and weaknesses of each sensor type. My example is not specific to automotive sensors (I haven't worked in the automotive sector, but I do have now-outdated experience in obstacle detection and ranging, along with avoidance algorithms). Sonar sensors are most accurate at medium ranges, but they are notorious for detecting ghost objects that do not really exist. Infrared range sensors are more reliable but are only accurate at very short range. So when a sonar sensor detects an object 8.4 meters away, you use the infrared sensor to double check. If the infrared sensor says there's an object 9 meters away in the same direction, you assume the object is real but is actually 8.4 meters away. If the infrared sensor says the nearest object in that direction is 20 meters away, you assume the sonar sensor made something up. If you have enough types of sensors, you can also use a "majority rule". If two of 3 sensor types agree, you assume the 3rd is an anomaly. Lidar is excellent for this because it is accurate across a very large range, so it tends to overlap with most of your other sensors. This increases that odds that when there is a disagreement, one of the agreeing sensors will be capable of accurately measuring the distance to the object.
• ajross 3y ago
  That's mostly spin too. You want a strategy that works well enough, because that's the regime we live in currently. Other human drivers certainly aren't implementing a LIDAR-based sensor fusion strategy and you have to share the roads with them. Your algorithm for evaluating their safety is "how likely are they to kill you" and not an absolutist position on the specific equipment in their heads or cars, and you're clearly OK with that. As far as LIDAR itself: sure, yeah, you get depth info out of it. But depth info is only part of the problem, and frankly it's clear at this point that it's one of the easiest. The hard parts are in the recognition side: not "is that pedestrian in your path" (easy), but "is that pedestrian going to step into the street or not" (hard). And that's a computer vision problem. You can use LIDAR output as vision input, sure, but it's has no advantages. Tesla was right, basically.
  • BugsJustFindMe 3y ago
    > But depth info is only part of the problem, and frankly it's clear at this point that it's one of the easiest. That's proven false by the cars continuing to drive into stationary objects. This failure mode is not ambiguous. Camera input is garbage for interpreting geometry, especially from very smooth or very discontinuous surfaces, and especially with the shitty low resolution and low dynamic range cameras they use, and especially with non-stereoscopic cameras with no motion freedom relative to the vehicle body. Lidar is a necessary crutch for working around the fact that, while hypothetical cameras that don't exist might work well, all available cameras are unsuitable for the purpose, and calculating multi-view geometry accurately costs time. > The hard parts are in the recognition side: not "is that pedestrian in your path" (easy), but "is that pedestrian going to step into the street or not" (hard). And that's a computer vision problem. Pedestrian motion is not strictly a vision challenge but a general category of environment understanding (mass, momentum, motion mechanics). Vision is only one possible input mode preliminary to modeling.
    • ajross 3y ago
      > That's proven false by the cars continuing to drive into stationary objects. It has? This again gets to "are they safer than human drivers?", because the competition hits stationary objects all the time. If you have data let's discuss data, but "proven false" is, again, just spin. > Pedestrian motion is not strictly a vision challenge but a general category of environment understanding Semantic evasion. You agree that it's "not a problem solved by LIDAR", right? It needs a camera. You can use a LIDAR output as a (somewhat inferior) camera, but it's not providing any advantages.
• ggreer 3y ago
  The problem with sensor fusion is what do you do when the sensors disagree? You have to decide which sensor to trust. But LiDAR is subject to interference, and it can’t see lane markings, traffic lights, speed limit signs, or emergency vehicle lights. If you ignored camera data and just relied on LiDAR, you wouldn’t be able to drive safely for very long. If you choose to ignore the LiDAR, then the sensor fusion is just adding noise to your camera vision model of the world. If you somehow do sensor fusion perfectly and the models never disagree, then why even have the LiDAR? At that point you’ve solved vision using cameras.
  • nomel 3y ago
    LiDAR easily passes the “featureless white wall” test. That’s why I’ll be waiting.
  • yomlica8 3y ago
    Don't you just make a conservative decision in that case? My eyes and ears disagree all the time, but if one of them says danger that is the one I listen too.
    • parineum 3y ago
      I'm sure this example isn't new at all but it's the first time I've encountered the argument phrased this way and I find it incredibly compelling. Looks good, smells bad is another one (that probably happens more often). It did get me thinking about different senses and how I prioritize them. Given that humans strongest sense is visual, it's interesting to me that the priorities of what to trust seem opposite of my expectations. It seems to me that visual signals are seemingly least "trusted" compared to the other senses. As if the logic goes, "my sense of smell is so bad that if it detects danger, it must be very dangerous". Similarly, I'm sure dogs can smell rotting flesh long before meat is unsafe.
      • afterburner 3y ago
        Even simpler: if one sensor tells you confidently there's something in your way, and the other doesn't, assume there's something in your way.