4 ms·

This LIDAR wank annoys me. If you can train a policy that drives well on cameras, you can get self-driving. If you can't, you're fucked, and no amount of extra

This LIDAR wank annoys me. If you can train a policy that drives well on cameras, you can get self-driving. If you can't, you're fucked, and no amount of extra sensors will save you. Self-driving isn't a sensor problem. It always was, is, and always will be an AI problem. No amount of LIDAR engineering will ever get you a LIDAR that outputs ground truth steering commands. The best you'll ever get is noisy depth estimate speckles that you'll have to massage with, guess what, AI, to get them to do anything of use. Sensor suite choice is an aside. Camera only 360 coverage? Good enough to move on. The rest of the problem lies with AI.

lateforwork 9mo ago
Even the best AI can't drive without good sensors. Cameras have to guess distance and they fail when there is insufficient contrast, direct sunlight and so on. LiDARs don't have to guess distance.
- slfnflctd 9mo ago
  Cameras also fail when weather conditions cake your car in snow and/or mud while you're driving. Actually, from what I just looked up, this is an issue with LiDAR as well. So it seems to me like we don't even have the sensors we need to do this properly yet, unless we can somehow make them all self-cleaning. It always goes back to my long standing belief that we need dedicated lanes with roadside RFID tags to really make this self driving thing work well enough.
  - ACCount37 9mo ago
    Nah. That's a common "thought about it for 15 seconds but not 15 minutes" mistake. Making a car that drives well on arbitrary roads is freakishly hard. Having to adapt every single road in the world before even a single self-driving car can use them? That's a task that makes the previous one look easy. Learned sensor fusion policy that can compensate for partial sensor degradation, detect severe dropout, and handle both safely? Very hard. Getting the world that can't fix the low tech potholes on every other road to set up and maintain machine specific infrastructure everywhere? A nonstarter.
    - slfnflctd 9mo ago
      Well, we already provide dedicated lanes for multi-passenger vehicles in many places, nearly all semi-major airports have dedicated lots and lanes for rideshare drivers, many parts of downtown/urban areas have the same things... and it didn't exactly take super long to roll all that out. Also, 99% of roads in civilized areas have something alongside them already that you can attach RFID tags to. Quite a bit easier than setting up an EV charging station (another significant infrastructure thing which has rolled out pretty quickly). And let's not forget, every major metro area in the world has multi-lane superhighways which didn't even exist at all 50-70 years ago. Believe me, I've thought about this for a lot more than 15 minutes. Yes, we should improve sensor reliability, absolutely. But it wouldn't hurt to have some kind of backup roadside positioning help, and I don't see how it would be prohibitively expensive. Maybe I am missing something, but I'm gonna need more than your dismissive comment to be convinced of that.
      - ACCount37 9mo ago
        You are missing the sheer soul-crushing magnitude of the infrastructure problem. You are missing the little inconvenient truth that live in a world full of roads that don't even consistently have asphalt on them. That real life Teslas ship with AI that does vibe-based road lane estimation because real life roads occasionally fail to have any road markings a car AI could see. Everything about road infrastructure is "cheap to deploy, cheap to maintain". This is your design space: the bare minimum of a "road" that still does its job reasonably well. Gas stations and motels are an aside - they earn money. Not even the road signs pay for themselves. Now, you propose we design some type of, let's say, a machine only mark that helps self-driving cars work well. They do nothing for human drivers, who are still a road majority. And then you somehow manage to make every country and every single self-driving car vendor to get to agree on the spec, both on paper and in truth. Alright, let's say we've done that. Why would anyone, then, put those on the road? They're not the bare minimum. And if we wanted to go beyond the bare minimum, we'd plug the potholes, paint the markings and fix the road signs first.
        slfnflctd 9mo ago
top_sigrid 9mo ago
> If you can train a policy that drives well on cameras, you can get self-driving. If you can't, you're fucked, and no amount of extra sensors will save you. Source: trust me, bro? This statement has no factual basis. Calling the most common approach of all other self-driving developers except Tesla a wank also is no argument but hate only.
- ACCount37 9mo ago
  [flagged]
  - top_sigrid 9mo ago
    This is so dumb, I don't even know if you are serious. Nobody ever said it is lidar instead of cameras, but as additional sensor to cameras. And everybody seems to agree that that is valuable sensor-information (except Tesla).
    - sejje 9mo ago
      I'm able to drive without lidar, with just my eyeball feeds. I agree that lidar is very valuable right now, but I think in the endgame, yeah it can drive with just cameras. The logic follows, because I drive with just "cameras."
      - senordevnyc 9mo ago
        Yeah, but your "cameras" also have a bunch of capabilities that hardware cameras don't, plus they're mounted on a flexible stalk in the cockpit that can move in any direction to update the view in real-time. Also, humans kinda suck at driving. I suspect that in the endgame, even if AI can drive with cameras only, we won't want it to. If we could upgrade our eyeballs and brains to have real-time 3D depth mapping information as well as the visual streams, we would.
        ACCount37 9mo ago
ActorNightly 9mo ago
You are correct, but the problem is nobody at Tesla or any other self driving company for that matter knows what they are doing when it comes to AI If you are doing end to end driving policy (i.e the wrong way of doing it), having lidar is important as a correction factor to the cameras.
- ACCount37 9mo ago
  So far, end to end seems to be the only way to train complex AI systems that actually works. Every time you pit the sheer violent force of end to end backpropagation against compartmentalization and lines drawn by humans, at a sufficient scale, backpropagation gets its win.
  - ActorNightly 9mo ago
    >Every time you pit the sheer violent force of end to end backpropagation against compartmentalization and lines drawn by humans, at a sufficient scale, backpropagation gets its win. I fully agree, but your statement is quite ironic. For driving, humans drive well because we operate more like Mu Zero model does - we can "visualize" the possibilities of the future states depending on what we do and pick the most optimal path. We don't need to know what the specific object is on the road, the fact that we can recognize that its there, in our path, and understand the physical interacting of a car hitting something that is taller than a bump means we can avoid it. The way to implement self driving is exactly that - train your model to take sensor data and reconstruct a 3d space in latent dimensions, train another model to predict evolutions on the 3d space given time history with probabilistic output, and then your inference is a probabilistic guided search in that space with time constraints based on hardware. Mu Zero is nothing new, and already proved that you don't even need a hardcoded model of environment to operate in. And you don't even need human driving data for this, as the model will be able to predict things collisions solely based on pure physics. And as a bonus, as you enhance it with things like physical models of the cars, where it can reconcile what it thinks the system is going to do versus what the physics calculations predict, you can even make it drive well in snow with low traction. The irony of your statement is that everyone who is going end to end is manually hand coding all these hacks (like image warping in the case of Comma AI) to make the training work, all because the training data is just not sufficient, which is the exact same exercise as humans drawing lines. And if you doubt that what Im saying is true, again, Mu Zero was proven to work. Driving is just another game where you can easily define a winning scenario, the board, and moves you can make, and apply the same concepts. The only technical part becomes accurately determining the board from sensor data.
mrexcess 9mo ago
>Self-driving isn't a sensor problem. It always was, is, and always will be an AI problem. AI + cameras have relevant limitations that LIDAR augmented suites don't. You can paint a photorealistic roadway onto a brick wall and AI + cameras will try to drive right through it, dubbed the "Wile E. Coyote" problem.
- sejje 9mo ago
  Will humans?
  - 1718627440 9mo ago
    No. Even if you use a real photo, instead of just a photorealistic painting.
jasondigitized 9mo ago
Just don't drive up north in the snow and your good.

lateforwork 9mo ago

Even the best AI can't drive without good sensors. Cameras have to guess distance and they fail when there is insufficient contrast, direct sunlight and so on. LiDARs don't have to guess distance.

slfnflctd 9mo ago
Cameras also fail when weather conditions cake your car in snow and/or mud while you're driving. Actually, from what I just looked up, this is an issue with LiDAR as well. So it seems to me like we don't even have the sensors we need to do this properly yet, unless we can somehow make them all self-cleaning. It always goes back to my long standing belief that we need dedicated lanes with roadside RFID tags to really make this self driving thing work well enough.
- ACCount37 9mo ago
  Nah. That's a common "thought about it for 15 seconds but not 15 minutes" mistake. Making a car that drives well on arbitrary roads is freakishly hard. Having to adapt every single road in the world before even a single self-driving car can use them? That's a task that makes the previous one look easy. Learned sensor fusion policy that can compensate for partial sensor degradation, detect severe dropout, and handle both safely? Very hard. Getting the world that can't fix the low tech potholes on every other road to set up and maintain machine specific infrastructure everywhere? A nonstarter.
  - slfnflctd 9mo ago
    Well, we already provide dedicated lanes for multi-passenger vehicles in many places, nearly all semi-major airports have dedicated lots and lanes for rideshare drivers, many parts of downtown/urban areas have the same things... and it didn't exactly take super long to roll all that out. Also, 99% of roads in civilized areas have something alongside them already that you can attach RFID tags to. Quite a bit easier than setting up an EV charging station (another significant infrastructure thing which has rolled out pretty quickly). And let's not forget, every major metro area in the world has multi-lane superhighways which didn't even exist at all 50-70 years ago. Believe me, I've thought about this for a lot more than 15 minutes. Yes, we should improve sensor reliability, absolutely. But it wouldn't hurt to have some kind of backup roadside positioning help, and I don't see how it would be prohibitively expensive. Maybe I am missing something, but I'm gonna need more than your dismissive comment to be convinced of that.
    - ACCount37 9mo ago
      You are missing the sheer soul-crushing magnitude of the infrastructure problem. You are missing the little inconvenient truth that live in a world full of roads that don't even consistently have asphalt on them. That real life Teslas ship with AI that does vibe-based road lane estimation because real life roads occasionally fail to have any road markings a car AI could see. Everything about road infrastructure is "cheap to deploy, cheap to maintain". This is your design space: the bare minimum of a "road" that still does its job reasonably well. Gas stations and motels are an aside - they earn money. Not even the road signs pay for themselves. Now, you propose we design some type of, let's say, a machine only mark that helps self-driving cars work well. They do nothing for human drivers, who are still a road majority. And then you somehow manage to make every country and every single self-driving car vendor to get to agree on the spec, both on paper and in truth. Alright, let's say we've done that. Why would anyone, then, put those on the road? They're not the bare minimum. And if we wanted to go beyond the bare minimum, we'd plug the potholes, paint the markings and fix the road signs first.
      - slfnflctd 9mo ago

top_sigrid 9mo ago

> If you can train a policy that drives well on cameras, you can get self-driving. If you can't, you're fucked, and no amount of extra sensors will save you. Source: trust me, bro? This statement has no factual basis. Calling the most common approach of all other self-driving developers except Tesla a wank also is no argument but hate only.

ACCount37 9mo ago
[flagged]
- top_sigrid 9mo ago
  This is so dumb, I don't even know if you are serious. Nobody ever said it is lidar instead of cameras, but as additional sensor to cameras. And everybody seems to agree that that is valuable sensor-information (except Tesla).
  - sejje 9mo ago
    I'm able to drive without lidar, with just my eyeball feeds. I agree that lidar is very valuable right now, but I think in the endgame, yeah it can drive with just cameras. The logic follows, because I drive with just "cameras."
    - senordevnyc 9mo ago
      Yeah, but your "cameras" also have a bunch of capabilities that hardware cameras don't, plus they're mounted on a flexible stalk in the cockpit that can move in any direction to update the view in real-time. Also, humans kinda suck at driving. I suspect that in the endgame, even if AI can drive with cameras only, we won't want it to. If we could upgrade our eyeballs and brains to have real-time 3D depth mapping information as well as the visual streams, we would.
      - ACCount37 9mo ago

ActorNightly 9mo ago

You are correct, but the problem is nobody at Tesla or any other self driving company for that matter knows what they are doing when it comes to AI If you are doing end to end driving policy (i.e the wrong way of doing it), having lidar is important as a correction factor to the cameras.

ACCount37 9mo ago
So far, end to end seems to be the only way to train complex AI systems that actually works. Every time you pit the sheer violent force of end to end backpropagation against compartmentalization and lines drawn by humans, at a sufficient scale, backpropagation gets its win.
- ActorNightly 9mo ago
  >Every time you pit the sheer violent force of end to end backpropagation against compartmentalization and lines drawn by humans, at a sufficient scale, backpropagation gets its win. I fully agree, but your statement is quite ironic. For driving, humans drive well because we operate more like Mu Zero model does - we can "visualize" the possibilities of the future states depending on what we do and pick the most optimal path. We don't need to know what the specific object is on the road, the fact that we can recognize that its there, in our path, and understand the physical interacting of a car hitting something that is taller than a bump means we can avoid it. The way to implement self driving is exactly that - train your model to take sensor data and reconstruct a 3d space in latent dimensions, train another model to predict evolutions on the 3d space given time history with probabilistic output, and then your inference is a probabilistic guided search in that space with time constraints based on hardware. Mu Zero is nothing new, and already proved that you don't even need a hardcoded model of environment to operate in. And you don't even need human driving data for this, as the model will be able to predict things collisions solely based on pure physics. And as a bonus, as you enhance it with things like physical models of the cars, where it can reconcile what it thinks the system is going to do versus what the physics calculations predict, you can even make it drive well in snow with low traction. The irony of your statement is that everyone who is going end to end is manually hand coding all these hacks (like image warping in the case of Comma AI) to make the training work, all because the training data is just not sufficient, which is the exact same exercise as humans drawing lines. And if you doubt that what Im saying is true, again, Mu Zero was proven to work. Driving is just another game where you can easily define a winning scenario, the board, and moves you can make, and apply the same concepts. The only technical part becomes accurately determining the board from sensor data.

mrexcess 9mo ago

>Self-driving isn't a sensor problem. It always was, is, and always will be an AI problem. AI + cameras have relevant limitations that LIDAR augmented suites don't. You can paint a photorealistic roadway onto a brick wall and AI + cameras will try to drive right through it, dubbed the "Wile E. Coyote" problem.

sejje 9mo ago
Will humans?
- 1718627440 9mo ago
  No. Even if you use a real photo, instead of just a photorealistic painting.

jasondigitized 9mo ago

Just don't drive up north in the snow and your good.