5 ms·
Because testing this stuff in the real world is the only way. The human in the loop did exactly what they were supposed to do.
by Karunamon 3y ago
Because testing this stuff in the real world is the only way. The human in the loop did exactly what they were supposed to do.
- manuelabeledo 3y agoThis is the equivalent to "let's test it in production, what could go wrong?". Also, why wouldn't they be able to test it in a simulator?
- deleted 3y ago[deleted]
- lolinder 3y agoYou can't test it in a simulator because half of what you're testing is the interaction with the sensors. You can't simulate sensors with high enough fidelity to be meaningful. And this is more like a staging environment than it is production—production is when you put it in the hands of customers, this is an internal release available only to employees. I'm firmly opposed to FSD beta, but this video doesn't concern me at all.
- tantalor 3y ago> can't simulate sensors with high enough fidelity Why not? What's the limiting factor? Video bandwidth? We can run video games at 4k 60fps. Is that not enough?
- deleted 3y ago[deleted]
- andsoitis 3y ago> What's the limiting factor? Even if you could perfectly simulate conditions for the sensors to pick up (incl. precipitation, light, dust) you cannot possibly simulate all possible scenarios that can happen in the real world.
- tantalor 3y agoMost software systems are not tested on "all possible scenarios". Instead you run some basic must-pass scenarios, like the one in the video, where there is a red light in front of the car, and a left turn green, in clear conditions and daylight. If the model doesn't pass, then don't ship it!
- manuelabeledo 3y ago> You can't test it in a simulator because half of what you're testing is the interaction with the sensors. Yeah, that's what integration testing is for. I would be surprised if they couldn't test the brains of FSD independently in a simulator. That's just QA 101.
- ra7 3y ago> You can't simulate sensors with high enough fidelity to be meaningful. You can if you invest in good simulators. See: https://waymo.com/blog/2021/06/SimulationCity.html https://waymo.com/blog/2021/06/SimulationCity.html > One way we ensure statistical realism in our simulated world is by creating realistic conditions for our autonomous driving technology to experience. So, if the Waymo Driver is going through spring showers in Detroit at sunset in SimulationCity, we can recreate raindrops on our sensors and even simulate other minute details such as the dimming light and solar glare. If they can simulate rain drops, it's definitely possible to simulate traffic lights. They are a basic scenario for a self driving car. What this video highlights is the extreme lack of validation culture within Tesla FSD.
- lolinder 3y agoMy point isn't that you can't simulate it at all, my point is that at some point you have to take it out of the simulation and it will behave differently in the real world than it did in the simulated one, because your simulation will never be good enough to account for everything. We can't expect people to never test self-driving cars on public, populated roads unless we are just acknowledging that we don't believe self-driving cars should be a thing, because I sure as hell am not comfortable with the idea of cars going straight from simulation to production.
- bluefishinit 3y ago> I sure as hell am not comfortable with the idea of cars going straight from simulation to production. This is what you're advocating for by saying people should test these things on public roads. That is production, none of the other drivers have consented to be part of the "test".
- ra7 3y agoFSD makes basic errors, the kind that can easily be identified in simulation rather than requiring 450k cars on the road. They don't have to go straight from simulation to production either. That's why all the serious players have closed course testing and safety driver supervised testing.
- 3y ago
- rob74 3y agoHas Tesla built an entire "staging town" where no unsuspecting public is allowed to enter in which they test their cars? No? Then I would definitely call it "testing in production"...
- tantalor 3y agoIt's not open to the public, but Aurora has a testing track in Pittsburgh: https://goo.gl/maps/pqvXWptAEWjC918z5 https://goo.gl/maps/pqvXWptAEWjC918z5
- biosboiii 3y agoThey already do, it suceeds in most cases, the edge cases are what messes things up. See https://carla.org/ https://carla.org/
- wahnfrieden 3y agoIs it the only way, or the profitable way
- holmesworcester 3y agoHow could it not be the only way, at some point at least? You'd need a simulation that captured all the edge cases of the real world, and that's logically impossible.
- wahnfrieden 3y agoAre you suggesting Musk already exhausted testing and safety development opportunities to the point that the only alternative is a logically impossible simulation
- friendzis 3y agoThat is only impossible if you have not meticulously specified operational domain. Once you have operational domain specified, you can enumerate each and every tiny little detail that can interact. In safety critical systems design specifying operational domain is quite literally one of the first things that are done, even before any serious work on the design starts. The only reasonable phase where safety critical systems can be interacted with closely monitored public are acceptance testing that either validates the model or drags the team back at drawing board.
- bluefishinit 3y agoIf it's being sold on the open market it's moved beyond the "testing" phase and the executives should be fully legally and financially culpable for the damages caused by their untested yet heavily marketed product.
- zwily 3y agoThe version Elon is testing is not on the open market.
- bluefishinit 3y agoI'm talking about FSD in general, but maybe I should broaden my statement: If it's being used on public roads with other drivers, it's moved beyond "testing" and into liability territory. None of the people on the road with Musk consented to being part of his life threatening experiment.
- frumper 3y agoThey also don't consent to dangerous behavior from other drivers. The world is messy and the article is about a trial to judge exactly what you're talking about.
- bluefishinit 3y ago> They also don't consent to dangerous behavior from other drivers. Yes they do. When you get your license you have to past a test that talks extensively about defensive driving and how to deal with other drivers.
- frumper 3y agoThat's a weird way to say you consent to other drivers being drunk on the road.
- 3y ago
- smt88 3y agoStoplights are always red colored circles of LEDs. It's by far the easiest thing SDCs need to be able to do. Any version of SDC software that has a stoplight bug is proof that the QC process of that software is broken and that the makers don't care about safety.
- biosboiii 3y agoTell me you never worked with computer vision without telling me you never worked with computer vision
- smt88 3y agoDo you have anything of substance to say, or are you only capable of childish, condescending meme speak? Do you have examples of stoplights that aren't red circles? Or do you have examples of when a Tesla approaching a glowing red circle should not recognize it a stoplight? I built a CV system in 2013 that detected vehicle makes/models in varying weather and time of day. It was a lot harder than stoplights, we had a team of one, and it was with technology that is ancient by today's standards. Tesla has no excuse.
- frumper 3y agoI can think of 3 distinct styles of red lights in my town alone. One is too glossy and quite difficult for even me to see in certain lights.
- smt88 3y agoIn the aforementioned video[1], the car seems to detect 1 out of 2 stoplights, both of which are clearly visible from the camera. They're illuminated red and not too glossy to detect. Most Teslas have multiple cameras, so the viewing angle shouldn't matter as much to the vehicle as it does to a human. 1. https://twitter.com/RealDanODowd/status/1695268712910348471 https://twitter.com/RealDanODowd/status/1695268712910348471
- frumper 3y ago