58 ms·
Researchers trick Tesla autopilot into driving into opposing traffic
- mises 8y agoMaybe we just shoudln't have self driving cars. The bias of the HN crows seems to be that "oh, technology will advance and solve the problems; just give it time". No code is perfect; no system is 100% secure. This isn't just a malicious case; what of the situations where old lane lines are clearly visible? Can't test perfectly. I'd make a similar argument in things like nuclear centrifuges, namely that there are places where computers should be relegated to their absolute minimum involvement.
- tgsovlerkhgsel 8y agoOr maybe we should accept that no system is perfect and that cars that kill people at a lower rate than human-driven cars are a good idea, even if they keep killing people.
- notatoad 8y agoUnless i missed something, they don't demonstrate tricking the Tesla into driving into opposing traffic, but rather into a lane that could contain opposing traffic but doesn't. If there was opposing traffic in that lane, would the car still change into it? If you put up a sign saying "change into opposite lane" and it looked safe to do so, i'd probably obey the sign while driving my car too.
- deleted 8y ago[deleted]
- logifail 8y ago> If there was opposing traffic in that lane, would the car still change into it? Thought experiment for human drivers: if there were a stationary fire truck in your lane, under what circumstances would you drive into the back of it, without braking?
- gridspy 8y agoPerhaps if I was following a tall vehicle too close then they swerved to avoid the firetruck rather than stopping....? "The Model S was traveling behind a pickup truck with Autopilot engaged. Due to the truck’s size, the Tesla’s driver was unable to see beyond the vehicle in front. “... The pickup truck suddenly swerved into the right lane because of the firetruck parked ahead. Because the pickup truck was too high to see over, he didn’t have enough time to react.”" https://www.teslarati.com/tesla-model-s-firetruck-crash-details/ https://www.teslarati.com/tesla-model-s-firetruck-crash-deta...
- logifail 8y ago> Perhaps if I was following a tall vehicle too close [..] "The Model S was traveling behind a pickup truck..." I'm struggling to see how the geometry works here, how close do you have get to a pickup for it to completely obscure a fire truck in the lane ahead? I can't imagine I'd be happy being in a vehicle that close to another travelling at those speeds. Q: Does Tesla's Autopilot simply let you drive(or perhaps more accurately: "be driven") that close to the vehicle in front?
- EGreg 8y agoThis is a general problem with trusting computer AI. I am talking about image recognition stuff etc. At the very least we need some invariants on a higher level to kick in and heavily penalize any such results! Not just bottom up but we need top down. Where is this feeback mechanism?
- ricardobeat 8y agoWhile the idea is worrying, note that there is no “opposing traffic” in the test performed, and very little info on how the car behaved other than a couple seconds of a TV show. The entire conclusion is: > Tesla autopilot module’s lane recognition function has a good robustness in an ordinary external environment (no strong light, rain, snow, sand and dust interference), but it still doesn’t handle the situation correctly in our test scenario. Not one word more, see last two pages in the paper.
- tomcooks 8y agoWhy link to some dude's twitter, I WONDER, when he links the [actual paper](https://keenlab.tencent.com/en/whitepapers/Experimental_Security_Research_of_Tesla_Autopilot.pdf https://keenlab.tencent.com/en/whitepapers/Experimental_Secu...).
- deleted 8y ago[deleted]
- ccnafr 8y agoProbably because it's 40-pages long. There's even a better summary of the entire research in this Twitter thread: https://twitter.com/campuscodi/status/1112064046369591296 https://twitter.com/campuscodi/status/1112064046369591296 Research is pretty smart, but highly impractical for real world attacks. It's also been patched since 2017, with additional patches in 2018. So, it's also impractical because it doesn't work anymore.
- ady_anr 8y agoExactly. I wouldn't have even attempted to read the 40 page paper. Twitter is more eye catching.
- lucb1e 8y agoWhile I agree about the first and third issue, he's rather dismissive about the second one: "The attack relies on placing dots on the road to trick the car. Lots of them. \n\n I wouldn't worry about it." The demo shows 3 small dots can trigger a lane change. Hypothetically, it could be a very targeted attack if the attacker drives in front of the target to put the paint on the road, and then a car behind the target covers it up. If the cover-up looks just like normal road (no idea how easy that is to achieve), and if nobody is looking for it because it's not a common thing to do, it might never be found. There are probably much better attacks that one could come up with if one gives it more than two minutes' thought, but it seems realistic enough to me to be interesting. Or if not a targeted attack, it can be very disruptive once there is some adoption. Then again, people could already be disruptive and leave stuff on the road or rails, and experience tells us it's very rare (per driven person-kilometer at least). Something about tricking the computer seems more likely to happen than putting physical objects to harm others directly, but that's probably just between my ears.
- dawnerd 8y agoNot just Tesla that gets confused. Other cars with lane assist also get easily confused if there’s markings on the road. Pretty sure my Passat would do the same thing.
- cheeze 8y agoAgree, but Tesla is the poster-child for "self-driving" which certainly makes something like this that much more alarming.
- ccnafr 8y agoI thought Waymo (Google cars) was the poster-child for self-driving cars
- darkpuma 8y agoOne has better tech, the other has bolder marketing.
- imtringued 8y agoTesla's autopilot is very well known to cause deadly accidents.
- ironmagma 8y agoIs it? I would say Waymo is a lot closer to being the poster child; a significant number of people have realized that Tesla isn’t true self-driving despite their claims. Disclosure: I work at Cruise (a self-driving car company)
- cma 8y agoTesla's is supposed to be more than lane assist, something like only regulators keeping it from being released by now. When combined with the big neural link announcement that was supposed to happen weeks ago, you may be able to monitor autopilot while you sleep by having it tap into your dreams. They are also going to give us point-to-point earth rocket travel for less than a first class ticket within 8 years. I'm joking about neural link, though it did miss an announcement, but I'm not joking about the rocket one.
- lelf 8y agoThe actual post https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Security-Lab-Experimental-Security-Research-of-Tesla-Autopilot/ https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Secur...
- Smaug123 8y agoIn fairness, if you put enough fake markings on the road, any human will get confused too. The important thing here is how small/few were the fake markings, not the fact that this was possible at all.
- jkmcf 8y agoThere is a road on my way to work that has a darker asphalt line that goes diagonally across the road for about a quarter of a mile. Inevitably, I and others follow this line before realizing we are leaving our lane.
- function_seven 8y agoEspecially when the sun is low on the horizon and the glare off the road surface makes it hard to distinguish markings from changes in surface sheen. I never drive without polarized lenses now.
- WillPostForFood 8y agoOf course if you put bad input into a system you are going to get bad results, whether human or computer. You periodically read about dumb kids messing with stop signs and causing accidents. https://www.abc12.com/content/news/Stolen-stop-sign-leads-to-crash-that-left-teenager-in-intensive-care-494153591.html https://www.abc12.com/content/news/Stolen-stop-sign-leads-to... https://www.kansas.com/news/nation-world/national/article218982795.html https://www.kansas.com/news/nation-world/national/article218... https://www.foxnews.com/us/charges-dropped-against-1-in-stop-sign-prank-death https://www.foxnews.com/us/charges-dropped-against-1-in-stop... https://www.washingtonpost.com/archive/politics/1997/06/21/3-who-stole-traffic-signs-sentenced-to-15-years/14c0a68c-e8fd-42ba-881c-454f442a27f8/ https://www.washingtonpost.com/archive/politics/1997/06/21/3...
- cimmanom 8y agoHumans have enough intelligence, though, to be able to figure out when the other lane is oncoming traffic and those spurious markings are in fact spurious (snow or a plastic bag).
- joemaller1 8y agoCalifornia has incredibly good roads, generally maintained, thoughtfully designed and well-marked. It's the last place we should be testing self-driving cars.
- hopler 8y agoIt's the first place to test, or else you can't make any incremental progress.
- hellcow 8y agoClearly you've never driven in Los Angeles. :)
- captainredbeard 8y agoThis is sarcasm, right?
- notimetorelax 8y agoChecking in from Europe, California roads can be very confusing or badly marked.
- Elzear 8y agohttps://xkcd.com/1958/ https://xkcd.com/1958/
- Robotbeat 8y agoDon’t know why you were down-voted. It’s quite true that humans have the same kind of vulnerability. Drawing misleading things on the road could kill human drivers, too. It’s not a new threat, it’s just that most people aren’t murderers so it’s not a serious problem.
- rohit2412 8y agoBecause it is not just about adversarial samples. How about old Lane markings, some paint spillover, contrasting color in asphalt, tire marks in snow conditions? They'll all come across as new lanes. These Lane keeping assist systems are nowhere ready to be called driverless.
- telchar 8y agoI downvoted to discourage low-effort posts. I don't want to see this place turn into Imgur where most comments are just links to gify or youtube.
- PorterDuff 8y agohttps://www.youtube.com/watch?v=VkTrQTt2Zdw https://www.youtube.com/watch?v=VkTrQTt2Zdw
- rachelbythebay 8y ago101 at 85 in Mountain View has a bunch of lane dividers with their far ends lifted and turned hard to one side. Might this already be happening, albeit accidentally? The fact it’s the same junction as the fatal crash is also interesting.
- ryanmarsh 8y agoI live along US 290 near Houston which has been a total shit show of construction and hazard for the past few years. Adversarial lane markings don’t necessarily require malice. On 290 confusing lane markings, old lane markings, changes in surface and grade, are hard enough for human drivers. I wonder how state of the art self driving would handle these situations. It’s been a treacherous stretch of highway. I can’t imagine the construction is being done is a safety compliant manner but I’m not an expert.
- deleted 8y ago[deleted]
- alex_lfw 8y agoso that's why elon released a rap song. the ol' trump tactic.
- deleted 8y ago[deleted]
- shambolicfroli 8y agoI wonder how this Tesla issue relates to the admonition "stay in your lane".
- Invictus0 8y agoFigs. 33 and 35 are really lacking. Can we see the stickers at another angle? What is on them, just a white mark? I'm interested to see something more along the lines of Figure 19. That is, could you take a sticker of a stop sign, add some secret noise to it, stick the sticker on top of a regular stop sign so no one knows, and then have the car roll through the stop sign?
- frosted-flakes 8y agoPerhaps. There was a road sign on my street that got hit with a paintball or an egg by some kid, and for years after that it was difficult to see at night. Somehow, whatever was on the sign interfered with the retro-reflector coating, even though it looked perfectly normal during the day. (Not a stop sign, just a narrow road sign for a single lane bridge.)
- samstave 8y agoWhat ports does a Tesla have open? Anyone NMAPed their tesla?
- kaycebasques 8y agoEddie Valiant uses the same technique in Who Framed Roger Rabbit: https://youtu.be/1mGFCGgH1-A https://youtu.be/1mGFCGgH1-A
- tw1010 8y agoI used to be afraid of stuff like this, then I realized that you know what, prison time is a heck of a deterrent against this, which will probably take care of most of any actual risk (at least on a "this is the end of society as we know it" level).
- bo1024 8y agoI think you're interpreting this work too narrowly. Don't think of it as having found the one thing that can go wrong with autopilot. Think of it as one example piece of evidence for how (non) robust the autopilot is.
- hellllllllooo 8y agoThe problem isn't someone doing exactly this, it's that the Tesla auto pilot is based on a bunch of heuristics and assumptions that if they do not hold on certain stretches of road people may die. It's just not safe enough to claim it's a robust working system which is what Tesla and Musk have pitched and sold it as.
- Robotbeat 8y agoHuman driving behaviors are also based on a bunch of heuristics and assumptions that do not hold in all road conditions. It's good to expect autonomous/semi-autonomous to improve on humans, but it's not good to expect zero edge cases to be the bar that must be crossed before allowing this. It's likely that even any significantly improved (vs humans) system would also have new fatal edge cases while also sharing some existing fatal edge cases. What's important is that the systems are not overall worse than humans alone and that they continuously improve.
- hellllllllooo 8y ago> What's important is that the systems are not overall worse than humans alone and that they continuously improve. I work in the industry. These systems are currently far worse than humans and the edge cases are numerous. We have so much brain power and years of experience that the heurustics that a L2+ system uses to make decisions are not really comparible and are far more simplistic and much worse. We can handle new situations, these cars cannot. The fact that you are making this comparison between human heurustics and the vehicle system just highlights that people don't understand the issues. Mix that in with the amount of overpromising and overselling that Tesla is doing wrt the abilities of these systems people will be overconfident in them and end up getting complacent leading to accidents. Tesla's approach to development, release and marketing is not a good way to build safety critical driving systems. Overhyping them to people who don't understand the nuance is my biggest concern.
- sokoloff 8y agoI worked on self-driving cars in 1991 at Daimler-Benz. We used prominent horizontal lines with significant bilateral symmetry and an overall trapezoidal shape as a proxy for a car ahead. (Such was the state of computer vision [and prevailing automotive design] at the time.) We were running a test day on a wet day at a disused airfield in Bavaria and during a test run, the car (bus really) slammed on the brakes, throwing some of the occupants forward, and then, like a recalcitrant horse, the car refused to move forward. It turns out that the drying pavement on one of the taxiways had created a visual pattern on the ground that our vision system ID'd as a nearby car.
- newman8r 8y agoBack in 1991, when were you (and your team) predicting that self-driving cars would be viable?
- sokoloff 8y agoBased on that overall experience, I held the same opinion then as I do today: "never". (I was an intern at the time, though I quickly took over the implementation and operation of our speed control and vision system.) I have to admit: much more progress has been made than I expected in the intervening ~30 years. I still think that full self-driving with zero outside communications/support is "never", though "mostly self-driving with selective help sometimes" might be possible within the next 25 years. It is interesting to see logical descendants of work that group did appearing in road cars. (CAN bus, Adaptive Cruise, park assist, probably others that were lab projects at the time...)
- deleted 8y ago[deleted]
- newman8r 8y agoInteresting. By the time everything's fully automated, it might be hard to call them "cars" anymore anyway.
- dkfndodbxob 8y agoThese sorts of systems tend to be far easier to trick than humans with certain sorts of "camouflage". Is the reverse true? Are there some kinds of markings that confuse people but not modern image recognition systems?
- cromulent 8y agoI was driving towards an oncoming Tesla recently on a 2 lane country road and it lane changed neatly into my side and drove straight at me. Closing speed 160km/h or so. Fortunately it chose to do so with enough time for me to stop, and for the driver to re-take control and take my shoulder as a stopping place. If it had made the decision a few seconds later, it would have been a different story. I think old snow across the centre lane markings was the cause, but if you can't handle the edge cases, then perhaps you shouldn't release.
- StreamBright 8y agoYep. The standard method of software development (release it early and fix it later) does not work for self driving cars and medical instruments.
- thekingofh 8y agoAnd airplanes apparently.
- PunchTornado 8y agowhy are ppl blaming tesla? Isn't it in their manual to always keep hands on wheel?
- admax88q 8y agoIts much easier to get distracted or fall asleep when you're not engaged with driving the vehicle at all.
- rohit2412 8y agoAnd this is how the CEO advertises it https://www.businessinsider.com/elon-musk-breaks-tesla-autopilot-rule-2018-12 https://www.businessinsider.com/elon-musk-breaks-tesla-autop...
- xienze 8y agoThe real problem with any self-driving system is that a good one will breed complacency. After you spend enough time seeing how well it works in practice you’ll become more and more prone to do stuff like check your phone or watch a YouTube video. I mean the car can handle itself, right? You’ve gone thousands of miles watching this thing effortlessly drive itself, why not enjoy the promise of self-driving technology, which is to free you from the tedium of driving? It’s just human nature to do so,
- peterwwillis 8y agoIf you added up all the R&D money spent on these self-driving features, we probably could have already covered 80% of American roads with simple guidance systems that would assist in piloting cars safely. Then nav systems wouldn't need to be as complex, and wouldn't depend so much on trying to guess road factors. If self-driving really is the future, the government's going to need to be involved, the way it is in building and maintaining roads. Many roads already have cameras and sensors built into them just for measuring traffic, so it's not a big leap to say it should also have tech to improve automated driving.
- marcinzm 8y agoIt's the classic approach taken by Ford with the Pinto. If the profit made from releasing is higher than the cost of wrongful death lawsuits then you release. After all, getting money is more important than lives.
- willio58 8y agoNot sure it’s as diabolical as in Ford’s case. Tesla’s are some of the safest cars in the country and autopilot is a work in progress. I do agree that the tolerance for edge case errors should be lower, but also as a driver you should know it isn’t a completely hands-off feature yet.
- cromulent 8y agoNot sure what country, but you'll have a hard time convincing me (and my passengers on the day) that they are safe, no matter what statistics show. Totally a fan of the energy tech, not a fan of the self-driving tech.
- marcinzm 8y agoHumans are not machines, a system where we absolutely have to pay attention 1% of the time but have to just stare mindlessly the other 99% doesn't work. Humans don't work like that and we will space out even if we try not to. Especially when you have to do it for hours on end.
- caycep 8y agoThis is more fantasy thinking (but hey...all the 20th anniversary Matrix articles are popping up...) At what point do machine learning "Adversarial" attacks converge with real life "adversarial attacks". Or do we already have this, but it's called "camouflage"?
- dboreham 8y agoInteresting. This means that one of my predictions: that self driving would happen but only for highly controlled road sections where markings allowed very reliable lane following, is likely not to be realized. Unless we invent digitally signed lane marking..
- cimmanom 8y agoAre pedestrians and cyclists going to have to be "digitally signed" too in order to avoid getting killed?
- Mirioron 8y agoPerhaps it'll be in areas without pedestrians or cyclists? Eg some version of the interstate. But then the question becomes: what happens if there's an accident and humans are on the road? What if there are wild animals in the road?
- codelord 8y agoPreviously I worked at Waymo for a year on the perception module of the self driving car. Based on what I know about the state of the art of computer vision, I can pretty much guarantee that current Tesla cars will never be autonomous. This is probably a huge risk for the investors of Tesla, because currently Tesla is selling a fully-autonomous option on its car which will never happen with current hardware. We need several breakthroughs in computer vision for no-fail image based object detection, and you will need higher resolution cameras, and much much more compute power to be able to process all the images. It's hard to estimate when we will reach that level of advancement in computer vision, my most optimistic wild guess is 10-20 years. And then Tesla will need to upgrade all those cameras and install a dedicated TPU which should be at least 10x or probably more faster than the nVidia chip they have installed on their cars, and they should do it for zero dollars because they have already sold the option. It is kinda amazing that Elon Musk is selling cars based on speculative future breakthroughs in technology. That must be a first.
- jasonhansel 8y agoI think the real potential issue for self-driving cars is that they need to have an understanding of how people's minds work, to understand their gestures, intentions, and movements. This requires, not just visual processing, but something closer to AGI. (Not an expert, so feel free to correct me.)
- dogma1138 8y agoNot really, you don’t really gauge traffic by looking at the drivers firstly you can’t see most of them secondly it’s dangerous to assume their intention which is why we have things like right of way and road signs. The big issue is still in terms of processing speed and sensor fusion and Tesla isn’t leading the pack in either. As far as wide scale autonomous driving goes for it to be good enough there needs to be an agreeable and verifiable decision making model for all manufacturers to follow and for regulators to validate, this could be as simple as never to perform an action that would cause a collision but that’s not exactly an ideal model either because you might then get a silly decision such as you don’t engage the breaks to slow down because you’ll end up hitting the car in front of you in either case. Sadly I’m seeing too many people that tend to complicate things and chase the red herrings when it comes to autonomous driving such as “ethics” while it’s an interesting philosophical exercise in the real world it’s not the important part as when it comes to accidents the vast majority of drivers act on basic instinct and don’t weight the orphan vs the disabled veteran simply because they don’t have that data nor the capacity to incorporate it into their decision making process. First we need to get sensors that don’t get confused by oddly placed traffic cones, birds and shadows the rest is pretty much irrelevant at this point.
- Emendo 8y agoPeople could also be tricked by specifically designed stickers as well. This organization designed a sticker that is designed to scare people into stopping their car. http://news.blogs.cnn.com/2010/09/09/3d-illusion-in-street-tries-to-change-drivers-attitudes/ http://news.blogs.cnn.com/2010/09/09/3d-illusion-in-street-t... https://www.youtube.com/watch?v=8r26AwT7PTM https://www.youtube.com/watch?v=8r26AwT7PTM While people would get angry at being tricked and would put pressure on the city to remove the adversarial sticker, self-driving cars not have the sympathy of the public.
- baby 8y agoThis is awesome! Where can we purchase such stickers?
- 1123581321 8y agoPlease don’t do anything to desensitize drivers to the appearance of children in the road.
- megy 8y agoI know, only a million people are killed on the road every year. Imagine if drivers were desensitized!!
- pavel_lishin 8y ago"This tire fire has been burning for twenty years, there's no reason not to throw this can of gasoline into it."
- 1123581321 8y agoAbout 300 non-occupant (i.e., in the street or on the walk) children a year die from car crashes each year in the US. About a third of those are in school zones. This means two things: one, a solution to force slower driving in school zones would not prevent the majority of fatalities, so if that solution was counterproductive outside of school zones, it would reasonably increase fatalities. Two, 300, or 100 in a school zone, suggests the vast majority of car-child interactions are not fatal or even injurious. A solution that prevented a couple of the existing fatalities could easily turn more of the much larger number of safe encounters into dangerous encounters. And this is plausible because right now, people are watchful and quick to slam on the brakes when they see a child, even out of the corner of their eye. Teaching them that an area where children walk has a “child” they can ignore would weaken that reflex where it should actually be strengthened. I understand you are concerned about children as well, so I would advise looking into policies regarding parking on the side of the road. Roundabouts in neighborhood intersections near schools also have demonstrated a natural slowing effect that does not seem to have negative externalities beyond the cost to implement the roundabout.
- mindfulplay 8y agoThis is an example of releasing things that are potentially unsafe but 'meh we are part of silicon valley enthusiasts, let's fail fast and fail often'. The slight problem with this approach is the failures end up killing a few human beings. Sure, you can learn from the failures but is this the right way? Very callous and self-righteous behavior from these Elon Musk-style people.
- emcq 8y agoFor articles like this there always seems to be a few comments suggesting statistically Tesla's are the safest cars. Are the cars safe or is it the drivers? The population of Tesla drivers is far from average; the average driver certainly can't afford a 40-80k+ car and may not be interested in electric.
- rohit2412 8y agoAnd what exactly are they comparing. Just because Lane keeping assist reduced accidents doesn't mean that we remove human drivers. As long as the driver has to keep hand on the wheel like autopilot, it remains an assistive technology, and does not support any arguments for driverless technologies.
- paul7986 8y agoYup not going to pay thru the nose to be any billionaires guinea pig! Unfortunately others will posing a new threat on our roads that previously didn’t exist. Progress is a killer and that kills the innocent at the hands of the mega wealthy.
- maxander 8y agoThe “trick” was to add fake lane markings; did they test to see how many human drivers are susceptible to the same attack?
- mistrial9 8y agohey, World ! I dont want a self-driving car with tracking ! thank you
- iabacu 8y agoTesla self-driving is like Theranos. A big fake-until-you-make promise that gambles with peoples lives, in exchange for some hype and funding.
- jaimex2 8y agoGood work by the researchers. I'm sure Andrej Karpathy and team are already working hard on improving this part of Autopilots neural network. Most of the updates seem to show the system is memorising things it sees at a distance and building a "mental image" of what the environment is and adding more detail as it get closer instead of assessing every frame individually.
- Robotbeat 8y agoInteresting. I wonder if, as the number of Teslas on the road increases, their "mental models" could be shared so that obstacles detected (with high confidence) by earlier cars can be shared the later cars traveling on the same stretch of road. Like how drivers sometimes signal road conditions (speed traps being probably the most common one) to other drivers. Eventually, that should probably be standardized in an open, broadcast format to other non-Tesla vehicles as well.
- jaimex2 8y agoThat would certainly make sense. A huge benefit would be it would make it a lot easier for the cars to co-ordinate which lanes they should be in to maximise road throughput.
- paulorlando 8y agoImagine boing able to do something like this at scale: https://unintendedconsequenc.es/autonomous-vehicles-scaling-risk/ https://unintendedconsequenc.es/autonomous-vehicles-scaling-...
- gzak 8y agoNext, someone will stand on a street corner wearing a stop sign t-shirt... All hell will break loose.
- dgudkov 8y agoI don't understand the haste with self-driving cars. The technology and related legal issues (who is responsible for accidents?) are not prepared for mass use. Why put cars with half-baked self-driving technology on regular roads and highways with other drivers who didn't consent to this? Why not start from predictable and tightly controlled environments (e.g. inner territory of a factory or a cargo port), limit their speed to 10-20mph, spend a few years refining the technology, and only then gradually expand the operating envelope. When Boeing puts half-baked autopilot technology into MAX8 that's a horrible negligence. When Tesla does a similar thing it's for some reason OK.
- web007 8y agoIt seems like a Wile E. Coyote attack on Autopilot - draw a line that turns into a wall, and the car will follow it blindly. I don't see how this is significantly different from human driver actions. You could put cones up and a human would deviate as well, or put up something they don't recognize to effect the same reaction.
- frogpelt 8y agoI feel like we are going to need smart roads before we can have autonomous vehicles. If the road could talk to the car, the car would be harder to fool or confuse. Maybe.
- elamje 8y agoThis is hardly tricking the Tesla. In the video they have no cars nearby for the Tesla to also infer from, so saying it's "driving into opposing traffic" is misleading. I would imagine the Tesla would do fine if there were cars in front of it, and cars in the oncoming lane that it could infer with. With that being said, I'm sure there are plenty of corner cases where the car can be legitimately tricked.
- blueface123 8y agocool beans
- blueface123 8y agoi tricked the autopilot to drive in the right direction is that normal?