12 ms·

Hacked billboards make Teslas see phantom objects causing them to swerve or stop

• mike_d 6y ago
  States just need to start mandating LIDAR for any level of autonomous certification, in the same way we finally figured out seatbelts were the minimum level of safety for passenger retention.
  • njarboe 6y ago
    Mandating how a certain outcome should be achieved generally is not the best (efficient, quickest, cheapest, etc.) way in getting to that outcome. Autonomous certification should have its safety requirements for each level and require the vehicles meet that standard. Autonomous vehicles have the potential to reduce deaths and injuries dramatically and increasing the time to get there will cause thousands to millions of more deaths and an order of magnitude more injuries, not to mention property damage.
    • nexuist 6y ago
      This isn't necessarily that strict of a regulation. They could require the sensor be on board but not actually regulate any of its usage. It could be a scenario like the Model 3's rear view mirror camera, where it wasn't used at all for the first two or so years and then later got a software update that started interacting with it. The important part is that the LIDAR unit is physically wired in to the system so that they can start using it in software whenever they feel like it's necessary.
    • tshaddox 6y ago
      I don't know. In general I think you're going to need some balance between requiring certain outcomes and requiring certain specific implementation details. It might be possible to make an automobile out of sufficient thick and squishy parts that it could pass reasonable crash safety tests without having a seat belt, but it's still not ridiculous for regulations to require that automobiles have seat belts. (I have no idea where lidar for self-driving cars falls within that balance, although I can't help but notice that human drivers do not utilize lidar.)
      • DenseComet 6y ago
        Well, lidar is just a stand-in for the depth perception humans already do. It is absolutely valid for regulators to require that the systems on an autonomous car do depth perception, and provide a standard to prove that the system works well. I suspect that right now, ML based approaches to depth perception would not pass the test to standards we would like, in practice requiring lidar. But, this would also provide an option to switch to an ML based gets good enough.
        • acadien 6y ago
          This is a common misperception. You cannot estimate the shape of simple things in low light. Even humans are susceptible to optical illusions and are effectively blind during certain times of day (try driving in the direction of the sun during sun rise/set). Lidar is not susceptible to any of these things. Similarly ML will never solve low light or direct light situations, that is a physical limitation of cameras.
        • tshaddox 6y ago
          There are popular cars in the US that use two cameras for their self-driving functionality. Kinda like humans.
          • lttlrck 6y ago
            Those cameras are fixed whereas eyeballs have multiple degrees of movement via the eye socket and skull mounting apparatus allowing very sensitive parallax measurements. It's really not the same at all.
    • seanhunter 6y ago
      Autonomous vehicles do of course have the potential to reduce deaths but there are lots of examples of this being the preferred method of regulation in the auto industry. For example if you want to sell a motorcycle in the US it has to have a kill switch in a specific place on the right handlebar. You don't (as a manufacturer) get to innovate about this because it's a safety feature and a first responder needs to be able to locate the kill switch so it always has to be in the same exact place so they can always find it. If you want to sell a car in Europe (I believe) it has to have seatbelts, ABS and a catalytic converter that meets a particular specification etc etc.
  • aaaxyz 6y ago
    Not necessarily mandating LIDAR but at least mandating functional redundancy for self-driving / driving assistance. I'm guessing that won't happen until the deaths start piling up though.
  • contravariant 6y ago
    Is LIDAR not susceptible to this? I guess it might be more robust against random hiccoughs (though I have no basis for this whatsoever), but is it also protected against malice?
    • ianmobbs 6y ago
      LIDAR maps out the world around you in three dimensions, so with that data, there's no way it would mistake "a 'Stop' sign hidden within a fast food commercial [...] only flashing on-screen for a fraction of a second" (article) for an actual stop sign
      • brianwawok 6y ago
        But it would fall for a cardboard stop sign on a stick held up by a troublemaker.
        • dwohnitmok 6y ago
          As would a human given a sufficiently official-looking stop sign. In fact I think it's actually impossible for an automated car or human without figuring out the "troublemaker"'s identity whether the stop sign is fake or not. And it's not always possible to pin down someone's identity (if the person is wearing a brightly colored construction vest what do you do?).
          • couchand 6y ago
            There's a lot of construction in our neighborhood right now which made me start thinking about that possibility. It kind of sounds like something out of a heist movie (or taken to the extreme, Die Hard With a Vengance?), but really, who would know and care enough to ask the questions to notice something like that?
        • noodlesoups 6y ago
          That requires a whole different kind of effort. These hacks can be done remotely from the other side of the world and leave little trace
  • Diederich 6y ago
    Can LIDAR reliably detect stop signs? I think it would have to be based on the shape, and LIDAR loses resolution pretty quickly over distance.
    • airstrike 6y ago
      You could always map the stop signs in advance
      • kurthr 6y ago
        until they move, get removed, or are run over...
      • dreamcompiler 6y ago
        Which would fail in the case of construction workers holding up stop signs or school buses deploying stop signs, to cite just two examples.
        • nexuist 6y ago
          It's actually comical in a way that we possess the technology to build robots that can operate heavy machinery and practically interpret the world the same way we do, but we still have to worry about people holding up plastic signs. It makes me think that the first real wide scale self driving vehicle deployment will have to come from a nation state rather than a corporation; the easiest way to solve all of these problems is by putting heavy investments into smart road infrastructure such that the car depends on sensors embedded in the signs or the asphalt rather than any camera feed. A school bus could just have a transmitter to indicate whether its lights are on or not; construction workers could be given road signs that transmit similar signals or better yet use a portal to construct a virtual geofence around wherever the construction is happening before it begins. These are all easily solvable problems in today's world; what is missing is the political capital to invest in these solutions despite the insane ROI.
          • dreamcompiler 6y ago
  • GuB-42 6y ago
    Tesla cars are far from being autonomous. Autopilot (including the so called "full self driving") is level 2, meaning that the driver has to pay attention at all times. From a driver responsibility point of view, it is a fancy cruise control, there is no question about autonomous certification because the car isn't autonomous. If one day, Tesla thinks it is OK for drivers not to pay attention and wants to make it legal, that's when the question about what kind of tech is reasonable will be asked. But for now, it looks like we are far from it. There is a huge gap between "works well most of the time" and full autonomy.
    • thelean12 6y ago
      I've said it for a long time but: Tesla implementing level 2 is dangerous and irresponsible. Calling it "autopilot" is even worse (and has been ruled as misleading in at least German courts). If a human has to sit there and monitor the car without interacting with it, then we shouldn't allow it on the roads. That means skipping level 2 and probably level 3. Go the Waymo route and go straight to full self driving cars.
    • nexuist 6y ago
      This day came and passed years ago. It's evident from their marketing that there is no attempt to discern fancy cruise control from "zomfg your car will drive you around in a year!!" Whether this is false advertising or even just moral is up in the air, but Tesla is making no attempts at educating consumers about the limits of their autonomy other than the bare minimum (yeah you technically have to keep both hands on the wheel, but that mechanism is easily defeated and you can even buy clips on Amazon that do it for you). I'm pretty sure Elon even liked a tweet about a Pornhub video where someone banged a model on the highway with autopilot. Which is an excellent day for PR and a disastrous day for legal. The point being, sure, academically they are a far way away, but they are trying to convince consumers Level 5 is just around the corner.
      • tshaddox 6y ago
        I'm coming around to agreeing with you about the deceptiveness of Tesla's marketing of Autopilot. I recently bought a new car from a non-luxury brand that's quite popular in the US, and a few days after buying it I was a bit surprised to realize that its "dynamic cruise control" and "lane keep assist" features amount to roughly the exact same thing as Tesla's current Autopilot offerings. I am still shocked at how undersold these features were, both by the dealers and the car's literature. I was driving for several days before I realized that, in addition the the dynamic cruise control I was familiar with, a simple button press could get the car to steer itself to stay in its lane.
  • dawnerd 6y ago
    Why? Cameras can work just as well as your vision. Humans manage just fine and don't have LIDAR. Just because it causes the cars to slow down now doesn't mean it will in the future.
    • sli 6y ago
      It's a little strange to make a direct comparison between human cognition and computer vision given that the latter lacks the true ability to understand what it sees. You can train the computer for these edge cases, but humans largely do not have them in the first place because we're capable of formulating highly accurate understandings of what we see. You can of course pick out discrete examples of humans failing tasks like these, but entire classes of computers routinely fail at things like this.
    • roughly 6y ago
      Your vision is backed by a neural network with around 80 billion neurons implementing a general learning and intelligence model we don't have the first clue how to replicate. We're a bit further away from "Cameras working just as well as your vision" than I think you're anticipating.
      • throwaway2048 6y ago
        Also your eyes are far better at quickly adapting to things like wild changes in brightness and contrast than any camera
      • tshaddox 6y ago
        > We're a bit further away from "Cameras working just as well as your vision" than I think you're anticipating. Well surely that's what we should be testing. Is a self-driving car system safer than a human, or not? You're correct that self-driving car systems lack a "general learning and intelligence model," but they also have several obvious advantages over humans, like potentially better-positioned cameras, a more reliable "attention" system, faster reaction times, potentially more "experience" with different road conditions and scenarios, etc.
    • 6y ago
  • gruturo 6y ago
    While I agree with you on mandating it, a LIDAR "only" gives you a 3D point cloud. Great for detecting if an object is in your way - but it cannot read a stop sign, you still need a camera system for that - so they still have to harden it against this hack (and who knows how many to come).
    • tshaddox 6y ago
      Wouldn't that "hack" just be as easy as sticking an actual stop sign somewhere it's not supposed to be, or removing a real stop sign? That would obviously fool a human driver just as much, and it has actually happened, and killed people.
    • bonestamp2 6y ago
      > so they still have to harden it against this hack True, but LIDAR should make that detection more robust. For example, if a stop sign is on the same plane as the object that surrounds it, that's a strong signal that it is not a real stop sign but rather a projected or displayed one. That's much harder to detect with computer vision alone, especially if they're dealing with hackers who can test it with a real car and continually iterate on it to find different ways to trick it.
      • nitrogen 6y ago
        For example, if a stop sign is on the same plane as the object that surrounds it, that's a strong signal that it is not a real stop sign but rather a projected or displayed one. It's conceivable that in denser or older areas, signs are sometimes placed on structures because there isn't room for a pole.
        • bonestamp2 6y ago
          Agreed. I didn't say it's definite, I said it's a strong signal. Even strong signals need to be weighed against other available data. Apple Maps shows stop signs and traffic lights on the map. A database like that would be another strong signal, but also not always accurate.
  • glaucon 6y ago
    Not being sarcastic but how would LIDAR help here ? Maybe .... before acting on a Stop sign sensed by the cameras the control system would confirm, via LIDAR, there is a post with a plate mounted on it in a position which corresponds to where the camera thought it saw a Stop sign ? Can LIDAR do that level of sensing, ie unambiguously find a metal post/plate out of all the other stuff which might be in front of the car ? And assuming my guess about how LIDAR would help is correct what happens when an obstruction between the car and the Stop sign means that LIDAR can't 'see' the post but the cameras can see the sign ?
• klyrs 6y ago
  Hacked billboards are hard to come by. We've already seen signs that autopilot can be tricked by malicious lines on a road. Not that long 'til somebody points a projector at the ground in front of a Tesla and it crosses into oncoming traffic. You could even mount that projector on a panel van, and use image recognition to target Teslas and otherwise stay inactive
  • dx87 6y ago
    That's what the article says. Hacked billboards were one method, drones with a projector were another, and the image only had to be shown for less than a second to cause the car to react.
    • markdown 6y ago
      Did they try the same with unwitting human drivers?
      • dx87 6y ago
        Read the article and find out.
        • ecpottinger 6y ago
          Interestingly the article says it does not work on humans because it is just a image shown for a short time that people do not register the images. Seem simple to handle, as the car computer see an image, it checks if the image show up in a second scan. Since the computer scans are very fast they should not effect to computer processing/reacting to driving the car.
  • dheera 6y ago
    > projector at the ground in front of a Tesla and it crosses into oncoming traffic I don't understand why people come up with all these ridiculous "what if" ideas against autonomous cars as if it's anything new. It's already exceedingly easy to foul up a human driver. Stand next to a freeway and hold up a stop sign. Drop a foam mannequin from a bridge over the road. Paint some bogus lane lines. Shine a laser at their face. Any of the above will cause accidents with humans, and you'll be arrested and charged with a felony. Deliberate attempts to compromise autonomous car systems carrying humans inside should be treated no differently.
• tinus_hn 6y ago
  Clever but don’t forget you can also ‘hack’ human drivers by showing flashing patterns or just generally distracting them.
  • bsder 6y ago
    There is a reason for all the stereotypical commercials showing a pretty woman walking by causing chaos in her wake ...
  • darepublic 6y ago
    Not this easily
  • uniqueid 6y ago
    If Tesla made humans, that fact would be a problem for them.
  • tantalor 6y ago
    It's so easy all you have to do is shine lasers at their eyeballs and it bricks the car!
    • tinus_hn 6y ago
      You can’t make a billboard do that remotely.
  • InitialLastName 6y ago
    Just because Tesla isn't very good at making cars that drive themselves doesn't mean that humans are any better at driving those cars.
    • mhh__ 6y ago
      I think it's fairly obvious that the humans are better in this particular circumstance.
• mattbee 6y ago
  This is clever, and Tesla's software is awful, but anyone doing this for real, with intent to divert cars from the road etc. would be committing a pretty serious crime.
  • Polylactic_acid 6y ago
    The wonderful thing about IoT is the criminal can be on the other side of the world with no chance of any justice coming to them.
    • falcolas 6y ago
      Not to mention, from TFA: "They're also so fleeting that they're difficult for the human eye to detect, and leave behind very little evidence." We've already got people who do things purely for the lulz (swatting, anybody) who could easily justify away these kinds of activities.
      • mattbee 6y ago
        Both good points - so this kind of crime would likely be indistinguishable from a random, terrifying software failure, with the blame falling on the car maker.
        • hellotomyrars 6y ago
          I mean it is a failing of the manufacturer on some level. Not saying someone intentionally exploiting the car isn't also at fault, but I also don't think it's unrealistic to hold the manufacturer accountable as well.
        • piracy1 6y ago
          Tesla logs so much data and stores it and has used it to defend itself in various legal / PR matters. I wonder if they would catch it if someone did this. Or maybe suddenly they wouldn't have that data lol
        • a_bonobo 6y ago
• liquidify 6y ago
  Why don't we build some experimental infrastructure that is designed specifically for self driving vehicles?
  • aeroplain 6y ago
    Having to invest is specialized infrastructure for self-driving cars defeats the whole point of them. Any money spent on infrastructure for self-driving cars would be better spent on public transportation.
    • nexuist 6y ago
      This is like saying that investing in asphalt is pointless when horses can walk on dirt or cobble. The fact of the matter is that building new public transport is monumentally expensive, but roads themselves already exist. Installing new sensors and beacons would certainly not be a cheap task, but the bulk of the work is already done and there's no reason why these new systems couldn't integrate with existing roads that humans drive on. You can imagine for example a speed limit sign with a QR code containing information for the car to read, or a stop light using a radio transmitter to indicate its state. This infrastructure has to be upgraded anyways; why not make it friendlier for computers to use it? The costs pay themselves when crashes and traffic jams go away.
      • otherwiseguy 6y ago
        Is public transportation really monumentally expensive when compared to almost every adult spending somewhere between $5,000 and $150,000 every 5-10 years on a car? AAA says the average American spends nearly $9k/year owning/operating a vehicle. That'd buy a lot of public transportation infrastructure.
        • hansvm 6y ago
          Public transportation is often less convenient. I've lived places where the hub-and-spoke bus model wound up taking longer than just walking from one end of the city to the other. How much is an extra half hour twice a day from a shorter commute worth to you? If nothing changes other than the public transportation infrastructure then I'm not sure that gap is easily bridged.
• smallstepforman 6y ago
  There are many reasons why I dont believe in self driving cars, the first (and strongest) is to imagine that the software was written by my collegues (and myself).
  • colinmhayes 6y ago
    I mean in that case imagine these people being allowed to drive. There's no reason computers can't be better than people, and many reasons they could be better.
    • throwaway2048 6y ago
      Not sure what being able to program has to do with being able to drive.
    • pskinner 6y ago
      Being able to drive safely is not the same skill as being able to construct a system that can drive safely.
    • falcolas 6y ago
      > There's no reason computers can't be better than people It is entirely possible, however, that the computers and software systems available today can't be better than humans.
      • ggreer 6y ago
        There are advantages and disadvantages to each. The computer failure methods are new to us, but we're all too familiar with human failings. Humans have much slower reaction times. Humans get sleepy, drunk, or distracted. Humans sometimes get angry or impatient and do dangerous things with their vehicles. The important metric is accidents per mile driven. Right now, Teslas on autopilot seem to be significantly safer than human drivers. From Tesla's vehicle safety report[1]: > ...we registered one accident for every 4.53 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.27 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.56 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 479,000 miles. Yes autopilot has issues and limitations, but it's already 3x safer than unaided drivers. Also, it seems to be improving in safety. Two years ago, autopilot's crash rate was only 1.5x lower than unaided Tesla drivers. 1. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport
• roughly 6y ago
  This appears to be the actual research: https://www.nassiben.com/phantoms https://www.nassiben.com/phantoms
  • rigrassm 6y ago
    Thank you, that newsweek article was a shit show on mobile.
    • nitrogen 6y ago
      Not much better on desktop. There wasn't even an image or relevant video, just an unrelated "20 facts about Tesla" video.
• maxkwallace 6y ago
  This is a special case of a general class of vulnerabilities in AI models, where an adversary can cause undesired output from the model by constructing input data not represented in the training set. However, it is legit much more concerning than, e.g. the issue of image classification models mis-identifying well-constructed noise as "panda". This is currently a research frontier for AI so us non-experts likely won't be able to say a ton about it. I thought this was a good talk on the issue: https://www.youtube.com/watch?v=SS9DMr4VkbY https://www.youtube.com/watch?v=SS9DMr4VkbY
• robocat 6y ago
  Are humans similarly vulnerable?
  • nefitty 6y ago
    Humans can hallucinate and are susceptible to optical and auditory illusions.
  • zeta0134 6y ago
    The thing humans have, that so far trained machine-learning models seem to lack, is a fairly reliable BS detector. I think the best example of this is an inkblot test. Sure I can look at an inkblot and see fanciful shapes, maybe a chair, maybe two rabbits kissing, whatever. But in the back of my mind, I also know perfectly well that I'm looking at an inkblot; I can entertain all the other classifications, but would never accept them as fact. The question becomes, how do we give an AI the concept of a BS detector? How can we train it to know, somehow, that the classification it has just made with a high degree of confidence makes no sense? This seems like a very difficult problem to solve.
    • noodlesoups 6y ago
      Context awareness is what one needs for sensing BS, a herculean task
  • saulrh 6y ago
    Yes. Flashing lights at the correct frequency and intensity can cause a noticeable fraction of the population to immediately seize. Emergency vehicles are a known trigger. Less glibly, yes. There are particular arrangements of roads which are known to get people killed by cars. For example, an intersection at a particular angle with particular speed limits cause cars to stay hidden in each others' a-pillars as they approach the intersection. There are bits of highway that are known to cause human drivers to drift; this is why many stretches of highway now have textured strips down the side to notify drivers that they've moved out of their lane.
    • heavyset_go 6y ago
• fyp 6y ago
  There are a lot of similar failure modes in humans. Sometimes they're are used for good, like making drivers slow down: https://www.insider.com/optical-illusions-3d-crosswalk-driving-2019-5 https://www.insider.com/optical-illusions-3d-crosswalk-drivi... But other times they cause crashes: https://imgur.com/a/kYr94 https://imgur.com/a/kYr94
  • Barrin92 6y ago
    >There are a lot of similar failure modes in humans. there are lot of similar failure modes in individual humans. The difference is individual driver errors are not correlated, a bad reflection tricks a few humans at once instead of tens of thousands of cars running the same model. There is significantly more brittleness in a fleet of cars than in a population of humans, because the human population has diversity in judgement and experience. This is not simple to fix because this uniformity is actually a feature of automated systems, it makes them explainable, produces expected results and conform and cheap, training a human for 20 years is more expensive than training all the cars once. However it also makes them collectively vulnerable, which is why heavy machines tend to be locked away on factory floors.
    • newen 6y ago
      True. People need to stop comparing self driving cars favorably to the worst performing humans. Gives companies license to release these badly performing cars into the streets.
      • hughisthis 6y ago
        They are assistive systems. The only thing that's badly performing are the users who misuse them and get themselves into trouble.
        • gmadsen 6y ago
          hard braking or swerving for clear false positives is not very assistive... it seems to be the opposite of assistive. How can a person know all the weird things that will make a computer vision system badly fail. It shouldn't be used at all if that is the case
• ip26 6y ago
  Ok, so can we agree to get rid of billboards?
  • kovek 6y ago
    The idea to remove all billboards for teslas is really funny. I would say that this billboard hack means anyone could install a banner on their balcony to get a tesla to swerve onto a neighbour’s house. However, I believe this will be fixed in a later update.
    • ip26 6y ago
      I'm not saying we should remove them just for Tesla. There's a hundred other reasons too.
    • bonestamp2 6y ago
      > I believe this will be fixed in a later update Maybe. I hope so. But from my understanding, context is the hardest part of computer vision.
      • toxik 6y ago
        Yes, human reasoning is not just pattern matching but mixed with deductive thinking. A picture of a stop sign is not a stop sign, neither is a reflection of one. Trying to teach a neural network all of these exceptions seems like a terrible approach and would likely not work very well.
        • bitwize 6y ago
          That's why getting a computer to do basic real-world shit that humans do with aplomb will likely require a merger of GOFAI with the new, statistics-based AI.
  • goodcanadian 6y ago
    Hawaii (among other places) has banned billboards. It is actually quite nice not to have annoying advertisements all over the place. So, it is possible, but it will take a concerted political push.
• jvanderbot 6y ago
  Cars should not detect "obstacles". They should detect roads. They should not seek _exceptions_ to driveable space through classification- or segmentation-first strategies. They should only detect driveable space. Full stop. You either detect a full-stopping-distance's worth of clear road through reliable (non visual) ranging with dense enough samples to preclude wheel-catching voids ... or you will continue to run into "objects" which slipped through your classifier. I can accept reduction in this margin (e.g. in dense traffic flow) when it is first shown to work with margin.
  • krainboltgreene 6y ago
    > They should only detect driveable space. I'm interested to hear how you decide what's drivable without detecting things that decide if roads are undrivable.
    • jvanderbot 6y ago
      You don't need to know it's a "car" or a "cow" or a "corvette" to know that the flat surface you are on ends in less than your stopping distance. Semantically, it's a meaningless distinction. But in implementation it's much easier to detect the shape of the upcoming terrain using, say, lidar or regular old multi-camera disparity. Flat and following your map: that's driveable. I'm guessing many take this approach, but it's seeming more likely that Tesla does not, and instead trusts monocular or arrayed cameras to track objects though frames from detection-first regimes. Seems dangerous. I'm glad you were interested.
      • dmitriid 6y ago
        A one-way street is also a drivable flat surface. As is a road with no lane markings. Can you guess why blindly driving down each of these will be dangerous?
        • toxik 6y ago
          Right, but can you guess why stopping in the middle of traffic for a fake stop sign is bad? Walk before you run is the essence of the argument here.
• timpattinson 6y ago
  Better article: https://www.wired.com/story/tesla-model-x-autopilot-phantom-images/ https://www.wired.com/story/tesla-model-x-autopilot-phantom-... Original research: https://www.nassiben.com/phantoms https://www.nassiben.com/phantoms
  • salmon 6y ago
    The newsweek article has basically no information. Either of these would be much better
• morpheuskafka 6y ago
  I think the concept of the line of death, from browser security UI design, is relevant here. The space outside of the highway right of way is completely uncontrolled and the Tesla should know better than to look at a billboard hundreds of feet up in the air for regulatory signs. The MUTCD has rules about where signs are to be placed--look only in those places. [1] https://textslashplain.com/2017/01/14/the-line-of-death/ https://textslashplain.com/2017/01/14/the-line-of-death/
  • shajznnckfke 6y ago
    The problem is that they don’t have lidar and therefore can’t reliably measure depth, right? A billboard far away high up in the air may appear in the same direction as a closer road sign if you don’t have depth perception. You should be able to infer it from a camera with the right model but those methods are more fallible. Similar root cause to that time the guy got beheaded by a white truck perpendicular to the highway that looked like the sky.
    • kirillzubovsky 6y ago
      I believe they are making depth perception using computer vision. The idea is that if you can take an image and predict the distances to the objects in it, you can then create an image that is a transpose of the original based on your predictions, but at a different distance. In case of Tesla moving forward, that would be a split second later, closer to the original image. As the car moves forward, you actually take that second image and compare the virtual to the original. At this point you probably get it wrong, but you adjust your predictions and recalculate until your virtual transposed image looks exactly like the real image, at which point you've got the distances pretty darn close. I remember watching a talk about their computer vision, and it was pretty impressive. The challenge is that despite knowing where the billboards are, they still look that way because sometimes, in obscure situations, in little old towns with fuzzy rules, road signs actually do appear in weird places, and that's where the car often trips up because without sufficient data it just can't build a prediction model to know when to respond, and when to suppress the new input.
• quickthrower2 6y ago
  Of course you could hack a billboard to show a picture that would distract human drivers. Infact most billboard ads aim to do just that, since most humans on the road are driving (most cars commuting with 1 person).
• solarkraft 6y ago
  I'f you put a fake road sign somewhere people are probably going to obey it as well.
  • nojster 6y ago
    That is likely, if the fake street sign were to resemble a real one both in appearance and context. If it were a stop sign in a billboard ad, humans would ignore it, though.
  • techbio 6y ago
    I spent a second trying to figure out that apostrophe, and found it humorous, about the same as i'f I saw a stop sign on the shoulder of a freeway.
  • jtsiskin 6y ago
    Would they, if you put on in the middle of the interstate?
• cblconfederate 6y ago
  Anything with software sadly becomes a weapon. Especially when its heavy bulky and potentially deadly. There will be a lot of money thrown on adversarial attack mitigation. But how much is too much? At some point it might become so expensive, that it will be cheaper to switch to guiderail-based self driving, which can be overseen and secured centrally. And self-driving planes are probably going to be a nonstarter
• deleted 6y ago
  [deleted]
• hughisthis 6y ago
  4D rewrite will solve this. Labelled video datasets FTW
  • deleted 6y ago
    [deleted]
• hughisthis 6y ago
  the internet will never work
• NiceWayToDoIT 6y ago
  Can this simply be prevented by sign edge detection? Are there standard traffic sign dimensions? If new Tesla can detect depth, then it should be possible to understand what is billboard and to ignore it. I do not know laws in America is it possible to display real traffic signs digitally in billboards?
• wazoox 6y ago
  My wife nearly had an accident because of a huge ad billboard that suddenly displayed huge roaring flames, to advertise for some steakhouse. My general feeling is that these sort of intrusive, dangerous, anti-ecological (such a billboard consumes as much power as a family of 4), anti-social (one less worker to change billboards) ads should be banned, period.
  • kalium-xyz 6y ago
    What do you think about e-ink billboards?
  • zimpenfish 6y ago
    I worked at a place trying to do roadside LED billboards (~2009 before they were common) and the rules about content (in the UK, at the time) were fairly strict - nothing animated, no cuts only fades even between distinct adverts, etc. From what I've seen of the fancy new ones, those rules seem to be mostly still around (in London anyway.)
  • chungus_khan 6y ago
    They also needlessly drive up land value around major roads, and I don't feel much sympathy for companies that subsist off of simply owning said eyesores.
• shultays 6y ago
  I mean if I see a stop sign on a billboard I would stop as well, I wouldn't consider this as Tesla's fault.
• molszanski 6y ago
  If I saw a projection of a person I would emergency brake too.
• pritovido 6y ago
  Billboards at night should be forbidden. Period. I almost had an accident when a billboard that was dark suddenly displayed a very bright image on my side. My body reacted moving suddenly the steering wheel as a reflex action. At another one having some big image being moved actually made my eyes follow the movement instead of the road with cars on it. I had to make work in order to focus on the road and not the screen. It also makes your eyes to loose your vision in the dark.
  • black_puppydog 6y ago
    > Regularly operating billboards make Humans see phantom objects causing them to swerve or stop (somehow-no-newspaper-ever.com)
  • noodlesUK 6y ago
    I suggest a simpler solution. Billboards (particularly the LED array ones) should be forbidden, period. They are explicitly designed to capture attention away from drivers whilst they are doing a very dangerous activity. Why? Because they attract customers and make businesses money. This kind of thing needlessly has a human toll, and we could eliminate it with the stroke of a pen.
  • proactivesvcs 6y ago
    The entire concept of roadside advertising should be seriously looked at. The roads are dangerous enough as it is - especially with attention gradually being degraded by cockpit distractions - without superfluous equipment which is specifically designed to attract attention.
    • eyelidlessness 6y ago
      As an anti capitalist (who has taken a fair bit of flak on this board for it for obvious reasons), that sounds intriguing. As a road trip enthusiast, that sounds either: - terrifying: finding services in far flung places could be nearly impossible - likely to recreate both of the problems I’d want it to address in potentially more harmful ways: for road safety, it means discovering services would depend even more on electronic devices inside the vehicle and further from the safe driving field of vision; for profit opportunism preying on drivers’ eyes, it consolidates the market much further than it already is, and with even greater harmful incentive
• emiliobumachar 6y ago
  Malicious attacks on human-driven cars would work too. Relevant xkcd: https://xkcd.com/1958/ https://xkcd.com/1958/
• helsinkiandrew 6y ago
  I'm not sure that counts as a phantom object. That's something that looks like a stop sign, positioned where a stop sign could be. Admittedly its only shown for half a second but I imagine a distracted driver who looked up suddenly could be equally fooled. Having video ads on screens that drivers can see and be distracted by is the real problem.
  • alistairSH 6y ago
    Possibly, but many/most humans would quickly realize the sign was out of place, quickly vanished, etc and not stop the car or swerve too far. This is a solvable problem. Just another example of where "AI" is lacking a sense of intuition.
    • deleted 6y ago
      [deleted]
    • dontcarethrow2 6y ago
      It could be given an extra sense with LIDAR but that is 'a fool's errand' and not as profitable. I should start carrying a stop sign in my pack, have some fun on the highway.
      • alistairSH 6y ago
        As a cyclist who frequently gets buzzed by cars, it's certainly interesting to ponder the possibilities.
        • proactivesvcs 6y ago
          A thought which had come to my mind while reading this thread. One of the UK's "lane closed" signs, perhaps? ;-)
• black_puppydog 6y ago
  Anyone else getting strong "I, Robot" (the movie) vibes from this line of research? I can see this happening as an opening shot to an action movie. Some reporter finds out that $government is doing $bad_thing. They pack up the evidence, rush out the door to talk to their editor. A slick, self-driving car pulls up as they exit the building, they enter and pull away. It is a rainy night, and the car zips along the city roads along thousands of others; just another regular evening. Cars passing each other at break-neck speeds with safety margins that are only really safe with positronic brains at the wheel. We see the street, as seen by the computer. Bounding boxes and labels hovering over multiple viewpoint video feeds, all drawn in slick black & blue cyber aesthetic. Suddenly something changes at the left of the image, alerts go off. Cut to the exterior, super slow motion shot of the care from back right angle, with a big billboard visible on the left side of the road. It shows cryptic patterns that seem to depict running children, but never the whole child, just portions of them, in changing shapes and positions, with neural network adversarial noise covering the rest of the image. Cut back to the board computer's view, still slow motion. Red bounding boxes surrounding the billboard, and then a big red warning flashes in the center of the screen: COLLISION IMMINENT. All kinds of meters at the bottom of the dashboard start going wild as the car tries to find a safe exit state, and with the slow-motion slowly approaching realtime again. It swerves to the right. The image gets rocked as it gets hit by another passing car. Another hit. The world on screen turns as the car goes flying. Static noise fills the screen. Next scene: A news broadcast announces that there was a high casualty in accident on Suchandsuch Road last night. Investigators find that it was most probably caused by a repair shop installing a faulty aftermarket AI module into a car.
  • Unklejoe 6y ago
    I have to admit, that was captivating. The part about cryptic images that resemble children gave me the chills for some reason.
  • Dork_Sider 6y ago
    I mean you've basically described the opening to Upload on Amazon.
    • 6y ago
• GhostVII 6y ago
  I always find these things over hyped. Like yes, you can trick an autonomous car into stopping or crashing. But you can also trick a human into stopping or crashing by running into traffic with a construction hat and stop sign, or dropping rocks from an overpass. The reality is that road safety relies on the vast majority of people agreeing to not kill each other.