While it points to some interesting new potential surprise issues, I'll admit I'm not a fan of the leadoff Tesla example and that I think it points to a reason

While it points to some interesting new potential surprise issues, I'll admit I'm not a fan of the leadoff Tesla example and that I think it points to a reason that not all of these things will actually be as big a deal as has been hyped in the last year or two. The Tesla one got a fair amount of coverage and discussion on tech sites. But >"and by exploiting flaws in the image recognizer attackers were able to steer a Tesla into the oncoming lane" The fact is though that if someone is interested in maliciously causing people to crash on the roads via IRL actions there are endless ways to do so and always have been. And not just "throw rocks of bridges" either, but things that someone could quite conceivably get similar distance from before the fireworks like spreading various substances/objects on a stretch of road at night. Some fancy new "adversarial patch" ultimately still falls in the same bucket as rocks: a physical effort to cause harm. We mostly deal with that via a mixture of A) law -and more important- B) most people do not in fact want to hurt/kill random fellow humans. Or break into their places and steal their stuff for that matter, for targeted entities dazzle paint or whatever may be another part of cat/mouse, but it's still in the same topological category as shimmying a lock. I think to represent some actual new category, there has to be an arbitrary remote attacker element and/or some level of automatability. So IOT is more worrisome, since the attacker can be on the other side of the world, and attacks can be scripted for mass effect without further human intervention or effort. That fragments typical expectations around social bonds, the extent to which law might respond, cost/benefit ratios for attackers and defenders, etc. "Greater Internet Fuckwad Theory" writ large. But I'm not sure anything that actually requires a real person to do a real physical thing in order to cause harm will ultimately be any more disruptive than so many of the previous new direct physical ways in which people could in principle cause harm. There will be interesting demos, tricks and hobbyist hacks around them, the occasional usage in genuine APT toolkits and activities of statecraft that makes the news. There will be niches that make a selling point of opening the black boxes, others that don't care, the normal back and forth of law. But I'm not sure how much more "demon haunted" it is than things already are: that it takes specialists to understand the true implementations stuff is a bridge humanity crossed, well, with complex bridges let alone other tech. >"I've been saying for years that most people relate to computers and information technology as if they're magic, and to get the machine to accomplish a task they have to perform the specific ritual they've memorized with no understanding." Yeah, how is that different from a lawn mower or dishwasher or microwave or whatever? There is a danger in making such a high level statement and then treating it as profound, without stepping back and realizing it states nothing specific at all. Lots of people have a sort of vague notion of how an ICE works, but how many people really can do the slightest car maintenance beyond windshield or coolant fluid top offs? How many people really understand electric motors and what to do inside of a broken appliance vs just calling the repair person? How many people really understand even the basics of dielectric heating (have they even ever heard of the term in their lives) let alone any other components in there? Or medicines, or enzymatic cleaners, or complex metallurgy we depend on for everything, and on and on. We all constantly work with black boxes in everything because nothing else is possible, no single human has the physical memory capacity to hold even a minuscule fraction of the sum total of human applied knowledge. This isn't some special new thing.