11 ms·
> Pilots are getting far too comfortable flying what the computer is telling them instead of flying the actual aircraft.
I'm not a pilot but this stuff does co
> Pilots are getting far too comfortable flying what the computer is telling them instead of flying the actual aircraft.
I'm not a pilot but this stuff does concern me.
If you Google around for GPS related accidents with cars the results are pretty mind boggling. There's a number of cases of folks driving straight into a lake because GPS told them to go that way. This plane incident reminds me of that.
I would hope I would not drive into a lake or any other obvious hazard, but I have driven the wrong way onto a one-way portion of a street when I was following Google Maps.
My gf once asked me why I wouldn't just park on the street, it was night, it was Delft (Netherlands), it was very still standing water in one of the canals with leaves on top.
It did look surprisingly like a street, but I knew I was in Delft.
that's just very bad design keeping water at same level with street, I was under impression channels in Netherlands are always below ground level for many obvious reasons
It wasn't designed that way. In other locations in the city, street level is at least 150 cm above the water. But because of natural subsidence, ground water extraction, construction and peat oxidation, some parts of city's peat soil have subsided in the last few decades to only 25 cm above the water.
https://www.delft.com/routes/delft-waterways-and-the-compartmentable-city-centre-canal-system/points/5 https://www.delft.com/routes/delft-waterways-and-the-compart...
Sometimes the water decides to do that on its own.
I've ridden through Chiang Mai after a heavy rain when the streets were all a foot deep, so you'd get a nice foot wash riding along and despair for your flip flops when stopped.
And if you strayed too far from the car in front or cut a corner, you'd find the 4 foot deep drainage ditches and 12 foot deep canals along the sides of the main roads.
This reminds me of a story when I was a kid.
We used to have a big pond in the frontyard. One day it was completely covered in duckweed (kroos in dutch) and one of my friends just walked straight into it, not aware of the water underneath.
Glad you didn’t try that out with your car!
This is exactly why the mass delusion propagated by Elon that Tesla's have full self driving is terrifying, and why "self driving" in general is a sinister lie that endangers the public.
The premise behind self driving cars that can handle most situations themselves relies on the assumption that a human operator is paying attention and can take over in situations the computer can't handle. I'm not a pilot, but it's likely that assumption can hold while flying at cruising altitude, but not so much when driving in various inconsistent road and traffic conditions.
>human operator is paying attention
Think of how silly this is though. You offload driving to a car, then...demand passenger give absolute attention every second that car operates itself. For one, this is a pointless tool if you have to monitor it all the time. For another, it is impossible that humans will be able to focus their attentions fully when they aren't really doing anything to drive the car. The better self-driving gets (say, safe and effective 99.999% or any incremental level higher), the less the human monitor will actually be watching. Thus the more likely any error will be catastrophic.
I've just gone from a 2004 Corolla to a 2020 Serrato. It's a low end car but it has lane assist, braking detection when a car slows in front, cruise, speed limiter, etc. Combine all those settings with google maps and it easy for me to see how you could get complacent driving a new car and make mistakes.
All these features work well most of the time, but they all notably fail on, still frequent, edge cases.
Using a Hyundai hybrid and it got all these. It helped quite a lot as long as you consider yourselves a driver and in control.
Still it is very weird when the car turn the wheel for you with your hand being guided.
No hand up display and not sure if just voice to tell me to drive to lake would I do that.
Yes and as with Tesla "FSD", supposedly the driver is to devote full attention to monitoring the car whenever activated. But this is silly because as these tools get better, the driver will inevitably monitor less. But it won't any time soon (ever?) avoid random errors every now and then. But now these will be catastrophic failures.
The phenomenon is called Death by GPS. Explained very well in this article : https://arstechnica.com/cars/2016/05/death-by-gps/ https://arstechnica.com/cars/2016/05/death-by-gps/
Not to mention the GPS doesn't do half the things I want it to do.
"OK Google zoom in the map by 2X"
nope
"OK Google don't hide all the businesses while on navigation mode"
nope
"OK Google show all street labels don't hide them from me"
nope
"OK Google rotate the map to the right 30 degrees for 5 seconds"
nope
"OK Google show me satellite maps instead"
nope
"OK Google pause notifications from Facebook while driving"
nope
... I bet most people have to fiddle with their screens due to OK Google's utter incompetence.
Meanwhile the people on the OK Google team are busy figuring out how to arrange a damn haircut which I never needed.
Last time I tried, I couldn't even figure out how to exit navigation using voice. I'm sure it's possible, even easy, but the combination of not knowing the right incantation, car noise, language issues (does it expect German or English commands?) and overall stress meant I just ended up muting the sound system, which is a physical button.
That's brilliant, and so funny because it's true.
I've been wondering why we haven't really had any advances in our voice commands for nearly a decade - things like zooming and such are actions I've long wanted to be able to perform.
Even that rotate map.. when your car is shown at the bottom and map is hard locked to current heading, when navigation is about to do a sharp right you end up with zero visibility of the rest of the route as they still haven't bothered doing a projection and modifying rotation dynamically - they're not even trying.
All your examples are exactly the kinds of things that an intelligent system should be able to respond to, in order to be useful.
Getting that haircut ordered is the _last_ thing we need.
It doesn’t follow that errors will be catestrophic. It could be the car just fails to handle edge case situations and just stops… granted on a highway that’s bad but everywhere else, that’s a hassle to traffic at worst. Also highway full self driving is much closer to being a solved problem.
Even the failures you see in the beta of Tesla FSD, aren’t catestrophic. The worst I’ve seen is a curb strikes in terms of actual “accidents”
What you are describing is level 4. And if tesla were anywhere near that that would be freaking brilliant.
Tesla FSD has many deaths and countless serious accidents under it's belt. Which doesn't necessarily have to be that bad, but as expected a computer doesn't make the same kind of mistakes that a human does and the kind of mistakes that tesla does doesn't inspire confidence (but rather nightmare fuel that you just can't wake up from).
They should be banned until they have level 4 and take responsibility for their cars actions. Blaming it on a human driver and calling it a day is ludicrous.
> a computer doesn't make the same kind of mistakes that a human does
I think this is particularly problematic because with other humans, we can at least develop a sense of what types of mistakes they tend to make and try to adjust our driving accordingly. Things like not driving in another person's blind-spot, etc. but also more subtle stuff. I feel like there a sort of hive-mind that develops amongst drivers and probably makes things safer than they would otherwise be.
Sorry. I haven’t seen any evidence of anything you said.
There have been 0 deaths from FSD and as far as I’m aware only 1 crash. Do you mean autopilot? Or navigate on auto pilot?
I just looked up a random video of someone testing FSD 10.8 and in the first few minutes, it failed to signal before changing lanes and it almost caused an accident. All the while the driver is saying how good it is and how comfortable he is.. The car drives not smoothly at all and does random braking when around pedestrians.
The car also features a yoke instead of a wheel. Which makes holding the yoke while autopilot is driving almost impossible.
Autonomous driving is a mess.
I almost crashed into a tesla on the highway when I changed lane right behind a Tesla. The Tesla was going much faster than me, and when I changed to its lane a bit too close, it started breaking for no reason, forcing me to jump on the brakes as well. I do not believe it was the driver, but the automatic brake assist that did not like me coming in too close…
I think that is not really related. Following a “guidance system” that’s using gps (and altitude in the planes case) is very different from having a neural net interpret the surroundings visually and make decisions based on that.
I don’t know where/why you are attributing sinister motives here… it seems a very understandable motive to make transport better.
After working in the industry, I can safely say self-driving is a massively profitable lie. It can work in limited instances, but I don't think we'll have full self driving aka level 5 in urban environments until infrastructure changes. You simply need 5G/V2X for that. Just relying on cameras will not work. On that page, I have my concerns about EVs in general, who knows what the long-term health effects are for sitting on top of a HV battery and being surrounded by that electro-smog.
> On that page, I have my concerns about EVs in general, who knows what the long-term health effects are for sitting on top of a HV battery and being surrounded by that electro-smog.
Say what? Is this some kind of industry joke that I'm not getting?
> Just relying on cameras will not work.
If it’s good enough for us (two small cameras and two microphones all mounted right next to each other) why is it inherently infeasible for a computer?
Those cameras are backed by processing that's so far ahead of anything silicon can do it's not even funny. It's not inherently infeasible, but it seems computers will need a huge leap in certain capabilities before they will be able to act as a drop-in replacement for human drivers.
I find that argument being thrown around once in a while, but let me ask you this. Why are we spending billions of dollars in research that we can vaguely hope to reach human perception levels in one distant day. This whole thing is just not worth it. That's why I said especially for dense urban traffic driving, we need extra sensing, and need from the environment so it can help the car. Imagine if the car knew there is nothing coming from a junction we don't need to stop all the way etc.
I think there is more senses being used in driving then just eyes and ears. I am not the best driver in the world, but I use a lot of senses while driving, abnormal vibrations that are felt cause me to be more alert. Constant awareness of hills and road conditions are felt.
Saying it's just cameras and microphones vastly over simplifies what driving involves for a human.
who knows what the long-term health effects are for sitting on top of a HV battery and being surrounded by that electro-smog.
A fellow named Planck knew, when he used the expression E=hv to describe the difference between ionizing and non-ionizing radiation. You just missed class that day, I imagine. Fortunately it's never too late to learn.
> If a car needs monitoring, and they will probably need it for a very long time, you should as well drive it.
This is the crux of the intractable problem of AI/self-driving marketing vaporware, and why until this nonsense is shut down, the situation will probably have to get scandalously worse before it gets better (better meaning regulating fake "self driving" off the road as a public safety travesty).
> But this is silly because as these tools get better, the driver will inevitably monitor less.
And for good reason. The main benefit to FSD, as I see it, is the ability to do something more productive with my time while getting from A to B. Most wealthy people end up paying for drivers not because they want an exceptional driver, but because once you reach a certain threshold of income it is more profitable to work (phone calls, laptop, w/e) or sleep, and pay to be driven everywhere than to drive yourself. FSD lowers that income threshold tremendously and could be a significant boon to overall human productivity. If it ever happens.
>is the ability to do something more productive with my time
Seems to miss my point, I may have not stated clearly. Yes I understand the sales pitch of self driving cars stated above...the "marketing" / "hype" of AI/self driving cars/ Tesla FSD evokes the idea of completely offloading the responsibility to the car and doing whatever you want with your time during commutes, as if it was public transit.
But my point was anything less than this is a complete waste of time and useless as a product because until you are completely hands and attention off, you have to be completely hands and attention on/ready...no matter how good or near actual FSD it gets.
Because 99.9, or 99.9999, or 99.999999% effective is effectively 0 percent effective given the catastrophic outcome of any error if you aren't devoting full attention to monitoring the car.
But aside from all of this theorizing, the actual product is NOWHERE near high 9's reliable. As it gets better, attention will drop but errors will still be pervasive. And with this as backdrop, the fact the marketing has ever even hinted at being general-AI type FSD (the name itself for example) is a scandal. The videos of people riding in the backseat, or any other instance of not monitoring the car fully because they think it can drive itself. I predict there's going to a be an endless stream of accidents from these cars then FSD will be shut down until fully proven, which won't be technically possible for a long long time if ever in our lifetimes.
> But my point was anything less than this is a complete waste of time and useless as a product because until you are completely hands and attention off, you have to be completely hands and attention on/ready...no matter how good or near actual FSD it gets.
Agreed. If I got a Telsa tomorrow I wouldn't do so with "FSD".
> Because 99.9, or 99.9999, or 99.999999% effective is effectively 0 percent effective given the catastrophic outcome of any error if you aren't devoting full attention to monitoring the car.
I'm not sure what effectiveness you're measuring. There will always be fatal accidents with heavy vehicles moving at high speeds. It's more of a regulatory issue where the accident rate of auto-pilot systems is sufficiently lower than human drivers (because I believe it is already lower in many situations, just not sufficiently lower or ubiquitously lower) and we decide its good enough to no longer require monitoring by the driver.
Edit: I guess its even more complicated than that. Using hypothetical numbers, if the accident-free rate of a human is 99%, and auto pilot is better at 99.9%, but combined (auto-pilot with human monitoring and rare intervention) is 99.999%, can we justify allowing no monitoring in that situation? Can we expect people to follow the law or will it turn in to more of a speed-limit type grey area.
Not all errors or accidents are created equal. As someone else mentioned in this thread the nature of human error is qualitatively different than the nature of seemingly random errors we see with self-driving. Code bugs can produce any scale of error. Humans do no in any given situation have open-ended likelihood of making any errors.
Search youtube videos of Teslas if you haven't seen them yet. The cars will randomly swerve into parked cars, inexplicably steer out of turns, aim directly at medians on the highway. Human errors on average do not equate to this if we want to compare human error "rates" to machine. What is the rate of a random catastrophic error? Machines are way way too high for the foreseeable future, and because of the nature of software bugs they arguably always be higher than humans. The scope of what AI has to be to achieve "general" driving on open-map roads with near infinite variables nearly equates to "general" AI itself.
You can't allow "halfway" monitoring the car. It's all or nothing. It's always monitored or it's not monitored because errors can happen at any time. But anyways, humans aren't robots. Their attention basically not focused if it's not focused completely... multi-tasking is false, and humans already fall asleep fairly easily driving manually. If they have to sit there "focused" but not driving this will lead to even more instances of sleeping or just not monitoring generally.
The numbers are a thought experiment what I mean by effectiveness or mistake or error is just the driver (man or machine) doing something they shouldn't have. Messing up. Humans have very low rates of these mistakes/bad actions/errors. They have very very low rates of catastrophic mistakes. I predict Tesla's won't be 99.anything for a long time. But think of how error-free they have to be to account for the huge domain of roads, signs, weather, terrains, other cars such that when they make their mistakes it doesn't result in disaster... or just "accidents" galore.
What is a Serrato? I've not heard of this kind of car. If it's a typo or something, apologies, I don't know what it could be. I'm not trying to be snarky, just genuinely curious what it is?
Woops yes a typo on a Kia Cerato. At least that is what it is called here in Australia.
https://www.kia.com/au/cars/cerato/features.html https://www.kia.com/au/cars/cerato/features.html
My guess is a Kia Sorento
Edit: Ah OP replied. It’s the Kia Forte in the US.
> Combine all those settings with google maps and it easy for me to see how you could get complacent driving a new car and make mistakes.
This along with the fact that in my 2020 Toyota Corolla those features can more than likely get you into an accident is the reason I have them turned off. I don't actually feel safe with them on after experiencing their performance. Also notice a stutter in acceleration due to fuel injection timing and response from the gas pedal that concerns me.
Because it takes a dedicated team reviewing commands all day, every day, and dev team implementing event handlers, coordinating with other product teams. It takes lots of dedication.
I used to run a chatbot that screened people. We thought we were asking simple enough questions, and yet two million chats in, we still discovered at least a Dozen unexpected/misdetected intents that required care. A day. And here is the problem: management will ask: "so if we add handling for 'rotate the map 30 deg', how many people will appreciate it?". The answer is usually "one person every 400k sessions". For an MbA trained manager that means there is no reason to do it. And its the same with every other similar discovery.
Running an NLP based tool requires staffing a 10-20 people full time, telling them to keep the quality high and leaving them the fck alone. Thats anathema for most companies that proclaim to be "metric based". Google is a prime example of this behavior.
> The answer is usually "one person every 400k sessions".
But you control that to some extent. If more people are aware of the command and believe it will work (I wouldn't) then you'll probably get higher incidence rates. I wouldn't believe it would work because like the person two parent comments above, less than half the novel crap I try works. So I try novel crap less and less frequently compared to things that I know will work.
Would you like to see a map of the nearby roads with no lables on it? I fyou zoom in to the point where a road is half the width of the screen I'll deign to add a label on one of the little side roads you don't care about.
Never will I ever tell you what the arterial route you're currently on is called though. Sucker.
I just disabled Google assistant and all of its functions, it's beyond useless and probably the most frustrating thing I have ever used on my phone. And I don't mean just the voice controls, but all the "helpful" tips and suggestions are just bad and I can't stand them.
siri's worse than google's voice system... Siri understands what I'm saying, and then CHANGES the correct words into something stupid.
I just risk a fine and use my hands for a short moment instead of spending 2 minutes being distracted by siri.
OK Google is awful for me, it repeats back exactly what I said but rarely interprets it, at all. "Don't know how to help with 'navigate home'", "'send a message'", "'find a supermarket'". Bleurgh. This is via Android Auto. Next minute it's reading me a text and asking if I want to reply by "SMS or WhatsApp".
None of these systems show you enough 'under the hood' to know why it's failing and so react. Clearly, here the voice recognition, tts are working well; some optional visual feedback would be great - like show on screen a list of responses:
- Keyword heard
- OK Google responding
- Voice1 recognised
- Input "navigate home"
- Input sent to maps
- Maps didn't understand
- Reporting failure
I guess Google know all this, at the time of the query, but as they don't know what I'm trying to do (though 'navigate home' is pretty clear, perhaps I was actually saying 'buy a gnome') ... doesn't seem like there's a proper feedback loop to help direct/correct the system?
When driving I don't bother to use Siri for anything other than "call <person>" or "text <person>".
The most hilarious example of how bad Siri is for me, is that literally 100% of the time that I'm playing something on Apple TV, if I say "Hey Siri, pause the TV" (or any variation, calling it Apple TV, including the name of the room, etc.) it always replies that there's nothing currently playing. And if I tell Siri on my phone to play on the Apple TV, all it can do is start Apple Music playing a random song on the TV. It can however respond OK to "resume on Apple TV" which restarts whatever I was watching that had been paused.
The sole thing I use it for is setting alarms/timers, or starting the process of a calendar appointment. Those are the two use-cases I've found useful.
Say "navigate to X" also can kinda work with less fumbling than typing half the time but it's a bit messy.
It's not just Google Maps though. Also Apple Maps, Music, Podcasts... I find it so frustrating to use voice commands on my phone in the car that I mostly just don't even try. I set things up before I head off as much as I can and mostly stop somewhere before I make changes.
> I have never seen a single failure where it full speeds into another object.
What do you make of this near-miss posted recently? Allegedly, FSD was on when the car veered into the lane of an oncoming car.
https://www.reddit.com/r/SelfDrivingCars/comments/re7vxr/fsd_beta_fail_reposting_video_that_was_taken_down/ https://www.reddit.com/r/SelfDrivingCars/comments/re7vxr/fsd...
Agree that isn’t good. And definitely looked like it would collide. I guess if you agree to sign up to train the NN you need to accept the fact it’s going to do this stuff.
As an aside, it seems strange to me that the high beams switch Off as they approach the oncoming car. Seems the driver disabled them… auto disabled would have meant they detected the car, which then seems unlikely they’d move into it’s path. But very hard to judge off camera alone… they might not even be on FSD. Shrug.
>it’s not catastrophic… it’s just shitty
This is because drivers are giving 100% attention to monitoring the car in these scenarios. So again, this means the tool is useless (right now it is), or if the driver stops paying attention and becomes an unfocused passenger (the actual point of self-driving) then the tool becomes absurdly deadly.
Everywhere along this spectrum, as attention declines, catastrophic risk goes up. Self driving is vapor ware at best, a scandal at worst. It's not going to end well.
As the sibling post said, lithium fires are a completely different class and beast than a petroleum fire.
Material science improvements won’t overcome the fundamental fact that the third atomic element on the periodic table just really wants to light things on fire.
It's interesting that you talk about using more senses while driving. I always crack the window to listen to the road noise as it helps me guage what the other cars are doing and has helped several times in avoiding collisions with other drivers that weren't paying attention since I could hear the change in noise within the standard noise of the road.