14 ms·
Watch Zoox’s autonomous car drive around San Francisco for an hour
- xiaolingxiao 6y agoWhat is the general view on Zoox's progress relative to other non-waymo playes. Such as Argo, Aurora and Cruise. There is the widely reported disengagement per mile, but most robotics people know it is just smoke and mirrors meant to make the regulators go away (disclosure, studied/researched robotics in grad school).
- IntenseChaos 6y agoThe general consensus among my AV friends (who work at a bunch of different companies) is that their AV driving stack is really good, but obviously not perfect. I have no idea about their business model and how COVID affects that, though.
- Fricken 6y agoRelative to competitors Zoox's automous OS is doing quite well and doesn't get enough respect. Relative to the objective everybody is fucked.
- xiaolingxiao 6y agoCould you provide more context on the first part.
- Fricken 6y agoThey've been keeping abreast. The co-founder, Tim Kentley Klay was somehow able to get Jesse Levinson on board, and Jesse Levinson had no problem getting infinite street cred on board. So they were able to attract a lot of key, original robotics talent before the hype got out of control. For a long time though, they were low on funds, so they did lots of closed course testing, and it wasn't until they closed a large funding round that Zoox began on public roads, and they performed quite well right out of the starting gate. Now Zoox and it's competitors are lost in an endless wasteland of testing, development, and validation. It's futile to attempt to do a comprehensive analysis between the different players, they all have their quirks, but Zoox has built all the critical infrastructure needed to do full scale testing, and they're eyeballs deep in it like everyone else. However, Zoox has stormy waters ahead financially. They need another $2 billion to stay abreast in this never ending race. It's getting harder to visualize scenarios where that happens. What nobody can do well enough to build a competitive and scalable robotaxi service is prediction in multi-agent scenarios. The AI for that just doesn't exist.
- matdehaast 6y agoThanks for the info. Can you clarify what predictions in multi agent scenarios are?
- ragebol 6y agoI think was is meant is: in case you have a few people and AVs in an interaction, predict who's going to do what in order to best anticipate the overall outcome. Not sure humans can do that outside of conversation and norms and rules.
- Fricken 6y agoImagine an uncontrolled intersection. The Robotaxi is approaching from one direction. In the opposite direction is a cyclist who intends to turn left across the Robotaxi. There is also a pedestrian that may or may not cross the street, and another vehicle about to cross in front of the Robotaxi from the other direction. There are a huge number of ways this scenario can play out, and any decision made by one agent can affect the behaviors of all the others, compounding it's complexity. Humans can game out these situations intuitively, but current AI cannot read deep enough into the matrix to deal with these situations quickly and reliably.
- BubRoss 6y agoThere are rules to that situation. You can start there, let other cars go ahead of you if they break the rules, go slow and not hit anything. It isn't as if self driving cars can't look to the side or stop if something changes. Beyond that Jim Keller would say that not getting hit by something else is a matter of ballistics.
- TheSmiddy 6y agoIn Australia there are no uncontrolled intersections (that I am aware of). Every single junction clearly marks who must give way and we don't have any 4-way stops, instead using roundabouts in these situations. It's possible that for self driving to work road systems will have to be more formalised to remove the ambiguous situations you've described. I can't imagine it working well in China or Indonesia where traffic flows much more like water in a stream and lanes are merely just suggestions.
- Gaelan 6y ago> There is the widely reported disengagement per mile, but most robotics people know it is just smoke and mirrors meant to make the regulators go away Are you saying that the numbers are inaccurately reported, or accurately reported but just don't tell the whole story?
- thenewwazoo 6y agoEach company gets to decide for itself what qualifies as a disengagement and each event’s severity, and the formula used is a VERY closely guarded secret.
- dehrmann 6y agoWatching this, it's so frustrating that we're 95-99% there on autonomous driving.
- phkahler 6y agoRemember that we've been at that level with voice recognition since the end of the last millennium.
- dnautics 6y agoNo, voice recognition got markedly better in 2015-2016. Now, I regularly to tasks with vr, which was not really a thing in 1999.
- vladislav 6y agomore like 0.1% there in terms of the work required to launch
- Fezzik 6y agoPossibly for driving in cities and highways on clear days, but we are nowhere close to having autonomous vehicles even match human drivers in 100% of possible/likely driving circumstances and road/weather conditions. That last few percent is the highest hurdle.
- candu 6y agoThat's the good old Pareto principle for you: the last few percent are going to take a lot more effort than the first 95%. More to the point, this falls into the category of safety-critical systems, with the added wrinkle of potentially being used daily by millions of people. Unlike many domains where software is applied, 80% of the way there doesn't cut it, nor does 95% or 99% or even 99.9%. (Leaving aside the fact that, for all of us not actively engaged in autonomous vehicle R&D, we likely have absolutely no idea how close we are to success here, or even what all the relevant goalposts would be.)
- stefan_ 6y agoCasually starting the turn and not yielding to pedestrians at 10:21. Companies actually put this kind of footage up without ever reviewing it?
- chrischen 6y agoI mean it's not 100% of the way there. Plus human drivers do that all the time and MUCH worse things. I'm talking from the point of view as a frequent Uber/Lyft passenger.
- dmitriid 6y agoCasually starting a turn and correctly identifying that the person on the right slowed down, stopped and turned to face the other crosswalk. You can see it on the top right camera.
- stefan_ 6y agoI take it the ped at 12:22 also "turned"?
- dmitriid 6y agoThis one looks like a regular situation handled like a regular driver would handle it: the person was a reasonably long distance away from the intersection and by the time the car got to the crosswalk it was probably going at a high enough speed when suddenly braking would cause problems. But yeah, it does look as something to re-analyse.
- hrishid 6y agoAre you referring to the pedestrian who's almost crossed the crosswalk on the left side of the screen? This is still a proper yield as far as I can see. The car just enters the intersection before that person has finished crossing.
- stefan_ 6y agoThat's already the problem. Don't enter the intersection if you can not speedily finish your turn. There is also already another ped on collision course the moment they start moving forward.
- vladislav 6y agoThis demo is not informative as to the readiness for scalable L4 deployment, for which it would be necessary to focus on the breadth/accuracy of perception features under the hood of intent prediction and what happens at the tail end with arbitrary situations that occur in urban driving environments.
- anonymous_car 6y agoThis demo is not informative as to the readiness for scalable L4 deployment does anyone make that claim?
- vladislav 6y agoPresenting a subset of the information to let the uninformed jump to favorable conclusions for the presenter is not a new marketing strategy. If there's no indication about the true level of progress, what is the purpose of the demo?
- chrisseaton 6y ago> to let the uninformed The uniformed don't know what 'scalable L4 deployment' is, so they can't jump to that conclusion.
- yellowapple 6y agoNo, but they're familiar with the definition (start-to-finish entirely autonomous trip under somewhat-controlled driving conditions) even if they don't necessarily know the lingo to describe it. Being able to get from point A to point B without human intervention is what people expect when they hear "self-driving car", and the video does little (if anything) to temper that expectation (perhaps because it truly is ready for L4 deployment, or perhaps because it's all smoke and mirrors).
- chrisseaton 6y ago> under somewhat-controlled driving conditions I don't even know what this means, so I doubt the uninformed know the definition.
- databus 6y agoIs there a way to know that this isn’t done with remote control, other than the company says so?
- deleted 6y ago[deleted]
- anonymous_car 6y agoArgument from ignorance also known as appeal to ignorance, is a fallacy in informal logic. It asserts that a proposition is true because it has not yet been proven false or a proposition is false because it has not yet been proven true. https://en.wikipedia.org/wiki/Argument_from_ignorance https://en.wikipedia.org/wiki/Argument_from_ignorance
- chrisseaton 6y agoIf you think people are just going to simply lie to you then how do you ever get anything out of reading things on the internet?
- yellowapple 6y agoBy getting multiple opinions, like what the GP is presumably doing by asking such a question on a forum like Hacker News.
- dmitriid 6y agoBesides the sheer complexity of situations described in this video, I wonder how these vehicles will deal with differences in traffic rules in different countries (when even road signs can be different).
- yellowapple 6y agoIt sounds like it currently "cheats" a bit by already having driving rules, maps (including signs), etc. baked in; it'd be akin to a human driver memorizing the California Vehicle Code and a map of San Francisco word-for-word and lane-for-lane. Presumably Zoox deployments in other cities would work similarly, "cheating" by baking in local driving rules and road maps. A consumer-owned self-driving car would likely be able to do something similar by downloading the local ruleset and maps on the fly, assuming one exists.
- deleted 6y ago[deleted]
- netsharc 6y agoCheap criticism: the video starts with (I paraphrase) "This is 1 hour of driving", the last thing I expected after the fade-out/in was to see a man with a weird shirt... and then I notice the video is about 27 minutes long. Edit to add: After that I started watching it, it's actually a video of an impressive AI.
- ygra 6y agoIt's played back at twice the speed. Apparent from pedestrians waking twice as fast, and if course, the 2x speed indicator in the upper left.
- vardump 6y agoYawn. Good lane markings, no rain/snow or other bad weather, perfect road surfaces. Just like all other self-driving demos. I'd like to see a demo like this on snow covered roads, with no lane markings visible. I think that would tell a lot more about the system's ability to deal with an imperfect world.
- spacehome 6y agoI’m pretty sure I can’t meet that standard.
- vardump 6y agoActually I'm sure you'd do just fine. Humans can just pretty effortlessly deduce required information out of numerous clues.
- chrisseaton 6y ago> I'd like to see a demo like this on snow covered roads, with no lane markings visible. But humans can't drive well in those situations either. Why are you asking for something better than humans can do?
- vardump 6y agoHumans can and regularly do so pretty safely. Ask Canadians, Swedes or any other people living in a location with long winters.
- draugadrotten 6y agoI was once driving on a road I could not see at all. It was at night, in a blizzard on the road from Denver to Vail. It didn't take long until I was following the two red lights of the bus in front of me. As a human, I knew I could drive safely where the bus had been driving seconds ago. A self-driving car would have... tell me.
- rossjudson 6y ago
- yellowapple 6y agoThe two turns (one left and one right-on-red) leading up to getting to Market Street in the latter half of the video struck me as odd; the left turn looked like a bit of a lane sweep, and the right-on-red looked dubious (is it legal to turn right on red if you're not in the far-right lane?). SF intersections are hard, though, and the computer seemed to handle them about as well as I would've.
- lolc 6y agoOne thing they only mentioned casually towards the end is that they mapped the city beforehand. So the car is starting from a position where it knows all the intersections.
- m0zg 6y agoSuch videos don't get published just because. They're either looking for more funding or for an acquisition. Which is it?
- vladislav 6y agobeggars can't be choosers
- jasonv 6y agoI see these Highlanders (I drive the same model) parked in their lots in FiDi almost every day. Glad to see what they’re up to.
- NightlyDev 6y agoThis is really cool, but the environment is also really simple and I think we're definetly at least 15+ years out before self driving cars can handle somewhat challenging situations as well as humans. Just try to put one of this vehicles in a situation with varying road width, no markings, snow with no sticks to mark the edges so you really have to pay attention to where the road actually is. What would this do if you meet a car on such a road? Try to figure out who should go back, and maybe go back to the latest plase where its wide enough? Do random tests to check for grip every now and then? It also needs to know whether the road is salted, understand if the salt is working and so on and on and on...
- vladislav 6y ago"we're definetly at least 15+ years out". Similar statements were made about Go the year it was solved. AV is a vastly harder problem and requires new techniques to get there, but AI can progress can happen any time.
- kevin_thibedeau 6y ago> Handling yellow lights properly, involves us having to predict how long they will remain yellow for No. That isn't how yellow lights work in the US. If the light turns yellow and you have enough space/time to make a safe stop you do it. There's no need to predict the remaining time on yellow phase. We don't need robot cars bending these rules.
- travisporter 6y agoNot sure why you're being downvoted, but I think this is a classic example of why self-driving is so hard. They're not bending the rules, just copying what humans do. We also predict how long a light will be yellow for, but do it naturally (if you just saw it turn from green, or it was yellow as soon as it was in your line of sight).
- rrdharan 6y agoIn Delaware on Route 1 if you follow this advice you are likely to get rear ended. They have traffic lights on a 50mph route that stay in yellow for a long time. I often find myself slowing down to a stop then awkwardly realizing I’m stopped with multiple seconds of yellow remaining and drivers honking behind me. Maybe my brakes (or reflexes) are just too good?
- gpm 6y agoNo. That's what the law says but not how you drive. Suppose your 4 seconds from a yellow light traveling at a high speed. You can slam on your breaks and make a very abrupt stop, or you can cruise through that light and continue on your way. If the light is about to turn red you should probably slam on your breaks, because you risk being t-boned in the intersection. If you have time to get through the yellow light/before the cross light turns green, you should keep going because slamming on your breaks is mildly dangerous. The law isn't nuanced enough to understand this, with good reason. You don't want to make a bad call about the safest action made in good faith illegal.
- carapace 6y agoIt really really bothers me that these folks are using a live city with real, non-volunteer test subjects of all ages (little kids and old folks use public streets) as a test bed for their massive car-shaped robots. It's bad enough that people are driving cars all over the place, car collisions have killed more Americans than all the wars we've fought put together. I'm one of those people who say, "Self-driving cars can't happen soon enough." But I don't think that justifies e.g. killing Elaine Herzberg. Ask yourself this, why start with cars? Why not make a self-driving golf cart? Make it out of nerf (soft foam) and program it to never go so fast that it can't brake in time to prevent collision. Testing these heavy, fast, buggy robots in crowds of people is extremely irresponsible.
- torpfactory 6y agoThere is a different perspective that you could use (and I’m not necessarily advocating for it; hear me out): Human driven cars are dangerous to the tune of ~36,000 deaths per year. Every year without the implementation of full self driving we pay some large percentage of that number in lives. Self driving cars won’t make it out of the lab without real driving on real roads in real scenarios. Taking appropriate precautions (a human safety driver, maybe two) and testing in the real world might save more lives overall than keeping the vehicles in a more lab-like setting for longer, and missing some of the complexity of the real thing.
- carapace 6y agoWhat about the nerf-golf-cart idea?
- pj_mukh 6y agoMost of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation. In the end, the only test that matters is the real-test. Real speed (sensor/computation latency et al.), real sensor feeds (lighting et al.) and real car size (momentum et al.). Source: Am Controls engineer. There WILL be bugs and un-modellable sources of error. The real hedging in these situations is the safety driver. The death of Elaine Herzberg is very regrettable, but the fault ultimately lies with the safety driver and the training that was offered to her. She was on her phone, like 1000's of drivers are now.
- Animats 6y agoInteresting. Watch at 1/3 speed or so to see it in real time. (Self-driving car videos tend to be published sped up, so you don't see the mistakes.) The key part of this is, how well does it box everything in the environment? That's the first level of data reduction and the one that determines whether the vehicle hits things. It's doing OK. It's not perfect; it often misses short objects, such as dogs, backpacks on the sidewalk, and once a small child in a group about to cross a street. Fireplugs seem to be misclassified as people frequently. Fixed obstacles are represented as many rectangular blocks, which is fine, and it doesn't seem to be missing important ones. No potholes seen; not clear how well it profiles the pavement. This part of the system is mostly LIDAR and geometry, with a bit of classifier. Again, this is the part of the system essential to not hitting stuff. This is a reasonable approach. Looks like Google's video from 2017. It's way better than the "dump the video into a neural net and get out steering commands" approach, or the "lane following plus anti-rear-ending, and pretend it's self driving" approach, or the 2D view plane boxing seen from some of the early systems. Predicting what other road users are going to do is the next step. Once you have the world boxed, you're working with a manageable amount of data. A lot of what happens is still determined by geometry. Can a bike fit in that space? Can the car that's backing up get into the parking space without being obstructed by our vehicle? Those are geometry questions. Only after that does guessing about human intent really become an issue.
- emmelaich 6y ago> Watch at 1/3 speed or so to see it in real time... The note in the top-right says its 2x.
- harshulpandav 6y agoTop-left
- baby 6y agoThis is super cool! I'm wondering how the car would react if: * someone parked on the side opens their door too quickly and collide with the zoox car. * there is a car not moving in front, and the zoox car cannot see what's in the other lane without backing to get more insight I'm also super impressed at how it can understand where the lane is in this 5 lane intersection that crosses a tram line. Even I couldn't understand where I would have had to drive!
- wutbrodo 6y ago>I'm also super impressed at how it can understand where the lane is in this 5 lane intersection that crosses a tram line. Even I couldn't understand where I would have had to drive! This is actually one of those things that's easier for an AV than a human since they have localization and full lane maps of the city.
- Krasnol 6y agoI really appreciate the calm background music. I think background music is important. Especially on such long explanatory videos but often it becomes a reason for me to turn off a video if the music becomes to aggressive.
- lolc 6y agoAll in all I'm quite impressed with the demonstration. It was way more thorough than previous videos I've seen. The main things the car is failing at from what I see are the hard things: Object permanence and ad-hoc reasoning. So no surprises. Regarding object permanence: I was impressed overall with their detection. Still, you could see kids walking close to parents blink in and out of awareness of the car. Now I'm not saying humans are very good at tracking a multitude of actors. So at some point the machines will be "good enough". But that point seems way off when significant objects like kids can just disappear from awareness when they pass behind a stroller. And about the ad-hoc reasoning: They have the whole city mapped out! Including traffic lights and turn restrictions. I'm not even clear whether they try to detect the signs at all. I'd assume that they have an operations center that hot-patches the map with everything cropping up during the day. So the cars would send in unexpected changes to the road and they would classify those changes and patch the map. Meaning the car is tethered to that feed and not autonomous in the strictest sense. Sure, such a center would be marginal cost given a large enough fleet. Still it's a subscription you'd need for your own robocar. They mention a lot of things they are prepared for. And I can't help but think "oh they're really good" when they say "detect backed up lanes" or "creep into intersections". But that always leaves the question what happens when they're not prepared for something. When the rules don't fit. Can the car go over a curb if the situation warrants it? Does it back out of a blocked off section? Is it even able to weigh whether backing out is an option at this point? so I'd like to see a "what we're currently stuck at" video. But I understand one can't very well attract investors with such a video.
- idavidrein 6y agoI agree with a significant amount of your point, but with regard to object permanence, I would guess that they have prediction algorithms that don't only rely on the current-time perception, so if something blips out of sight for a second the system will still infer/predict it's existence (for a time - obviously if something is hidden for a long time it won't continue to not trust perception).
- lolc 6y agoI'd be very interested to know how that works. But I don't think they have it. The boxes they draw are very wobbly and dimensions expand and contract directly with sensor input. Maybe they only show fused output (in itself an achievement) and there is a later step they don't show. That would be weird though because if they want to brag about their model they would definitely show it if it was any good.