40 ms·

It's clear from the video that a human driver actually would've had more trouble since the pedestrian showed up in the field of view right before the collision,

It's clear from the video that a human driver actually would've had more trouble since the pedestrian showed up in the field of view right before the collision, yet that's in the visible spectrum. That's not at all clear to me. I don't know too much about cameras, but it looks to me like the camera is making the scene appear much darker than it actually is. In the video, you can see many street lights projecting down onto the ground, and the person was walking the the gap between two streetlights. The gap between street lights (and hence the person) was in the field of view of the camera the entire time; they just weren't "visible" in the camera because of the low lighting. I'm confident my eyes are good enough that I would have been able to see this person at night in these lighting conditions. (Whether I could have reacted in time is another question.) It seems to me like the camera just doesn't have the dynamic range needed for driving in these low light conditions, which is a major problem.

netsharc 9y ago
Which makes me wonder if the Uber autocar is just relying on camera vision to drive itself... If it is, and it's been lying to authorities (I don't know what they said to the authorities about their cars' capabilities), that could be big.
- deleted 9y ago
  [deleted]
- jonknee 9y ago
  Far more likely is that this is a standard dash cam that the police know how to pop the SD card out of and access the video...
  - hcknwscommenter 9y ago
    The point is not germane. What is germane is that a car that supposedly uses LIDAR and infrared, and presumably was approved by the regulators on the basis of such, should have had no problem seeing the pedestrian as LIDAR and infrared are unaffected by night and at least shown some indication of braking but did not. This suggests that the car does not in fact utilize any of those fancy (non-vis) detection methods. Alternatively, these fancy detection methods were fooled by the bicycle and thus misclassified as an error or something.
    - jonknee 9y ago
      My point is that it's likely the camera view we're seeing has nothing to do with the self driving portion of the vehicle (a good hint to this is the interior view--useless to autonomous driving, but a common feature of dash cams).
      - tqkxzugoaupvwqr 9y ago
        The car has a LIDAR sensor mounted on the roof. It is supposed to continuously scan 360° of the environment. Since LIDAR is an active sensor (it emits light), the car should have seen the person and bicycle even in the dark. That it did not do so suggests the car does not evaluate LIDAR input, or it dismissed the object as erroneous data.
        cr0sh 9y ago
        This is the LIDAR sensor being used (virtually all SDC companies use this LIDAR because it's the most advanced out there for 360 degree 3D coverage): http://velodynelidar.com/hdl-64e.html http://velodynelidar.com/hdl-64e.html (note that this LIDAR is expensive - costs way more than the car it is mounted to) Here's the manual for it - note the specs in the back: http://www.velodynelidar.com/lidar/products/manual/HDL-64E%20Manual.pdf http://www.velodynelidar.com/lidar/products/manual/HDL-64E%2... It has 64 lasers, spread out over about 27 degrees - about 0.4 degrees per laser, from almost horizontal to an angle of 24 degrees or so down. Now take a look at where it is mounted on the car, and envision these laser beams spreading out and being spun in a circular conical area around the car. Now - if you think about it - as the distance from the sensor increases, the beams are spread further apart. I'd be willing to bet that at about 200 feet or so away from the car, very few of the beams would hit a person and reflect back. Also - take a look at the reflectance data in the spec. Not bad...but imagine you are wearing a fuzzy black jacket on your top half. How much reflectance now? What do you think the point cloud returned to the car is going to look like? Will it look like a human? Hard to say - but you feed that into a classifier algorithm, there's a possibility that it's not going to identify the blob as a "human" to slow down. Especially when you add some bags, a strange gait, plus the bicycle behind the person. All of this uncertainty adds up. I am also willing to bet that only the LIDAR was used for collision detection (beyond the radar on the unit). Any cameras - even IR based - would likely only be used for lane keeping and following purposes, plus traffic sign identification. Maybe even "rear view of vehicle" detection. Ideally it would be used for "person/animal" identification and classification to - but again, given the camera sensor, and who knows what the IR sensor saw or didn't see, along with the weird lighting conditions - well, who knows how it would have classified that mix? Lots of variables here - lots of "ifs" too. All we can do is speculate, because we don't have the raw data. Uber would do well to release the entire raw dataset from all the sensors to the community and others to look over and learn from. Finally - I am not an expert on any of this; my only "qualifications" on this subject is having taken and passed a couple of Udacity MOOCs - specifically the "Self-Driving Car Engineer Nanodegree" program (2016-2017), and their "CS373" course (2012). Both courses were very enlightening and educational, but could only really be considered an introduction to this kind of tech.
jeffmould 9y ago
I have to agree. Just like a normal camera has issues in low-light, it is clear that this camera is diminishing exactly how light the road ahead was. While I can't say confidently that I would have been able to stop to prevent hitting them, watching the video in full screen does lead me to believe that I would have seen them and been able to apply the brakes at least enough to reduce the impact. Also, watching the video of the interior it is clear the driver was looking at his phone or doing something else just prior to the impact. This alone leaves me skeptical to just how much could have been done to prevent this accident.
- sydd 9y ago
  The footage from a normal camera should not matter, a self driving car is equipped with stuff that works regardless of light conditions like LIDAR oor IR cameras. This looks to me like a software failure.
  - danso 9y ago
    The footage from the normal camera does matter in that it's the main way that we (humans) can process the scene. The parent comments are just pointing out that the camera footage is likely darker than the actual scene in person.
    - pishpash 9y ago
      Even so, I count a full second from when a human paying attention would have seen something just using this video as eyes, until impact. The stopping distance at 35mph is 136ft, which is 2.65 seconds at 35mph, so the accident would still happen but the impact speed could be lower.
      - zo1 9y ago
        Yeah, but at that speed, it's more than possible to swerve around an obstacle rather than screeching to a halt before touching it. Even turning slightly to the left/right would have made a dramatic difference in the outcome to this person's life. Not to mention the person in the car that might have also been severely injured if this was a heavier obstacle. This was purely bad software, and no failure scenario being programmed in. I really don't think it's that difficult to program split-second reaction to obstacles that appear into the driving path. We need to get to a point where these vehicles can do stuff like this, even in a 2-dimensional way: https://youtu.be/uLasBsoZBi0?t=1m40s https://youtu.be/uLasBsoZBi0?t=1m40s
pdonis 9y ago
> The gap between street lights (and hence the person) was in the field of view of the camera the entire time But the gap between street lights is going to be very hard to see into. > I'm confident my eyes are good enough that I would have been able to see this person at night in these lighting conditions. I think you're overconfident. Human low light vision is very good if there is low light everywhere. But it is not good at seeing into low light regions when brightly lit regions are nearby. That said, I agree that a visible light video camera is likely to be even worse that human vision under the given circumstances. But as others have commented elsewhere in this thread, the car is not just supposed to be using a visible light video camera. It has LIDAR and IR sensors, which should have clearly shown the pedestrian well before visible light did.
- hcknwscommenter 9y ago
  > But the gap between street lights is going to be very hard to see into. This wholly contradicts my experience driving at night on a street with street lights. I can't recall a time in my entire life I have had significant difficulty seeing into the gap between street lights. Keep in mind that the gap is not arbitrarily chosen. Edited to note that I have experienced difficulties in low-vis conditions such as snow storms, sand storms, VERY strong rain storms, etc. None of which apply to this situation.
  - eanzenberg 9y ago
    Also in those stormy conditions you slow down enough where the stopping power should roughly equal your visibility.
  - jsmthrowaway 9y ago
    Keep in mind a lot of folks in this thread might be suddenly realizing they have reduced ocular ability at night, a likely common condition that pretty much nobody is aware of when it’s minor (because it’s not obvious something is amiss; maybe it’s just that dark). I agree with you that streetlights and headlights are almost universally sufficient in my experience. If they’re not, it’s worth getting your eyes checked out for light sensitivity at night. You never know. I’m not sure a typical eye exam checks for it, either, because none of the tests I can think of seem like they’d be useful. (As usual, an even keeled comment based on family experience is -2 and rapidly being silenced with zero feedback inside 5 minutes, which makes me wonder why I contribute to this community at all, probably time to stop)
hcknwscommenter 9y ago
Exactly. The dynamic range of the human eye is vastly better than a visible spectrum camera.
- extralego 9y ago
  It's not so simple. Technically, you are not wrong but a video feed should have been sufficient here. It should also be considered that digital video has improved drastically over the last decade. Even LIDAR aside, computer vision and a raw video feed should have been enough to have prevented this collision. When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. We are very capable of guessing the results of bright conditions but not dark conditions via compressed video. Moreso, these cars should not be using consumer CCDs with compression. They should be utilizing the full possible scope of video. See: https://en.wikipedia.org/wiki/Gamma_correction https://en.wikipedia.org/wiki/Gamma_correction
  - taneq 9y ago
    > When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. Gamma correction makes up for a bias against darker portions in the display, not in our eyes. It's a holdover from the CRT days where the change in brightness between pixel values of, say, 10 and 11, was far less than the change between 250 and 251. Human eyes have excellent low-light discernment which is why 'black' doesn't really look black and you can make out blocky shapes during dark scenes on some DVDs.
    - extralego 9y ago
      Compressed video lacks information in the blacks and that is why we see blocks. The blocks are not there before compression, so it’s not simply a matter of detecting them. While we are good at seeing objects in blacks, your explanation alone doesn’t account for why compression algorithms reason to remove so much of that data. Maybe we are saying the same thing. It’s hard to tell. Your assertion about the origins, however, are at odds with what I have been taught, my understanding, and all the supporting info I am finding in a quick search. My understanding is that luminance values from a sensor have something of an empirical scale but I’m sure this no complete explanation. I am speaking from my working knowledge. I can’t find anything supporting that it is simply a fix for discrepancies between display types. Can you link to something or explain what I am missing?
jjeaff 9y ago
Since in the video, we aren't seeing the original scene, but rather, the camera's interpretation of that scene, I think it would be hard to judge except to base it on what your average streetlight brightness is. Like a camera, your eye also has only so much dynamic range. So if those street lights are bright enough, or your interior lights are too bright, you might have nearly zero visibility in those shadows. But it is certain that a self driving car "should" be able to see. Even two cheap digital cameras one tuned to see the darker range and the other brighter should easily see in these type situations.
- bigiain 9y ago
  "Even two cheap digital cameras one tuned to see the darker range and the other brighter" Sounds a lot like rods and cones in our eyes, huh? There's another difference with eyeballs that would almost certainly have helped here - the low light sensitive peripheral vision that the rods provide is also attuned to movement, we're more sensitive to movement in peripheral vision as well as being better able to see in low light.
  - adamweld 9y ago
    You wouldn't need two different cameras, just one camera shooting HDR video (alternating between over/underexposing the frame so that no information is clipped) to get a clear image at all exposure levels. Eyeballs are pretty good at night vision once adjusted, but good high sensitivity cameras can be much better. And let's not get started on LIDAR/RADAR... it seems clear to me that this was not a sensory deficiency, it was poorly designed/tested software.
  - jjeaff 9y ago
    No, much more like the iris in our eye. Turn on a bright light inside a car and see how hard it is to see outside on a dark night. Modern HDR cameras have a much higher dynamic range than the human eye. Hence the surreal HDR photos you see.
bigiain 9y ago
I also suspect that human eyeballs would have a different view of the light/dark portions of what's depicted there, and especially eyeballs would have probably had a much higher chance of detecting movement in peripheral vision than that video gives any hint of. We typically cant see much detail in the scene out of our small region of focus, but you can bet if a tiger appears from behind a tree our visual system will scream to the brain _look over there right now!_ Our eyes and our entire visual processing system is very much not "just like a webcam, but made out of meat".
philip1209 9y ago
Is it possible that driving under intermittent street lights messes with the aperture or image recognition? It would be like flashing a strobe light at the camera.
- brokenmachine 9y ago
  Then WTF are they doing driving it down that road with intermittent street lights at night then?
mortdeus 9y ago
right, so the camera's night vision mode that detects objects in the dark would have been completely blinded by the street lights while passing under the streetlamp. Take night vision goggles and look at a light. It blinds the whole field of vision. The only thing that I think was the cars fault was that the car is programmed to drive when the driver is driving around distracted. There is no point to a human driver sitting behind the wheel of an autonomous vehicle if they aren't paying attention. People need to understand that self driving mode isn't a freedom from the responsibility of driving safely. Rather its a tool to help ensure that driving statistically becomes safer as more self driving vehicles find their ways onto the road. Hopefully someday all cars will be self driving and dangerous hazards/traffic reduced to the point that they are virtually none existent rather than being towards the top of the list of "preventable death" and "things humans don't want to waste most of their time during the day doing".
JepZ 9y ago
AFAIK, LIDAR doesn't use just some digicam. So the video we see there should be very different from what the sensors actually see in such a situation.
dusing 9y ago
I’ve driven that location on that road many many times at night and no it is not that dark, it is lit up well like most city streets. The video make the contrast appear greater.
- thyselius 9y ago
  If you could it would be very valuable if you did a test at that location and shared your experience
- brokenmachine 9y ago
  Thanks for your comment. That's what I thought. I have a dashcam, and I've seen night videos from it. In fact, the picture from my dashcam seems much better than this low quality mess, but still night videos from it come out much less visible than reality. I've tried to rewatch some parts of videos later, and I find I was able to see much more detail on the sidewalk and on the periphery than was captured by the dashcam. Everything gets blown out in the night videos by the headlights.
RobLach 9y ago
I believe that a human would be able to see in those conditions. It's a lit street with a car with functioning headlamps. It wasn't foggy or rainy. I've personally driven down country roads without any lighting except my headlights and saw deer poking their head out of the woods a ways away for which I slowed down in case they darted across the street. Someone slowly walking their bike would be trivial. Reminds me of this video of a Rally driver racing with malfunctioning lamps https://youtu.be/HwyRS_6Uqn0?t=2m36s https://youtu.be/HwyRS_6Uqn0?t=2m36s The video makes it seem impossible but afterwards in the interviews the driver said it wasn't too bad after his eyes adjusted. He did have some issues with his own lamp blinding him which lead to errors. (He actually won this stage.) As far as I'm concerned Uber's software/hardware is completely at fault and not ready for public testing. I'm uncertain how much better everyone else's tech is but Uber's typical carefree approach has ruined it for everyone. There are consumer level dashcam that can shift up to 12800 ISO which can create a fairly distringuishable picture with ambient moonlight.[1] Canon builds sensors with ISO's in the millions which should be able to see distinguishable shapes without ANY light. [2] [1] https://youtu.be/hHU-hWG5DDk?t=5m48s https://youtu.be/hHU-hWG5DDk?t=5m48s [2] https://petapixel.com/2015/09/13/this-is-iso-4560000-with-canons-crazy-new-camera/ https://petapixel.com/2015/09/13/this-is-iso-4560000-with-ca...
- u801e 9y ago
  > It's a lit street with a car with functioning headlamps. The headlamps may have been functioning, but they appeared to be aimed way too low. You can see that the car is able to traverse the distance lit up by the headlamp in about a second at 38 mph. If the headlamps were aimed properly, it should light up the road about 5 seconds ahead of the car.
  - kybernetikos 9y ago
    > If the headlamps were aimed properly, it should light up the road about 5 seconds ahead of the car. A system with this rule baked in would be driving slower. People adjust the way they drive based on what their environment is doing, how well their equipment is working and their own alertness. Except in the extremes we should not accept misconfigured equipment as an excuse. And if a system detects that there is no acceptably safe speed for it to go then it should not move at all.
eric_h 9y ago
The whole reason many people on here have been advocating for self driving cars is that they can see obstructions more or less perfectly in the dark with LIDAR. I am much more interested in what that sensor said. I'm reluctant to infer exactly what a human eye would have seen in that situation. I have absolutely driven down streets in suburbia where the gap between street lights was large enough to make them quite dark, and that video was an example of exactly what I was afraid of happening whenever I drove down those streets (though admittedly my fear was hitting a white tailed deer). I think it might also be fair to argue that the car's high beams were not on (but again, that shouldn't matter because of LIDAR, right?). I'm not confident even an above average human driver would be able to avoid that accident, even if good eyesight gave you an extra half second to respond. Dark clothing and no reflectors means that person was definitely invisible to both the camera and the driver for some time after they would have been visible in daylight. I've had a couple of situations where someone appeared close to my line of travel with low visibility clothing (at night) that scared the living shit out of me, and they weren't trying to cross the street. To be clear, I am not blaming the victim here, but do wear high visibility clothing when you're a pedestrian near high speed roads at night.
- sundvor 9y ago
  Sad side note that most people appear unaware of the benefits even the simplest and cheapest of reflectors do provide. The seemingly random design decision of many runner manufacturers to embed tiny reflector strips in their shoes have no doubt saved countless lives. And their owners would probably be none the wiser.
  - TeMPOraL 9y ago
    Yup. There's a place for education here. I know I wasn't really aware of the benefits until adult age, when I started to find myself more often in a car, at night, in rural areas. I still remember the first experience, in which I've noticed a cyclist on another lane ~0.5 seconds before we passed him. Dark clothes, dark bike, zero reflective elements.
    - 9y ago
ACow_Adonis 9y ago
I do know a bit about cameras, and you're spot on. You will frequently see dash cam footage and night photography blow out the relative highlights and blacken the relative shadows. This is because (cheap) hardware does not have the same dynamic range as human eyes, especially at night. So "properly exposed" it has to make a call to capture light values in the middle somewhere. Those light values too far out the top it interpreted as white, those out the bottom it interpreted as black, created an artificial high contrast version of what a human eye would see. This is pretty intuitive, generally when we're driving down the road with our lights on, we aren't literally moving between pools of black, often in many urban areas I'll even forget to turn my lights on because I can see well enough. You MAY be able to get a VERY BAD interpretation post processing of what a human would see by increasing the brightness of those pixels near the black threshold.
drawkbox 9y ago
The actual spot it happened has a bike lane, and crosswalk sidewalk by it [1]. [1] https://goo.gl/maps/gpugzAZKxcS2 https://goo.gl/maps/gpugzAZKxcS2
- kazinator 9y ago
  Not only is there no crossing here, but a sign forbidding pedestrian crossing. (Look at the left side of the road.) The sign directs people to use a crossing, which is some 100 meters away at the lights around the bend.
  - drawkbox 9y ago
    True but there are trails that cross over the road, it is an odd area. If you zoom out on google maps you will see some of the trails. Note the sidewalk/pathway. It is no pedestrian but has paths for them so it sends mixed signals.
    - kazinator 9y ago
      I saw that. The median is landscaped with a bizarre X-shaped paved area. It can't be intended for recreational use or walking; it's a divider between two fast roads. At all four entrances to the X, there is the no pedestrians sign.
Twisell 9y ago
Also even if the camera was perfectly accurate about human field of view, no human driver in his right mind would drive so fast with such a poor visibility. Any judge would qualify this as reckless driving. So either way the software failed: -If AI misjudged Lidar information and didn’t compute the slow moving pedestrian it’s a fail -If it didn’t have enough computer vision space it should have slowed down Possibly in the second scenario the human test driver is at fault too because he should have noticed bad condition and hit the autopilot kill switch.
- gowld 9y ago
  Is a driver ever at fault when a pedestrian is in the road at night without a crosswalk nor lights nor reflective gear?
  - nraynaud 9y ago
    In France it’s 100% (in civil cases), unless the driver can prove it’s a suicide. It just goes by kinetic energy: you store it, you are responsible for it. Other people don’t have to dodge your car. And since death penalty is not part of the arsenal, killing a pedestrian is not an appropriate sentence if they commit a infraction that’s punished with a 50€ fine.
rhacker 9y ago
If any of you have a dash cam it's very obvious how the light levels of images captured at night look like this video and is VERY different from what you see as a driver - objects are much brighter than this with your own eyes. Also - the car is driving way too fast. I did some driving tonight and paid close attention to when I naturally slowed down - and albeit I'm probably on the higher curve of good drivers in that I don't tailgate, drive the speed limit and generally slow much slower than the speed limit when conditions are poor (fog/rain/snow, night, slick/wet roads, near curves/hills where I can't see the road). I noticed that many of the times I naturally slowed down on the roads here I slowed considerably under the speed limit by 10 to 20 MPH in some areas. It seems this Uber SDV is generally going as fast as it is possibly allowed to regardless of what it can see.
snambi 9y ago
It is true. Even a very high resolution camera doesn't capture things that I can see with by eyes.
7dare 9y ago
I don't really know how it works in the US or in this state, but in my country, you simply can't drive when it's as dark as the video appears. Either you're not in a city and you can turn your mainbeam headlights on (the blinding ones), or you're in a city and the road is much more lit and the speed limit is 50kph. With those, the driver would've seen her from a mile away.
deleted 9y ago
[deleted]
ihattendorf 9y ago
Yep. I live half a mile away and just drove the same path tonight around 10pm, it's nothing like the video. There are spots that are darker than others, but they don't look nearly as dark. Nowhere on the street looks pitch black, there's ambient light everywhere. For anyone who's interested, try taking your phone with the camera app open into a dark room and comparing what you see to what's on the screen. Which shows more detail?
- fattire 9y ago
  Out of interest-- can you take a pic while the lighting outside is similar (assuming weather hasn't changed dramatically?) and maybe adjust exposure to what your eyes see? Or take a camera phone pic for comparison?
  - LoSboccacc 9y ago
    pic at 9pm of the spot taken out of a video circulating here earlier, sorry for the meme format https://i.imgur.com/eSre3hL.png https://i.imgur.com/eSre3hL.png
- coryfklein 9y ago
  Somebody recorded this stretch and posted it [1]. The police-released video looks much darker in comparison: [1] https://www.youtube.com/watch?v=1XOVxSCG8u0 https://www.youtube.com/watch?v=1XOVxSCG8u0
malandrew 9y ago
FWIW it was a new moon in Tempe, Arizona on Sunday night, so it was probably darker than normal.
agumonkey 9y ago
The driver wasn't fixed on the road but he glanced two times prior the collision and had no hesitation. It seems that it was at least dark enough for him not to notice a person + a bike on a large road. I'm also very (sadly) surprised that she crossed that kind of road at night without hurrying or reacting to the sound of cars approaching.
- thisacctforreal 9y ago
  However, the interior seemed filled with light, and they didn't glance up long enough for their eyes to adjust properly.
  - agumonkey 9y ago
    Good point.
reinaccess 9y ago
I too think visibility is better than it appears in the video, but I'm not so sure it's good enough to help all that much. However, even with visibility as bad as in the video, I'm confident in my ability to handle the situation. I would probably not be able to break in that short amount of time and from that speed, but neither would I drive at that speed. When there are less than ideal conditions (in this case visibility), it is our responsibility as drivers to adapt and lower the speed, possibly dramatically. This goes for autonomous cars too. If the road in front of me and the areas next to it are not clearly visible, I'd drive at such a speed that a collision would in all likelihood only result in scrapes.
deleted 9y ago
[deleted]
_pmf_ 9y ago
I agree; the video alone has exactly zero value in judging what the human driver would have seen under these conditions.
Arzh 9y ago
I think you are wrong since this people die of this exact scenario almost everyday. The camera might be making it darker but that doesn't mean that every driver (Everyone's eyes and reaction times are different) would have been able to see her and get out of the way.

netsharc 9y ago

Which makes me wonder if the Uber autocar is just relying on camera vision to drive itself... If it is, and it's been lying to authorities (I don't know what they said to the authorities about their cars' capabilities), that could be big.

deleted 9y ago
[deleted]
jonknee 9y ago
Far more likely is that this is a standard dash cam that the police know how to pop the SD card out of and access the video...
- hcknwscommenter 9y ago
  The point is not germane. What is germane is that a car that supposedly uses LIDAR and infrared, and presumably was approved by the regulators on the basis of such, should have had no problem seeing the pedestrian as LIDAR and infrared are unaffected by night and at least shown some indication of braking but did not. This suggests that the car does not in fact utilize any of those fancy (non-vis) detection methods. Alternatively, these fancy detection methods were fooled by the bicycle and thus misclassified as an error or something.
  - jonknee 9y ago
    My point is that it's likely the camera view we're seeing has nothing to do with the self driving portion of the vehicle (a good hint to this is the interior view--useless to autonomous driving, but a common feature of dash cams).
    - tqkxzugoaupvwqr 9y ago
      The car has a LIDAR sensor mounted on the roof. It is supposed to continuously scan 360° of the environment. Since LIDAR is an active sensor (it emits light), the car should have seen the person and bicycle even in the dark. That it did not do so suggests the car does not evaluate LIDAR input, or it dismissed the object as erroneous data.
      - cr0sh 9y ago
        This is the LIDAR sensor being used (virtually all SDC companies use this LIDAR because it's the most advanced out there for 360 degree 3D coverage): http://velodynelidar.com/hdl-64e.html http://velodynelidar.com/hdl-64e.html (note that this LIDAR is expensive - costs way more than the car it is mounted to) Here's the manual for it - note the specs in the back: http://www.velodynelidar.com/lidar/products/manual/HDL-64E%20Manual.pdf http://www.velodynelidar.com/lidar/products/manual/HDL-64E%2... It has 64 lasers, spread out over about 27 degrees - about 0.4 degrees per laser, from almost horizontal to an angle of 24 degrees or so down. Now take a look at where it is mounted on the car, and envision these laser beams spreading out and being spun in a circular conical area around the car. Now - if you think about it - as the distance from the sensor increases, the beams are spread further apart. I'd be willing to bet that at about 200 feet or so away from the car, very few of the beams would hit a person and reflect back. Also - take a look at the reflectance data in the spec. Not bad...but imagine you are wearing a fuzzy black jacket on your top half. How much reflectance now? What do you think the point cloud returned to the car is going to look like? Will it look like a human? Hard to say - but you feed that into a classifier algorithm, there's a possibility that it's not going to identify the blob as a "human" to slow down. Especially when you add some bags, a strange gait, plus the bicycle behind the person. All of this uncertainty adds up. I am also willing to bet that only the LIDAR was used for collision detection (beyond the radar on the unit). Any cameras - even IR based - would likely only be used for lane keeping and following purposes, plus traffic sign identification. Maybe even "rear view of vehicle" detection. Ideally it would be used for "person/animal" identification and classification to - but again, given the camera sensor, and who knows what the IR sensor saw or didn't see, along with the weird lighting conditions - well, who knows how it would have classified that mix? Lots of variables here - lots of "ifs" too. All we can do is speculate, because we don't have the raw data. Uber would do well to release the entire raw dataset from all the sensors to the community and others to look over and learn from. Finally - I am not an expert on any of this; my only "qualifications" on this subject is having taken and passed a couple of Udacity MOOCs - specifically the "Self-Driving Car Engineer Nanodegree" program (2016-2017), and their "CS373" course (2012). Both courses were very enlightening and educational, but could only really be considered an introduction to this kind of tech.

jeffmould 9y ago

I have to agree. Just like a normal camera has issues in low-light, it is clear that this camera is diminishing exactly how light the road ahead was. While I can't say confidently that I would have been able to stop to prevent hitting them, watching the video in full screen does lead me to believe that I would have seen them and been able to apply the brakes at least enough to reduce the impact. Also, watching the video of the interior it is clear the driver was looking at his phone or doing something else just prior to the impact. This alone leaves me skeptical to just how much could have been done to prevent this accident.

sydd 9y ago
The footage from a normal camera should not matter, a self driving car is equipped with stuff that works regardless of light conditions like LIDAR oor IR cameras. This looks to me like a software failure.
- danso 9y ago
  The footage from the normal camera does matter in that it's the main way that we (humans) can process the scene. The parent comments are just pointing out that the camera footage is likely darker than the actual scene in person.
  - pishpash 9y ago
    Even so, I count a full second from when a human paying attention would have seen something just using this video as eyes, until impact. The stopping distance at 35mph is 136ft, which is 2.65 seconds at 35mph, so the accident would still happen but the impact speed could be lower.
    - zo1 9y ago
      Yeah, but at that speed, it's more than possible to swerve around an obstacle rather than screeching to a halt before touching it. Even turning slightly to the left/right would have made a dramatic difference in the outcome to this person's life. Not to mention the person in the car that might have also been severely injured if this was a heavier obstacle. This was purely bad software, and no failure scenario being programmed in. I really don't think it's that difficult to program split-second reaction to obstacles that appear into the driving path. We need to get to a point where these vehicles can do stuff like this, even in a 2-dimensional way: https://youtu.be/uLasBsoZBi0?t=1m40s https://youtu.be/uLasBsoZBi0?t=1m40s

pdonis 9y ago

> The gap between street lights (and hence the person) was in the field of view of the camera the entire time But the gap between street lights is going to be very hard to see into. > I'm confident my eyes are good enough that I would have been able to see this person at night in these lighting conditions. I think you're overconfident. Human low light vision is very good if there is low light everywhere. But it is not good at seeing into low light regions when brightly lit regions are nearby. That said, I agree that a visible light video camera is likely to be even worse that human vision under the given circumstances. But as others have commented elsewhere in this thread, the car is not just supposed to be using a visible light video camera. It has LIDAR and IR sensors, which should have clearly shown the pedestrian well before visible light did.

hcknwscommenter 9y ago
> But the gap between street lights is going to be very hard to see into. This wholly contradicts my experience driving at night on a street with street lights. I can't recall a time in my entire life I have had significant difficulty seeing into the gap between street lights. Keep in mind that the gap is not arbitrarily chosen. Edited to note that I have experienced difficulties in low-vis conditions such as snow storms, sand storms, VERY strong rain storms, etc. None of which apply to this situation.
- eanzenberg 9y ago
  Also in those stormy conditions you slow down enough where the stopping power should roughly equal your visibility.
- jsmthrowaway 9y ago
  Keep in mind a lot of folks in this thread might be suddenly realizing they have reduced ocular ability at night, a likely common condition that pretty much nobody is aware of when it’s minor (because it’s not obvious something is amiss; maybe it’s just that dark). I agree with you that streetlights and headlights are almost universally sufficient in my experience. If they’re not, it’s worth getting your eyes checked out for light sensitivity at night. You never know. I’m not sure a typical eye exam checks for it, either, because none of the tests I can think of seem like they’d be useful. (As usual, an even keeled comment based on family experience is -2 and rapidly being silenced with zero feedback inside 5 minutes, which makes me wonder why I contribute to this community at all, probably time to stop)

hcknwscommenter 9y ago

Exactly. The dynamic range of the human eye is vastly better than a visible spectrum camera.

extralego 9y ago
It's not so simple. Technically, you are not wrong but a video feed should have been sufficient here. It should also be considered that digital video has improved drastically over the last decade. Even LIDAR aside, computer vision and a raw video feed should have been enough to have prevented this collision. When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. We are very capable of guessing the results of bright conditions but not dark conditions via compressed video. Moreso, these cars should not be using consumer CCDs with compression. They should be utilizing the full possible scope of video. See: https://en.wikipedia.org/wiki/Gamma_correction https://en.wikipedia.org/wiki/Gamma_correction
- taneq 9y ago
  > When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. Gamma correction makes up for a bias against darker portions in the display, not in our eyes. It's a holdover from the CRT days where the change in brightness between pixel values of, say, 10 and 11, was far less than the change between 250 and 251. Human eyes have excellent low-light discernment which is why 'black' doesn't really look black and you can make out blocky shapes during dark scenes on some DVDs.
  - extralego 9y ago
    Compressed video lacks information in the blacks and that is why we see blocks. The blocks are not there before compression, so it’s not simply a matter of detecting them. While we are good at seeing objects in blacks, your explanation alone doesn’t account for why compression algorithms reason to remove so much of that data. Maybe we are saying the same thing. It’s hard to tell. Your assertion about the origins, however, are at odds with what I have been taught, my understanding, and all the supporting info I am finding in a quick search. My understanding is that luminance values from a sensor have something of an empirical scale but I’m sure this no complete explanation. I am speaking from my working knowledge. I can’t find anything supporting that it is simply a fix for discrepancies between display types. Can you link to something or explain what I am missing?

jjeaff 9y ago

Since in the video, we aren't seeing the original scene, but rather, the camera's interpretation of that scene, I think it would be hard to judge except to base it on what your average streetlight brightness is. Like a camera, your eye also has only so much dynamic range. So if those street lights are bright enough, or your interior lights are too bright, you might have nearly zero visibility in those shadows. But it is certain that a self driving car "should" be able to see. Even two cheap digital cameras one tuned to see the darker range and the other brighter should easily see in these type situations.

bigiain 9y ago
"Even two cheap digital cameras one tuned to see the darker range and the other brighter" Sounds a lot like rods and cones in our eyes, huh? There's another difference with eyeballs that would almost certainly have helped here - the low light sensitive peripheral vision that the rods provide is also attuned to movement, we're more sensitive to movement in peripheral vision as well as being better able to see in low light.
- adamweld 9y ago
  You wouldn't need two different cameras, just one camera shooting HDR video (alternating between over/underexposing the frame so that no information is clipped) to get a clear image at all exposure levels. Eyeballs are pretty good at night vision once adjusted, but good high sensitivity cameras can be much better. And let's not get started on LIDAR/RADAR... it seems clear to me that this was not a sensory deficiency, it was poorly designed/tested software.
- jjeaff 9y ago
  No, much more like the iris in our eye. Turn on a bright light inside a car and see how hard it is to see outside on a dark night. Modern HDR cameras have a much higher dynamic range than the human eye. Hence the surreal HDR photos you see.

bigiain 9y ago

I also suspect that human eyeballs would have a different view of the light/dark portions of what's depicted there, and especially eyeballs would have probably had a much higher chance of detecting movement in peripheral vision than that video gives any hint of. We typically cant see much detail in the scene out of our small region of focus, but you can bet if a tiger appears from behind a tree our visual system will scream to the brain _look over there right now!_ Our eyes and our entire visual processing system is very much not "just like a webcam, but made out of meat".

philip1209 9y ago

Is it possible that driving under intermittent street lights messes with the aperture or image recognition? It would be like flashing a strobe light at the camera.

brokenmachine 9y ago
Then WTF are they doing driving it down that road with intermittent street lights at night then?

mortdeus 9y ago

right, so the camera's night vision mode that detects objects in the dark would have been completely blinded by the street lights while passing under the streetlamp. Take night vision goggles and look at a light. It blinds the whole field of vision. The only thing that I think was the cars fault was that the car is programmed to drive when the driver is driving around distracted. There is no point to a human driver sitting behind the wheel of an autonomous vehicle if they aren't paying attention. People need to understand that self driving mode isn't a freedom from the responsibility of driving safely. Rather its a tool to help ensure that driving statistically becomes safer as more self driving vehicles find their ways onto the road. Hopefully someday all cars will be self driving and dangerous hazards/traffic reduced to the point that they are virtually none existent rather than being towards the top of the list of "preventable death" and "things humans don't want to waste most of their time during the day doing".

JepZ 9y ago

AFAIK, LIDAR doesn't use just some digicam. So the video we see there should be very different from what the sensors actually see in such a situation.

dusing 9y ago

I’ve driven that location on that road many many times at night and no it is not that dark, it is lit up well like most city streets. The video make the contrast appear greater.

thyselius 9y ago
If you could it would be very valuable if you did a test at that location and shared your experience
brokenmachine 9y ago
Thanks for your comment. That's what I thought. I have a dashcam, and I've seen night videos from it. In fact, the picture from my dashcam seems much better than this low quality mess, but still night videos from it come out much less visible than reality. I've tried to rewatch some parts of videos later, and I find I was able to see much more detail on the sidewalk and on the periphery than was captured by the dashcam. Everything gets blown out in the night videos by the headlights.

RobLach 9y ago

I believe that a human would be able to see in those conditions. It's a lit street with a car with functioning headlamps. It wasn't foggy or rainy. I've personally driven down country roads without any lighting except my headlights and saw deer poking their head out of the woods a ways away for which I slowed down in case they darted across the street. Someone slowly walking their bike would be trivial. Reminds me of this video of a Rally driver racing with malfunctioning lamps https://youtu.be/HwyRS_6Uqn0?t=2m36s https://youtu.be/HwyRS_6Uqn0?t=2m36s The video makes it seem impossible but afterwards in the interviews the driver said it wasn't too bad after his eyes adjusted. He did have some issues with his own lamp blinding him which lead to errors. (He actually won this stage.) As far as I'm concerned Uber's software/hardware is completely at fault and not ready for public testing. I'm uncertain how much better everyone else's tech is but Uber's typical carefree approach has ruined it for everyone. There are consumer level dashcam that can shift up to 12800 ISO which can create a fairly distringuishable picture with ambient moonlight.[1] Canon builds sensors with ISO's in the millions which should be able to see distinguishable shapes without ANY light. [2] [1] https://youtu.be/hHU-hWG5DDk?t=5m48s https://youtu.be/hHU-hWG5DDk?t=5m48s [2] https://petapixel.com/2015/09/13/this-is-iso-4560000-with-canons-crazy-new-camera/ https://petapixel.com/2015/09/13/this-is-iso-4560000-with-ca...

u801e 9y ago
> It's a lit street with a car with functioning headlamps. The headlamps may have been functioning, but they appeared to be aimed way too low. You can see that the car is able to traverse the distance lit up by the headlamp in about a second at 38 mph. If the headlamps were aimed properly, it should light up the road about 5 seconds ahead of the car.
- kybernetikos 9y ago
  > If the headlamps were aimed properly, it should light up the road about 5 seconds ahead of the car. A system with this rule baked in would be driving slower. People adjust the way they drive based on what their environment is doing, how well their equipment is working and their own alertness. Except in the extremes we should not accept misconfigured equipment as an excuse. And if a system detects that there is no acceptably safe speed for it to go then it should not move at all.

eric_h 9y ago

The whole reason many people on here have been advocating for self driving cars is that they can see obstructions more or less perfectly in the dark with LIDAR. I am much more interested in what that sensor said. I'm reluctant to infer exactly what a human eye would have seen in that situation. I have absolutely driven down streets in suburbia where the gap between street lights was large enough to make them quite dark, and that video was an example of exactly what I was afraid of happening whenever I drove down those streets (though admittedly my fear was hitting a white tailed deer). I think it might also be fair to argue that the car's high beams were not on (but again, that shouldn't matter because of LIDAR, right?). I'm not confident even an above average human driver would be able to avoid that accident, even if good eyesight gave you an extra half second to respond. Dark clothing and no reflectors means that person was definitely invisible to both the camera and the driver for some time after they would have been visible in daylight. I've had a couple of situations where someone appeared close to my line of travel with low visibility clothing (at night) that scared the living shit out of me, and they weren't trying to cross the street. To be clear, I am not blaming the victim here, but do wear high visibility clothing when you're a pedestrian near high speed roads at night.

sundvor 9y ago
Sad side note that most people appear unaware of the benefits even the simplest and cheapest of reflectors do provide. The seemingly random design decision of many runner manufacturers to embed tiny reflector strips in their shoes have no doubt saved countless lives. And their owners would probably be none the wiser.
- TeMPOraL 9y ago
  Yup. There's a place for education here. I know I wasn't really aware of the benefits until adult age, when I started to find myself more often in a car, at night, in rural areas. I still remember the first experience, in which I've noticed a cyclist on another lane ~0.5 seconds before we passed him. Dark clothes, dark bike, zero reflective elements.
  - 9y ago

ACow_Adonis 9y ago

I do know a bit about cameras, and you're spot on. You will frequently see dash cam footage and night photography blow out the relative highlights and blacken the relative shadows. This is because (cheap) hardware does not have the same dynamic range as human eyes, especially at night. So "properly exposed" it has to make a call to capture light values in the middle somewhere. Those light values too far out the top it interpreted as white, those out the bottom it interpreted as black, created an artificial high contrast version of what a human eye would see. This is pretty intuitive, generally when we're driving down the road with our lights on, we aren't literally moving between pools of black, often in many urban areas I'll even forget to turn my lights on because I can see well enough. You MAY be able to get a VERY BAD interpretation post processing of what a human would see by increasing the brightness of those pixels near the black threshold.

drawkbox 9y ago

The actual spot it happened has a bike lane, and crosswalk sidewalk by it [1]. [1] https://goo.gl/maps/gpugzAZKxcS2 https://goo.gl/maps/gpugzAZKxcS2

kazinator 9y ago
Not only is there no crossing here, but a sign forbidding pedestrian crossing. (Look at the left side of the road.) The sign directs people to use a crossing, which is some 100 meters away at the lights around the bend.
- drawkbox 9y ago
  True but there are trails that cross over the road, it is an odd area. If you zoom out on google maps you will see some of the trails. Note the sidewalk/pathway. It is no pedestrian but has paths for them so it sends mixed signals.
  - kazinator 9y ago
    I saw that. The median is landscaped with a bizarre X-shaped paved area. It can't be intended for recreational use or walking; it's a divider between two fast roads. At all four entrances to the X, there is the no pedestrians sign.

Twisell 9y ago

Also even if the camera was perfectly accurate about human field of view, no human driver in his right mind would drive so fast with such a poor visibility. Any judge would qualify this as reckless driving. So either way the software failed: -If AI misjudged Lidar information and didn’t compute the slow moving pedestrian it’s a fail -If it didn’t have enough computer vision space it should have slowed down Possibly in the second scenario the human test driver is at fault too because he should have noticed bad condition and hit the autopilot kill switch.

gowld 9y ago
Is a driver ever at fault when a pedestrian is in the road at night without a crosswalk nor lights nor reflective gear?
- nraynaud 9y ago
  In France it’s 100% (in civil cases), unless the driver can prove it’s a suicide. It just goes by kinetic energy: you store it, you are responsible for it. Other people don’t have to dodge your car. And since death penalty is not part of the arsenal, killing a pedestrian is not an appropriate sentence if they commit a infraction that’s punished with a 50€ fine.

rhacker 9y ago

If any of you have a dash cam it's very obvious how the light levels of images captured at night look like this video and is VERY different from what you see as a driver - objects are much brighter than this with your own eyes. Also - the car is driving way too fast. I did some driving tonight and paid close attention to when I naturally slowed down - and albeit I'm probably on the higher curve of good drivers in that I don't tailgate, drive the speed limit and generally slow much slower than the speed limit when conditions are poor (fog/rain/snow, night, slick/wet roads, near curves/hills where I can't see the road). I noticed that many of the times I naturally slowed down on the roads here I slowed considerably under the speed limit by 10 to 20 MPH in some areas. It seems this Uber SDV is generally going as fast as it is possibly allowed to regardless of what it can see.

snambi 9y ago

It is true. Even a very high resolution camera doesn't capture things that I can see with by eyes.

7dare 9y ago

I don't really know how it works in the US or in this state, but in my country, you simply can't drive when it's as dark as the video appears. Either you're not in a city and you can turn your mainbeam headlights on (the blinding ones), or you're in a city and the road is much more lit and the speed limit is 50kph. With those, the driver would've seen her from a mile away.

deleted 9y ago

[deleted]

ihattendorf 9y ago

Yep. I live half a mile away and just drove the same path tonight around 10pm, it's nothing like the video. There are spots that are darker than others, but they don't look nearly as dark. Nowhere on the street looks pitch black, there's ambient light everywhere. For anyone who's interested, try taking your phone with the camera app open into a dark room and comparing what you see to what's on the screen. Which shows more detail?

fattire 9y ago
Out of interest-- can you take a pic while the lighting outside is similar (assuming weather hasn't changed dramatically?) and maybe adjust exposure to what your eyes see? Or take a camera phone pic for comparison?
- LoSboccacc 9y ago
  pic at 9pm of the spot taken out of a video circulating here earlier, sorry for the meme format https://i.imgur.com/eSre3hL.png https://i.imgur.com/eSre3hL.png
coryfklein 9y ago
Somebody recorded this stretch and posted it [1]. The police-released video looks much darker in comparison: [1] https://www.youtube.com/watch?v=1XOVxSCG8u0 https://www.youtube.com/watch?v=1XOVxSCG8u0

malandrew 9y ago

FWIW it was a new moon in Tempe, Arizona on Sunday night, so it was probably darker than normal.

agumonkey 9y ago

The driver wasn't fixed on the road but he glanced two times prior the collision and had no hesitation. It seems that it was at least dark enough for him not to notice a person + a bike on a large road. I'm also very (sadly) surprised that she crossed that kind of road at night without hurrying or reacting to the sound of cars approaching.

thisacctforreal 9y ago
However, the interior seemed filled with light, and they didn't glance up long enough for their eyes to adjust properly.
- agumonkey 9y ago
  Good point.

reinaccess 9y ago

I too think visibility is better than it appears in the video, but I'm not so sure it's good enough to help all that much. However, even with visibility as bad as in the video, I'm confident in my ability to handle the situation. I would probably not be able to break in that short amount of time and from that speed, but neither would I drive at that speed. When there are less than ideal conditions (in this case visibility), it is our responsibility as drivers to adapt and lower the speed, possibly dramatically. This goes for autonomous cars too. If the road in front of me and the areas next to it are not clearly visible, I'd drive at such a speed that a collision would in all likelihood only result in scrapes.

deleted 9y ago

[deleted]

_pmf_ 9y ago

I agree; the video alone has exactly zero value in judging what the human driver would have seen under these conditions.

Arzh 9y ago

I think you are wrong since this people die of this exact scenario almost everyday. The camera might be making it darker but that doesn't mean that every driver (Everyone's eyes and reaction times are different) would have been able to see her and get out of the way.