15 ms·

A popular self-driving car dataset is missing labels for hundreds of pedestrians

412 points by yeldarb 7y ago

yeldarb 7y ago
This is really scary. I discovered this because we're working on converting and re-hosting popular datasets in many popular formats for easy use across models... I first noticed that there were a bunch of completely unlabeled images. Upon digging in, I was appalled that fully 1/3 of the images contained errors or omissions! Some are small (eg a part of a car on the edge of the frame or a ways in the distance not being labeled) but some are egregious (like the woman in the crosswalk with a baby stroller). I think this really calls out the importance of rigorously inspecting any data you plan to use with your models. Garbage in, garbage out... and self-driving cars should be treated seriously. I went ahead and corrected by hand the missing bounding boxes and fixed a bunch of other errors like phantom annotations and duplicated boxes. There are still quite a few duplicate boxes (especially around traffic lights) that would have been tedious to fix manually, but if there's enough demand I'll go back and clean those as well.
- kleer001 7y ago
  I understand your concern and share it myself. This is an important time and we should be really careful training these things. However, training as used in the real world isn't on a still frame only basis, it's used in sequence. And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Then, of course, my intuition might be incorrect.
  - rhizome 7y ago
    >And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Would you bet a family member? That a distracted driver is a hazard does not mean other drivers are safe, or even safer.
    - pathseeker 7y ago
      >Would you bet a family member? "think of the children!" Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Thousands of people are dying every day in cars. This is not something we need to wait for it to be perfect. It only needs to be better.
      - rifung 7y ago
        > Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Maybe logically that makes sense but from an ethical perspective I argue it's much more complicated than that (e.g. the trolley problem) In the current system if a human is at fault, they take the blame for the accident. If we decide to move to self driving cars that we know are far from perfect but statistically better than humans, who do we blame when an accident inevitably happens? Do we blame the manufacturer even though their system is operating within the limits they've advertised? Or do we just say well, it's better than it used to be and it's no one's fault? When the systems become significantly better than humans, I can see this perhaps being a reasonable argument, but if it's just slightly better, I'm not sure people will be convinced.
        craftinator 7y ago
        I'm voting for the "less dead people" option. Mostly because I'm a selfish person, been in automobile accidents caused by lapsing human attention, and I want it to be less likely that I'll die in a car crash.
        cameronbrown 7y ago
        But it's not just about quantity. It's also _different_ people who will die. That radically alters things from an ethical perspective.
  - klyrs 7y ago
    At what framerate? With 30 fps, a 60mph vehicle advances about 5 feet per frame. Every incorrectly labeled frame delays the reaction by a nontrivial amount. My intuition is that your trained accuracy will not exceed the accuracy of the training set. This is literally a matter of life and death; every frame matters.
    - broehrl 7y ago
      Tesla claims using 110 fps in their older and 2,300 fps in their newer cars. (https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware_3 https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware_3)
      - jakecopp 7y ago
        Following the wikipedia source: > CEO Elon Musk says that it is capable of processing 200 frames per second and Tesla’s hardware 3 computer, which is optimized to run a neural net, will be able to handle 2,000 frames per second with redundancy. No way they're shooting 2,000fps, let alone making that many adjustments per second. Maybe that's just a radar ping/signal frequency? I like Elon, but why does he keep saying things like this?
    - Florin_Andrei 7y ago
      > With 30 fps What is the framerate actually used by real-world systems, such as Tesla cars?
  - barkingcat 7y ago
    Your intuition should be that almost everything is incorrectly labelled. And even more so on a large scale sequential basis. If you look at public datasets, it's more often that things are incorrectly tagged/labeled rather than correctly. Entropy is a real thing.
- tomnipotent 7y ago
  > This is really scary. No, it's not even remotely "really scary". No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space or is represented of the impact a few mislabelled samples have on the ability of these systems & algorithms to generalize.
  - mumblemumble 7y ago
    It's scary when you consider two other factors: First, the AI hype train. People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically, that it is intelligent. Many enthusiastic people, and also many policymakers, don't fully realize the extent to which machine learning is constrained by both the quality and nature of its training data, and the capabilities of the larger, non-intelligent, software framework that it's being plugged into. Second, that we have one case study - the post-mortem analysis of the fatal pedestrian collision in Arizona - that strongly indicates that commercial products are not free of the sorts problems being highlighted here, and that, unlike what others have suggested, misclassification problems aren't necessarily an issue that's isolated to individual frames and that will come out in the wash when the software is dealing with a stream of frames. Me, I think that self-driving cars are probably a lot like nuclear power. In theory, yes, it is a great idea. In practice, there are a lot of little details that one must get right, and there seem to be a whole lot of opportunities for flaky engineering decisions and incompetent public policy, both enabled by insufficiently-tempered optimism, to scuttle the whole thing.
    - tomnipotent 7y ago
      I'm not sure what point you're trying to make, but it has nothing to do with some mislabeled examples. Every system using supervised training assets has labeling issues, and the world hasn't ended yet. Take a look at Google Translate. This is only "scary" if you're ignorant of the problem. The OP is selling something, and it's in their best interest to spread FUD to sell it.
      - 7y ago
- deleted 7y ago
  [deleted]
- jacquesm 7y ago
  You would hope that the software would be resilient enough that missing or even faulty labels would not be the cause of problems. It is by the way relatively easy to - once you reach some arbitrary threshold of precision - to detect such missing or wrong labels automatically, a simple suggestion UI with an accept / reject button could take care of supplying the bulk of the missing labels and correcting the bulk of the errors.
  - SolaceQuantum 7y ago
    How would one reach a threshold of precision if the data on which precision is determined is missing or faulty?
    - jasonwatkinspdx 7y ago
      The simple idea is to do stuff like train your model on randomized subsets of your data and then compare its performance to using all the data you have. This doesn't overcome a true "black swan", but that's not what NN are meant to be doing anyhow.
      - SolaceQuantum 7y ago
        "The simple idea is to do stuff like train your model on randomized subsets of your data and then compare its performance to using all the data you have." How do you do this when you cannot verify that your data, in subset or in whole, is accurate? And furthermore you don't know how inaccurate it is?
        TomVDB 7y ago
        My intuitive take on it: You train on a subset of the initial data. Even if the data has a certain number of incorrect frames, it should still do a decent job getting a lot of things right. Then you manually loop through all the images of the data set for which the network has detected something that isn't present in the annotations (and vice versa). If the network correctly identified a missing item that wasn't in the original set, all you need to do is press "correct" (and, again, vice versa). You now have an improved data set. Retrain, rinse, repeat. Eventually, you'll converge to a case where you have consistency between training and annotations. And then, you manually go through all images again to weed out the final mistakes. The benefit of this method is that it's much faster to click "correct" that it is to draw rectangles on the screen to label something.
- jxramos 7y ago
  Does this dataset have an auxiliary field for each data point to note if it's been human reviewed? We used to train models with only those datapoints that had n=3 concordance in manual reviewers.
  - yeldarb 7y ago
    It doesn't, but since the original was supposedly human generated and I've now checked them all (twice) it should be n=2. Would love for someone to triple check! If you find errors let me know and I'll be happy to correct them.
- jasonwatkinspdx 7y ago
  No it's not scary in the slightest. Missing or scrambled labels are routine in ML and the algorithms are able to handle it. This was a huge paper a couple years ago that demonstrates deep nets will still find the structure in the data even with totally randomized labels: https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 There are also various well known techniques to verify whether you've over or underfit.
  - bo1024 7y ago
    Strongly disagree. ML is traditionally deployed in low-risk scenarios like image labeling or recommendations where 1% or 5% error rate is totally "routine" and acceptable. Doesn't mean this kind of ML pipeline can translate to cases where errors are literally fatal.
  - sukilot 7y ago
    > deep nets will still find the structure in the data Yes, but in the training data, not in the non-training data! We want the cars to avoid real pedestrians, not only the ones labelled in the training set! From the paper: > When trained on a completely random labeling of the true data, neural networks achieve 0 training error. The test error, of course, is no better than random chance as there is no correlation between the training labels and the test labels.
- mycall 7y ago
  I would think you want some images that cut off vehicles since this is how cameras will work at times, no?
  - notatoad 7y ago
    I think the problem is not that the car is cut off, its that the cut-off portion of the car isn't labeled. If that's your training data, then you're training the system to interpret cut-off bits of cars as not cars.
Der_Einzige 7y ago
Until self driving is possible with purely unsupervised data, self driving cars and other algorithms relying on massive datasets are unlikely to get really strong because label inaccuracies like this are inevitable Also, far worse has been found in large scale datasets. Pretty sure there was CP found somewhere in imagenet
danbrooks 7y ago
I work in the AV space. There's a lot of ambiguity in labeling. How should a crowd of people be annotated? A line of parked cars? A photograph of a car? Examine the training set!
- CydeWeys 7y ago
  A line of parked cars absolutely needs to be labeled as individual cars. Any one of them could pull out in front of you at any moment.
  - wlkr 7y ago
    Indeed, whilst additionally presenting the chance of a door opening or an obscured pedestrian stepping out from between them.
    - CydeWeys 7y ago
      Yup. You need to consider not only what you can see, but what you can't see. And this is a harder problem.
- TACIXAT 7y ago
  Are there any studies on how this affects recognition? I'm labeling some vehicles and will my neural net learn better (for say vehicle identification) from a row of identical vehicles, individual vehicles with overlap, non-overlapping parts only?
- PeterisP 7y ago
  A big thing is not that the example is missing, but that it counts as a negative example. I.e. if during training a ML system notices the ambiguous combination (i.e. a woman pushing a baby stroller or a crowd) and marks it as a pedestrian, then it gets penalized in a manner that teaches it to ignore these ambigious combinations and treat it as nothing; while in practice it should probably treat such ambiguous combinations as even more "avoid-worthy" as an ordinary pedestrian. The problem is that the default assumption is "clear road, you can drive there" - so what we need isn't "pedestrian detection" that finds pedestrians and only pedestrians, we need detection of random stuff that you shouldn't drive over. If a kid is wearing a weird Halloween costume, that doesn't look like a pedestrian, but it is one; If somebody has set up a tent in the middle of a supermarket parking lot, that's not a pedestrian but it should be avoided just like one.
FanaHOVA 7y ago
What's the dataset size? If there's billions of pedestrians tagged and a couple hundreds are missing, would it actually have a big impact on the training? Also looks like a lot of the people in those images are off the road. What's the current standard in AV? Is everything tagged on the sidewalk? (Genuine questions)
- mdszy 7y ago
  From the article: >Perhaps most egregiously, 217 (1.4%) of the images were completely unlabeled but actually contained cars, trucks, street lights, and/or pedestrians. This gives us a data set size of 15500.
- zentiggr 7y ago
  If they are reporting a 33% image error rate, I would expect a large effect on accuracy no matter what the individual annotation error rates were. I am unfamiliar with the detailed tagging standards but that also seems irrelevant until the more egregious problems are resolved. Get everything untagged first, then look for smaller scoped issues. And thanks Roboflow for doing whatever amount of this.
- yeldarb 7y ago
  The dataset is 15,000 images. Breakdown of the number of labels per class (post fixes) is here: https://i.imgur.com/bOFkueI.png https://i.imgur.com/bOFkueI.png Not all of the pedestrians and cyclists were on the sidewalk, no (eg the kid on his bike in the road and the lady with a stroller in a crosswalk). I stuck with what it looked like the conventions of the original dataset were (all people labeled as pedestrians whether on the road or not). They just didn't do it very well or consistently. I do also think that makes the most sense in this context; if you were building a self driving car this layer of the stack would want to know where the people are; higher up you can combine that with where you know the roads/crosswalks/stoplights, etc (and the delta of their position between frames) to make predictions about where they might go next so your car can act accordingly. For example, a stationary pedestrian at a corner will probably cross the street when the light turns green; if you're turning you need to factor that in.
stared 7y ago
It is a self-correcting problem: these pedestrains won’t be present in the next dataset.
- sgustard 7y ago
  People should learn not to go outside if they're not labelled.
  - JadeNB 7y ago
    > People should learn not to go outside if they're not labelled. I worry what will happen when this idea breeds with the "why worry about privacy if you've got nothing to hide?" fallacy.
    - ffk 7y ago
      Wouldn't it be nice if preserving your privacy simply meant not wearing your QR code?
      - jacquesm 7y ago
        It's just 'license plates for people'. And driving without a license plate is already illegal in many places. It would be a small step to require citizens to wear their nameber clearly visible at all times when outside. The penalty for failure would be a bit harsh. But on the plus side, all the corpses would be unidentified, so technically no-bodies...
  - netsharc 7y ago
    What a funny future it'd be, if we have to wear something distinctive (giant QR codes?) so we don't get killed outside. As a side effect it would make tracking us much easier...
    - deleted 7y ago
      [deleted]
    - 7y ago
Fricken 7y ago
Well, if an autonomous vehicle outfit were running unmonitored Level 4 vehicles on public roads using only an open source data set I'd be worried. Even if it was labelled thoroughly and correctly, there isn't nearly enough data in any open source dataset to train an autonomous vehicle perception system that can operate safely without human supervision. This is not a safety critical issue.
- cmiles74 7y ago
  The Uber vehicle that ended up killing the pedestrian in Arizona was running under conditions similar to the ones that you outline here. The only notable difference was that a person was monitoring the vehicle.
  - dylan604 7y ago
    The only notable difference was that a person was present and supposed to be monitoring the vehicle. Fixed that for you
    - ClumsyPilot 7y ago
      And it will happen again. There were cases of people forgetting Nuclear Weapons on a runway for a whole day by accident. People, eventually, will fuck up. Mislabelled datasets will be used.
allovernow 7y ago
This seems like the kind of problem that is suitable for and important enough to throw hundreds or thousands of (volunteer?) people at. Imagine assembling a massive public corpus of human verified training data - we'd collectively be that much closer to a technology which will change society! This problem is screaming for a consortium effort on behalf of major corporate entities in the space, who could each benefit without revealing internal secrets.
- kevingadd 7y ago
  If you've ever had recaptcha ask you to identify stop signs, traffic lights, buses etc you're helping improve Waymo's data set for free :) Maybe it'll be shared with the public instead of used for profit in a decade or two.
- yeldarb 7y ago
  It's interesting that it was a public dataset (ImageNet) that kickstarted the amazing strides in computer vision over the last decade! More public data would be awesome (if high quality).
Animats 7y ago
These are manually tagged stills, right? Not video? That's a data set for training CAPCHA breakers, not self-driving. You need to use video, where you get to see the same objects at different ranges. Recognition gets better as you get closer. Then track the recognized objects backwards to when they first appear, and try to recognize them at smaller sizes.
- myself248 7y ago
  I'm tangential to the field, but not directly in it myself, and I tend to agree. The goal of these systems should be to "perceive" their environment, not just to "recognize" it. Perception means understanding that just because a truck (that we recognized 3 frames ago) went behind a tree, doesn't mean it ceased being a truck. Furthermore, this knowledge should be used to refine the model, to say "hey I can still see the wheels, I know those wheels were attached to that truck a moment ago, therefore I still know where the truck is, even if I can't recognize it plainly as such right now". Furthermore, even if it's completely out of view, it's still there, probably moving close to the same speed and track it was. And if its path intersects ours, we need to assume that it'll reappear at some point. And the longer it's out of view, the bigger are the errors on its estimated position, according to our knowledge of the acceleration and braking limits of trucks of that type. I've never heard of anyone even working towards this sort of perception, much less having achieved it. And until we get there, these things are all toys. Dangerous, legally nebulous toys.
  - lahwran 7y ago
    you're correct that this sort of persistent world modeling is needed for self driving cars, but from what I've heard from friends who work in the industry, both cruise and waymo have it. they're very far from using a plain CNN on their video cameras, they've got depth mapping and such and carefully constructed software making use of the perception data to model how the world will change and react to that. idk if it works well, but they definitely know they need it and are trying. that said, I've driven a tesla on autopilot, and holy crap is was so incredibly bad. I'm optimistic about self driving cars in general, but not about tesla's. it will frequently lose track of the road lines at night and fail to make turns, suddenly beeping at you that you're in control now, with no warning! I only ever used it like cruise control, but I can't understand how anyone driving a tesla would dare use the tricks that allow bypassing the restrictions that prevent taking your hands off the wheel.
trhway 7y ago
amplification of the old complex/neurosis - not only people don't notice me, now robots too! Cellophane Mister Cellophane Shoulda Been My Name Mister Cellophane 'Cause You Can Look Right Through Me Walk Right By Me And Never Know I'm There... https://www.youtube.com/watch?v=WKHzTtr_lNk https://www.youtube.com/watch?v=WKHzTtr_lNk
haditab 7y ago
I acknowledge the issues in the dataset and that it has a lot of stars on github because it's from Udacity; but calling it 'a popular self-driving car dataset' is misleading as it implies this dataset is popularly used for self-driving cars when it is in fact only a small dataset Udacity uses to teach the basics of training neural networks for self-driving cars. I've been involved in the autonomous vehicle industry for a while and have been focused on perception for most of it. Most research papers will test their models on popular datasets for self-driving cars and show the results as a sort of benchmark. I've never seen this dataset mentioned anywhere. Heck the size of the dataset is an order of magnitude smaller than most of the popular ones as well. This is just a github repo. That's it.
- thanatropism 7y ago
  Are these larger datasets routinely subject to the same kind of inspection this titanic.csv of self-driving car datasets?
  - yeldarb 7y ago
    I hope so. I've personally tested Scale's labeling service and it was much higher quality than this dataset. But it's a pretty secretive industry so I'd bet some companies' data is better than others. It'd be interesting if the NHTSB had a held-back "test set" they used to evaluate self driving cars before letting them on the road.
lolc 7y ago
This just shows how clueless AI is still. Imagine your driving instructor insisted there was no person with a stroller on that corner. You'd dump them on the spot! Only when the machines start to complain when they are fed shitty data, we can talk about them being fit to drive.
a_t48 7y ago
Hold on a second here - are unlabeled pixels used in training a NN to do detection? Will a typical NN get trained to label those pixels as “not a human”? I agree that they should be labeled, but it’s the difference between needing to throw more data at the problem (because you aren’t getting as much learning per image as you could) and actively training the car to do something bad.
- yeldarb 7y ago
  Yes, if you don’t “punish” incorrect predictions in your loss function your neural net could just get “perfect” accuracy by putting a giant bounding box around the entire image. Technically that’s “right”; it did put a box around all the obstacles just like you asked it to. But that “solution” is not useful. You want it to find what it’s looking for and only what it’s looking for. In this case, if it detects an unlabeled pedestrian the loss function will penalize it a bit for that “wrong” answer and it will slightly deviate to try to not find that pedestrian but still find the correctly labeled examples. It’s trying to fit the examples you give it best as possible.
sheepstrat 7y ago
My knee-jerk is to be upset because the risk factor of a poorly functioning pedestrian classifier is obviously higher than something like a sentiment analyzer, but at the same time, this dataset is just for educational purposes, right? Is Udacity actively recommending people use this in production settings?
- yeldarb 7y ago
  They claim[1] they're working on building an "open source self driving car" but it looks like the project hasn't had much activity recently. [1] https://github.com/udacity/self-driving-car https://github.com/udacity/self-driving-car
shoeffner 7y ago
While I am happy about their efforts, it's interesting that in the bottom left image of their example I can clearly see another unlabeled car in the lower left half (standing at the sidewalk of the street). Also, I am not sure, but it seems like there's a cyclist on the sidewalk, visible between the stroller and the car (the wheel and hands are more clearly visible). The google image marks a car and a few pedestrians, but completely misses the traffic lights at the junction. So I guess even their fixed dataset still misses many labels, if already their showcases miss some.
- yeldarb 7y ago
  Hey, OP here, yeah you’re correct. The dataset doesn’t label any obstacles that small/far in the distance. I zoomed in on the region with errors for the sake of the screenshot. Here’s the original run through Google Vision AI. They actually don’t get the pedestrian either: https://imgur.com/a/84IVTV6 https://imgur.com/a/84IVTV6 (I fired up the labeling tool I use and grabbed a recording of the few seconds of video around that frame to give an idea of what’s labeled in the dataset and what’s not at that imgur link as well)
  - shoeffner 7y ago
    Nice, thank you for the upload and thank you for clarifying that small objects are not labeled, that explains it. I was suprised because other images do contain rather small labels for traffic lights or even cars, but I guess it's always in the eyes of the person who labels the data. I think you did an amazing amount of work and huge improvements over the original, have you considered contributing the changes back upstream?
    - yeldarb 7y ago
      I plan to. They use a custom CSV format that my labeling tool can't work with so I converted everything to VOC XML. I need to write a script to convert back to their format to submit a PR. Not sure if they'll accept the PR though; the original data had a "visualization link" back to the labeling company on each line which I can't reproduce.
joshvm 7y ago
This is the same for MS COCO: https://github.com/AlexeyAB/darknet/issues/4085 https://github.com/AlexeyAB/darknet/issues/4085 Lots of datasets are semi-automatically generated, with human oversight. Sometimes annotators miss things or are just plain lazy.
marta_morena 7y ago
It's a bit like saying "We found this store and it doesn't label kitchen knifes as potentially dangerous"... Nobody in their right mind would use this data set for developing a self-driving car and if one does, then they will not get a license to operate... And if by any chance they do it's still like stabbing other people with a knife snatched from the shelf a supermarket. You just don't do it.
sneak 7y ago
> REDACTED accelerates your computer vision workflow through automated annotation quality assurance, universal annotation format conversion (like PASCAL VOC XML to COCO JSON), team sharing and versioning, and exports directly to file format, like TFRecords. This is an ad, posing as a sensationalistic blog post.
gdsdfe 7y ago
Do they really think people are using these datasets for commercial applications?
- ClumsyPilot 7y ago
  Do you really think there is any possible fuckup that won't happen sooner or later? Will staff of a nuclear silo forget to lock the door, and then fall asleep? Because that happened in US Do you really think someone will put a plane in production with a single safety critical sensor with no backup or fall back? Do you really think someone will pour dissolved uranium down the drain, starting a nuclear reaction and dying horribly? Do you really think someone will crash a spacecraft into mars, by mixing up imperial and metric units?
  - gdsdfe 7y ago
    None of these is similar! The off the shelf dataset won't help you achieve nothing commercially, maybe learning or tinkering at best.
djsumdog 7y ago
So, do they claim to be human managed datasets where people drew all the object bounds? I wonder if they used their own classification AI on sample sets and just called it a day. AI blind leading the AI blind?
- yeldarb 7y ago
  They claim it was ("The dataset was annotated entirely by humans using Autti" via https://github.com/udacity/self-driving-car/tree/master/annotations https://github.com/udacity/self-driving-car/tree/master/anno...). But after looking at the data I'm almost certain there was some "tool assist" going on. There were dozens of frames in a row with phantom bounding boxes in the exact same location which made it look in some way automated (maybe user error combined with a "copy bounding boxes from the previous frame" feature?)
Karliss 7y ago
Isn't training against subset of data and then validating against rest a common practice? It wouldn't detect all the mislabeling but should detect some indicating that manual inspection is required, assuming error isn't very systematic.
- yeldarb 7y ago
  It is, and there are some interesting techniques published recently to help mitigate things like this. But if you don't have a good ground truth you're at the very least flying blind and at worst feeding garbage in and getting garbage out; your models will learn what you tell them to learn.
gateKeeper 7y ago
What the hell is Autti and how was it used for the labeling? The site is so cryptic...

412 points by yeldarb 7y ago

This is really scary. I discovered this because we're working on converting and re-hosting popular datasets in many popular formats for easy use across models... I first noticed that there were a bunch of completely unlabeled images. Upon digging in, I was appalled that fully 1/3 of the images contained errors or omissions! Some are small (eg a part of a car on the edge of the frame or a ways in the distance not being labeled) but some are egregious (like the woman in the crosswalk with a baby stroller). I think this really calls out the importance of rigorously inspecting any data you plan to use with your models. Garbage in, garbage out... and self-driving cars should be treated seriously. I went ahead and corrected by hand the missing bounding boxes and fixed a bunch of other errors like phantom annotations and duplicated boxes. There are still quite a few duplicate boxes (especially around traffic lights) that would have been tedious to fix manually, but if there's enough demand I'll go back and clean those as well.

kleer001 7y ago
I understand your concern and share it myself. This is an important time and we should be really careful training these things. However, training as used in the real world isn't on a still frame only basis, it's used in sequence. And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Then, of course, my intuition might be incorrect.
- rhizome 7y ago
  >And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Would you bet a family member? That a distracted driver is a hazard does not mean other drivers are safe, or even safer.
  - pathseeker 7y ago
    >Would you bet a family member? "think of the children!" Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Thousands of people are dying every day in cars. This is not something we need to wait for it to be perfect. It only needs to be better.
    - rifung 7y ago
      > Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Maybe logically that makes sense but from an ethical perspective I argue it's much more complicated than that (e.g. the trolley problem) In the current system if a human is at fault, they take the blame for the accident. If we decide to move to self driving cars that we know are far from perfect but statistically better than humans, who do we blame when an accident inevitably happens? Do we blame the manufacturer even though their system is operating within the limits they've advertised? Or do we just say well, it's better than it used to be and it's no one's fault? When the systems become significantly better than humans, I can see this perhaps being a reasonable argument, but if it's just slightly better, I'm not sure people will be convinced.
      - craftinator 7y ago
        I'm voting for the "less dead people" option. Mostly because I'm a selfish person, been in automobile accidents caused by lapsing human attention, and I want it to be less likely that I'll die in a car crash.
        cameronbrown 7y ago
        But it's not just about quantity. It's also _different_ people who will die. That radically alters things from an ethical perspective.
- klyrs 7y ago
  At what framerate? With 30 fps, a 60mph vehicle advances about 5 feet per frame. Every incorrectly labeled frame delays the reaction by a nontrivial amount. My intuition is that your trained accuracy will not exceed the accuracy of the training set. This is literally a matter of life and death; every frame matters.
  - broehrl 7y ago
    Tesla claims using 110 fps in their older and 2,300 fps in their newer cars. (https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware_3 https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware_3)
    - jakecopp 7y ago
      Following the wikipedia source: > CEO Elon Musk says that it is capable of processing 200 frames per second and Tesla’s hardware 3 computer, which is optimized to run a neural net, will be able to handle 2,000 frames per second with redundancy. No way they're shooting 2,000fps, let alone making that many adjustments per second. Maybe that's just a radar ping/signal frequency? I like Elon, but why does he keep saying things like this?
  - Florin_Andrei 7y ago
    > With 30 fps What is the framerate actually used by real-world systems, such as Tesla cars?
- barkingcat 7y ago
  Your intuition should be that almost everything is incorrectly labelled. And even more so on a large scale sequential basis. If you look at public datasets, it's more often that things are incorrectly tagged/labeled rather than correctly. Entropy is a real thing.
tomnipotent 7y ago
> This is really scary. No, it's not even remotely "really scary". No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space or is represented of the impact a few mislabelled samples have on the ability of these systems & algorithms to generalize.
- mumblemumble 7y ago
  It's scary when you consider two other factors: First, the AI hype train. People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically, that it is intelligent. Many enthusiastic people, and also many policymakers, don't fully realize the extent to which machine learning is constrained by both the quality and nature of its training data, and the capabilities of the larger, non-intelligent, software framework that it's being plugged into. Second, that we have one case study - the post-mortem analysis of the fatal pedestrian collision in Arizona - that strongly indicates that commercial products are not free of the sorts problems being highlighted here, and that, unlike what others have suggested, misclassification problems aren't necessarily an issue that's isolated to individual frames and that will come out in the wash when the software is dealing with a stream of frames. Me, I think that self-driving cars are probably a lot like nuclear power. In theory, yes, it is a great idea. In practice, there are a lot of little details that one must get right, and there seem to be a whole lot of opportunities for flaky engineering decisions and incompetent public policy, both enabled by insufficiently-tempered optimism, to scuttle the whole thing.
  - tomnipotent 7y ago
    I'm not sure what point you're trying to make, but it has nothing to do with some mislabeled examples. Every system using supervised training assets has labeling issues, and the world hasn't ended yet. Take a look at Google Translate. This is only "scary" if you're ignorant of the problem. The OP is selling something, and it's in their best interest to spread FUD to sell it.
    - 7y ago
deleted 7y ago
[deleted]
jacquesm 7y ago
You would hope that the software would be resilient enough that missing or even faulty labels would not be the cause of problems. It is by the way relatively easy to - once you reach some arbitrary threshold of precision - to detect such missing or wrong labels automatically, a simple suggestion UI with an accept / reject button could take care of supplying the bulk of the missing labels and correcting the bulk of the errors.
- SolaceQuantum 7y ago
  How would one reach a threshold of precision if the data on which precision is determined is missing or faulty?
  - jasonwatkinspdx 7y ago
    The simple idea is to do stuff like train your model on randomized subsets of your data and then compare its performance to using all the data you have. This doesn't overcome a true "black swan", but that's not what NN are meant to be doing anyhow.
    - SolaceQuantum 7y ago
      "The simple idea is to do stuff like train your model on randomized subsets of your data and then compare its performance to using all the data you have." How do you do this when you cannot verify that your data, in subset or in whole, is accurate? And furthermore you don't know how inaccurate it is?
      - TomVDB 7y ago
        My intuitive take on it: You train on a subset of the initial data. Even if the data has a certain number of incorrect frames, it should still do a decent job getting a lot of things right. Then you manually loop through all the images of the data set for which the network has detected something that isn't present in the annotations (and vice versa). If the network correctly identified a missing item that wasn't in the original set, all you need to do is press "correct" (and, again, vice versa). You now have an improved data set. Retrain, rinse, repeat. Eventually, you'll converge to a case where you have consistency between training and annotations. And then, you manually go through all images again to weed out the final mistakes. The benefit of this method is that it's much faster to click "correct" that it is to draw rectangles on the screen to label something.
jxramos 7y ago
Does this dataset have an auxiliary field for each data point to note if it's been human reviewed? We used to train models with only those datapoints that had n=3 concordance in manual reviewers.
- yeldarb 7y ago
  It doesn't, but since the original was supposedly human generated and I've now checked them all (twice) it should be n=2. Would love for someone to triple check! If you find errors let me know and I'll be happy to correct them.
jasonwatkinspdx 7y ago
No it's not scary in the slightest. Missing or scrambled labels are routine in ML and the algorithms are able to handle it. This was a huge paper a couple years ago that demonstrates deep nets will still find the structure in the data even with totally randomized labels: https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 There are also various well known techniques to verify whether you've over or underfit.
- bo1024 7y ago
  Strongly disagree. ML is traditionally deployed in low-risk scenarios like image labeling or recommendations where 1% or 5% error rate is totally "routine" and acceptable. Doesn't mean this kind of ML pipeline can translate to cases where errors are literally fatal.
- sukilot 7y ago
  > deep nets will still find the structure in the data Yes, but in the training data, not in the non-training data! We want the cars to avoid real pedestrians, not only the ones labelled in the training set! From the paper: > When trained on a completely random labeling of the true data, neural networks achieve 0 training error. The test error, of course, is no better than random chance as there is no correlation between the training labels and the test labels.
mycall 7y ago
I would think you want some images that cut off vehicles since this is how cameras will work at times, no?
- notatoad 7y ago
  I think the problem is not that the car is cut off, its that the cut-off portion of the car isn't labeled. If that's your training data, then you're training the system to interpret cut-off bits of cars as not cars.

Der_Einzige 7y ago

Until self driving is possible with purely unsupervised data, self driving cars and other algorithms relying on massive datasets are unlikely to get really strong because label inaccuracies like this are inevitable Also, far worse has been found in large scale datasets. Pretty sure there was CP found somewhere in imagenet

danbrooks 7y ago

I work in the AV space. There's a lot of ambiguity in labeling. How should a crowd of people be annotated? A line of parked cars? A photograph of a car? Examine the training set!

CydeWeys 7y ago
A line of parked cars absolutely needs to be labeled as individual cars. Any one of them could pull out in front of you at any moment.
- wlkr 7y ago
  Indeed, whilst additionally presenting the chance of a door opening or an obscured pedestrian stepping out from between them.
  - CydeWeys 7y ago
    Yup. You need to consider not only what you can see, but what you can't see. And this is a harder problem.
TACIXAT 7y ago
Are there any studies on how this affects recognition? I'm labeling some vehicles and will my neural net learn better (for say vehicle identification) from a row of identical vehicles, individual vehicles with overlap, non-overlapping parts only?
PeterisP 7y ago
A big thing is not that the example is missing, but that it counts as a negative example. I.e. if during training a ML system notices the ambiguous combination (i.e. a woman pushing a baby stroller or a crowd) and marks it as a pedestrian, then it gets penalized in a manner that teaches it to ignore these ambigious combinations and treat it as nothing; while in practice it should probably treat such ambiguous combinations as even more "avoid-worthy" as an ordinary pedestrian. The problem is that the default assumption is "clear road, you can drive there" - so what we need isn't "pedestrian detection" that finds pedestrians and only pedestrians, we need detection of random stuff that you shouldn't drive over. If a kid is wearing a weird Halloween costume, that doesn't look like a pedestrian, but it is one; If somebody has set up a tent in the middle of a supermarket parking lot, that's not a pedestrian but it should be avoided just like one.

FanaHOVA 7y ago

What's the dataset size? If there's billions of pedestrians tagged and a couple hundreds are missing, would it actually have a big impact on the training? Also looks like a lot of the people in those images are off the road. What's the current standard in AV? Is everything tagged on the sidewalk? (Genuine questions)

mdszy 7y ago
From the article: >Perhaps most egregiously, 217 (1.4%) of the images were completely unlabeled but actually contained cars, trucks, street lights, and/or pedestrians. This gives us a data set size of 15500.
zentiggr 7y ago
If they are reporting a 33% image error rate, I would expect a large effect on accuracy no matter what the individual annotation error rates were. I am unfamiliar with the detailed tagging standards but that also seems irrelevant until the more egregious problems are resolved. Get everything untagged first, then look for smaller scoped issues. And thanks Roboflow for doing whatever amount of this.
yeldarb 7y ago
The dataset is 15,000 images. Breakdown of the number of labels per class (post fixes) is here: https://i.imgur.com/bOFkueI.png https://i.imgur.com/bOFkueI.png Not all of the pedestrians and cyclists were on the sidewalk, no (eg the kid on his bike in the road and the lady with a stroller in a crosswalk). I stuck with what it looked like the conventions of the original dataset were (all people labeled as pedestrians whether on the road or not). They just didn't do it very well or consistently. I do also think that makes the most sense in this context; if you were building a self driving car this layer of the stack would want to know where the people are; higher up you can combine that with where you know the roads/crosswalks/stoplights, etc (and the delta of their position between frames) to make predictions about where they might go next so your car can act accordingly. For example, a stationary pedestrian at a corner will probably cross the street when the light turns green; if you're turning you need to factor that in.

stared 7y ago

It is a self-correcting problem: these pedestrains won’t be present in the next dataset.

sgustard 7y ago
People should learn not to go outside if they're not labelled.
- JadeNB 7y ago
  > People should learn not to go outside if they're not labelled. I worry what will happen when this idea breeds with the "why worry about privacy if you've got nothing to hide?" fallacy.
  - ffk 7y ago
    Wouldn't it be nice if preserving your privacy simply meant not wearing your QR code?
    - jacquesm 7y ago
      It's just 'license plates for people'. And driving without a license plate is already illegal in many places. It would be a small step to require citizens to wear their nameber clearly visible at all times when outside. The penalty for failure would be a bit harsh. But on the plus side, all the corpses would be unidentified, so technically no-bodies...
- netsharc 7y ago
  What a funny future it'd be, if we have to wear something distinctive (giant QR codes?) so we don't get killed outside. As a side effect it would make tracking us much easier...
  - deleted 7y ago
    [deleted]
  - 7y ago

Fricken 7y ago

Well, if an autonomous vehicle outfit were running unmonitored Level 4 vehicles on public roads using only an open source data set I'd be worried. Even if it was labelled thoroughly and correctly, there isn't nearly enough data in any open source dataset to train an autonomous vehicle perception system that can operate safely without human supervision. This is not a safety critical issue.

cmiles74 7y ago
The Uber vehicle that ended up killing the pedestrian in Arizona was running under conditions similar to the ones that you outline here. The only notable difference was that a person was monitoring the vehicle.
- dylan604 7y ago
  The only notable difference was that a person was present and supposed to be monitoring the vehicle. Fixed that for you
  - ClumsyPilot 7y ago
    And it will happen again. There were cases of people forgetting Nuclear Weapons on a runway for a whole day by accident. People, eventually, will fuck up. Mislabelled datasets will be used.

allovernow 7y ago

This seems like the kind of problem that is suitable for and important enough to throw hundreds or thousands of (volunteer?) people at. Imagine assembling a massive public corpus of human verified training data - we'd collectively be that much closer to a technology which will change society! This problem is screaming for a consortium effort on behalf of major corporate entities in the space, who could each benefit without revealing internal secrets.

kevingadd 7y ago
If you've ever had recaptcha ask you to identify stop signs, traffic lights, buses etc you're helping improve Waymo's data set for free :) Maybe it'll be shared with the public instead of used for profit in a decade or two.
yeldarb 7y ago
It's interesting that it was a public dataset (ImageNet) that kickstarted the amazing strides in computer vision over the last decade! More public data would be awesome (if high quality).

Animats 7y ago

These are manually tagged stills, right? Not video? That's a data set for training CAPCHA breakers, not self-driving. You need to use video, where you get to see the same objects at different ranges. Recognition gets better as you get closer. Then track the recognized objects backwards to when they first appear, and try to recognize them at smaller sizes.

myself248 7y ago
I'm tangential to the field, but not directly in it myself, and I tend to agree. The goal of these systems should be to "perceive" their environment, not just to "recognize" it. Perception means understanding that just because a truck (that we recognized 3 frames ago) went behind a tree, doesn't mean it ceased being a truck. Furthermore, this knowledge should be used to refine the model, to say "hey I can still see the wheels, I know those wheels were attached to that truck a moment ago, therefore I still know where the truck is, even if I can't recognize it plainly as such right now". Furthermore, even if it's completely out of view, it's still there, probably moving close to the same speed and track it was. And if its path intersects ours, we need to assume that it'll reappear at some point. And the longer it's out of view, the bigger are the errors on its estimated position, according to our knowledge of the acceleration and braking limits of trucks of that type. I've never heard of anyone even working towards this sort of perception, much less having achieved it. And until we get there, these things are all toys. Dangerous, legally nebulous toys.
- lahwran 7y ago
  you're correct that this sort of persistent world modeling is needed for self driving cars, but from what I've heard from friends who work in the industry, both cruise and waymo have it. they're very far from using a plain CNN on their video cameras, they've got depth mapping and such and carefully constructed software making use of the perception data to model how the world will change and react to that. idk if it works well, but they definitely know they need it and are trying. that said, I've driven a tesla on autopilot, and holy crap is was so incredibly bad. I'm optimistic about self driving cars in general, but not about tesla's. it will frequently lose track of the road lines at night and fail to make turns, suddenly beeping at you that you're in control now, with no warning! I only ever used it like cruise control, but I can't understand how anyone driving a tesla would dare use the tricks that allow bypassing the restrictions that prevent taking your hands off the wheel.

trhway 7y ago

amplification of the old complex/neurosis - not only people don't notice me, now robots too! Cellophane Mister Cellophane Shoulda Been My Name Mister Cellophane 'Cause You Can Look Right Through Me Walk Right By Me And Never Know I'm There... https://www.youtube.com/watch?v=WKHzTtr_lNk https://www.youtube.com/watch?v=WKHzTtr_lNk

haditab 7y ago

I acknowledge the issues in the dataset and that it has a lot of stars on github because it's from Udacity; but calling it 'a popular self-driving car dataset' is misleading as it implies this dataset is popularly used for self-driving cars when it is in fact only a small dataset Udacity uses to teach the basics of training neural networks for self-driving cars. I've been involved in the autonomous vehicle industry for a while and have been focused on perception for most of it. Most research papers will test their models on popular datasets for self-driving cars and show the results as a sort of benchmark. I've never seen this dataset mentioned anywhere. Heck the size of the dataset is an order of magnitude smaller than most of the popular ones as well. This is just a github repo. That's it.

thanatropism 7y ago
Are these larger datasets routinely subject to the same kind of inspection this titanic.csv of self-driving car datasets?
- yeldarb 7y ago
  I hope so. I've personally tested Scale's labeling service and it was much higher quality than this dataset. But it's a pretty secretive industry so I'd bet some companies' data is better than others. It'd be interesting if the NHTSB had a held-back "test set" they used to evaluate self driving cars before letting them on the road.

lolc 7y ago

This just shows how clueless AI is still. Imagine your driving instructor insisted there was no person with a stroller on that corner. You'd dump them on the spot! Only when the machines start to complain when they are fed shitty data, we can talk about them being fit to drive.

a_t48 7y ago

Hold on a second here - are unlabeled pixels used in training a NN to do detection? Will a typical NN get trained to label those pixels as “not a human”? I agree that they should be labeled, but it’s the difference between needing to throw more data at the problem (because you aren’t getting as much learning per image as you could) and actively training the car to do something bad.

yeldarb 7y ago
Yes, if you don’t “punish” incorrect predictions in your loss function your neural net could just get “perfect” accuracy by putting a giant bounding box around the entire image. Technically that’s “right”; it did put a box around all the obstacles just like you asked it to. But that “solution” is not useful. You want it to find what it’s looking for and only what it’s looking for. In this case, if it detects an unlabeled pedestrian the loss function will penalize it a bit for that “wrong” answer and it will slightly deviate to try to not find that pedestrian but still find the correctly labeled examples. It’s trying to fit the examples you give it best as possible.

sheepstrat 7y ago

My knee-jerk is to be upset because the risk factor of a poorly functioning pedestrian classifier is obviously higher than something like a sentiment analyzer, but at the same time, this dataset is just for educational purposes, right? Is Udacity actively recommending people use this in production settings?

yeldarb 7y ago
They claim[1] they're working on building an "open source self driving car" but it looks like the project hasn't had much activity recently. [1] https://github.com/udacity/self-driving-car https://github.com/udacity/self-driving-car

shoeffner 7y ago

While I am happy about their efforts, it's interesting that in the bottom left image of their example I can clearly see another unlabeled car in the lower left half (standing at the sidewalk of the street). Also, I am not sure, but it seems like there's a cyclist on the sidewalk, visible between the stroller and the car (the wheel and hands are more clearly visible). The google image marks a car and a few pedestrians, but completely misses the traffic lights at the junction. So I guess even their fixed dataset still misses many labels, if already their showcases miss some.

yeldarb 7y ago
Hey, OP here, yeah you’re correct. The dataset doesn’t label any obstacles that small/far in the distance. I zoomed in on the region with errors for the sake of the screenshot. Here’s the original run through Google Vision AI. They actually don’t get the pedestrian either: https://imgur.com/a/84IVTV6 https://imgur.com/a/84IVTV6 (I fired up the labeling tool I use and grabbed a recording of the few seconds of video around that frame to give an idea of what’s labeled in the dataset and what’s not at that imgur link as well)
- shoeffner 7y ago
  Nice, thank you for the upload and thank you for clarifying that small objects are not labeled, that explains it. I was suprised because other images do contain rather small labels for traffic lights or even cars, but I guess it's always in the eyes of the person who labels the data. I think you did an amazing amount of work and huge improvements over the original, have you considered contributing the changes back upstream?
  - yeldarb 7y ago
    I plan to. They use a custom CSV format that my labeling tool can't work with so I converted everything to VOC XML. I need to write a script to convert back to their format to submit a PR. Not sure if they'll accept the PR though; the original data had a "visualization link" back to the labeling company on each line which I can't reproduce.

joshvm 7y ago

This is the same for MS COCO: https://github.com/AlexeyAB/darknet/issues/4085 https://github.com/AlexeyAB/darknet/issues/4085 Lots of datasets are semi-automatically generated, with human oversight. Sometimes annotators miss things or are just plain lazy.

marta_morena 7y ago

It's a bit like saying "We found this store and it doesn't label kitchen knifes as potentially dangerous"... Nobody in their right mind would use this data set for developing a self-driving car and if one does, then they will not get a license to operate... And if by any chance they do it's still like stabbing other people with a knife snatched from the shelf a supermarket. You just don't do it.

sneak 7y ago

> REDACTED accelerates your computer vision workflow through automated annotation quality assurance, universal annotation format conversion (like PASCAL VOC XML to COCO JSON), team sharing and versioning, and exports directly to file format, like TFRecords. This is an ad, posing as a sensationalistic blog post.

gdsdfe 7y ago

Do they really think people are using these datasets for commercial applications?

ClumsyPilot 7y ago
Do you really think there is any possible fuckup that won't happen sooner or later? Will staff of a nuclear silo forget to lock the door, and then fall asleep? Because that happened in US Do you really think someone will put a plane in production with a single safety critical sensor with no backup or fall back? Do you really think someone will pour dissolved uranium down the drain, starting a nuclear reaction and dying horribly? Do you really think someone will crash a spacecraft into mars, by mixing up imperial and metric units?
- gdsdfe 7y ago
  None of these is similar! The off the shelf dataset won't help you achieve nothing commercially, maybe learning or tinkering at best.

djsumdog 7y ago

So, do they claim to be human managed datasets where people drew all the object bounds? I wonder if they used their own classification AI on sample sets and just called it a day. AI blind leading the AI blind?

yeldarb 7y ago
They claim it was ("The dataset was annotated entirely by humans using Autti" via https://github.com/udacity/self-driving-car/tree/master/annotations https://github.com/udacity/self-driving-car/tree/master/anno...). But after looking at the data I'm almost certain there was some "tool assist" going on. There were dozens of frames in a row with phantom bounding boxes in the exact same location which made it look in some way automated (maybe user error combined with a "copy bounding boxes from the previous frame" feature?)

Karliss 7y ago

Isn't training against subset of data and then validating against rest a common practice? It wouldn't detect all the mislabeling but should detect some indicating that manual inspection is required, assuming error isn't very systematic.

yeldarb 7y ago
It is, and there are some interesting techniques published recently to help mitigate things like this. But if you don't have a good ground truth you're at the very least flying blind and at worst feeding garbage in and getting garbage out; your models will learn what you tell them to learn.

gateKeeper 7y ago

What the hell is Autti and how was it used for the labeling? The site is so cryptic...