11 ms·

> This is really scary. No, it's not even remotely "really scary". No one is putting an actual self-driving car on the market using this specific data set. Dis

> This is really scary. No, it's not even remotely "really scary". No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space or is represented of the impact a few mislabelled samples have on the ability of these systems & algorithms to generalize.

mumblemumble 7y ago
It's scary when you consider two other factors: First, the AI hype train. People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically, that it is intelligent. Many enthusiastic people, and also many policymakers, don't fully realize the extent to which machine learning is constrained by both the quality and nature of its training data, and the capabilities of the larger, non-intelligent, software framework that it's being plugged into. Second, that we have one case study - the post-mortem analysis of the fatal pedestrian collision in Arizona - that strongly indicates that commercial products are not free of the sorts problems being highlighted here, and that, unlike what others have suggested, misclassification problems aren't necessarily an issue that's isolated to individual frames and that will come out in the wash when the software is dealing with a stream of frames. Me, I think that self-driving cars are probably a lot like nuclear power. In theory, yes, it is a great idea. In practice, there are a lot of little details that one must get right, and there seem to be a whole lot of opportunities for flaky engineering decisions and incompetent public policy, both enabled by insufficiently-tempered optimism, to scuttle the whole thing.
- tomnipotent 7y ago
  I'm not sure what point you're trying to make, but it has nothing to do with some mislabeled examples. Every system using supervised training assets has labeling issues, and the world hasn't ended yet. Take a look at Google Translate. This is only "scary" if you're ignorant of the problem. The OP is selling something, and it's in their best interest to spread FUD to sell it.
  - semi-extrinsic 7y ago
    Nobody gets killed by Google Translate etc. messing up, so no sweat, throw together something that works 99% or the time. But for safety critical systems it's six nines reliability or GTFO.
    - thaumasiotes 7y ago
      > Nobody gets killed by Google Translate etc. messing up, so no sweat It's not so much that "nobody gets killed by Google translate messing up" as "when people get killed by Google Translate messing up, I can't tell".
      - perl4ever 7y ago
        I think Google Maps is more relevant to self driving. Google Maps has become, in my opinion, the best way to navigate, far superior to any other GPS-enabled option I know of. But the better it gets, the more obvious it is that you can't rely on it totally. If self-driving was feasible tomorrow or in the next decade, navigation would be much better. And if anyone could do it, Google would. Think of the trope about idiots driving into lakes or off the road or whatever because they were following navigation. And now think about unconditionally trusting Google Maps or anything else, for say, a year. You know it's not up to that standard. I think you can debate whether it needs to be an order of magnitude better, or multiple orders of magnitude better.
    - tomnipotent 7y ago
      Again, how does that have anything to do with a public data set? The amount of pure BS FUD in this topic is astounding.
  - gugagore 7y ago
    That's probably the least charitable interpretation possible. No idea who the OP is, in any case. It's scary that the only way we know how to build something that detects pedestrians in an image with any kind of reliability is to use training data. You: All training data has labeling issues. Me: Training data is the only way we know how to build some aspects of systems. Other people here: Some of these systems are safety-critical. I doesn't feel like FUD to be concerned that we need to have strategies for mitigating quality issues in training data. That if you don't have anything like that, then you cannot be in the market of building a safety-critical system. As you have identified, labeling issues are always there. Kind of how human error is always there. Okay, so now that you believe that, you have to find ways to mitigate it. But when I hang around people who do self-driving cars, they don't have a good answer to how to mitigate this risk. Some of them don't even really believe in it (which is different than being ignorant of the problem), because they argue with enough data things will get good enough. It's all really sloppy. Instead of just saying "This is FUD", why not tell us why you feel so reassured that this is not a problem? So far, you said "this always has been a problem and always will be a problem", "the world hasn't ended yet" and "Google translate". What? re: The world hasn't ended yet: okay, but we also haven't been building self-driving cars
- Florin_Andrei 7y ago
  > People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically that it is intelligent. I mean, the exact same argument could be made about people. Just because you belong to Homo Sapiens doesn't imply you're an intelligent being at all times - e.g. drink a bottle of vodka and the sapiens part is gone.
  - TeMPOraL 7y ago
    The thing with people is that we all share the same brain architecture - we mostly think alike. As a species, we have a hundred thousand plus years of experience dealing with each other; as a civilization, a couple thousand. We've explored most corner cases, designed our infrastructure around those, and built systems protecting us from outliers. For instance, people deemed too unpredictable will not be given driving licenses (and extremely unpredictable people tend to get locked up). And DUI is severely punished; while the execution isn't perfect, the threat of consequences goes a long way towards reducing it. That's the bar an AI system has to meet today. And one big hurdle is that, viewed as minds, AI systems are completely unlike our own. They're extremely unpredictable. On top of that, they lack self-preservation instinct (this may be indirectly a side effect of how we structure companies - the level of effort put into an AI system is a reflection of how much people working on it care about getting it right vs. getting it to market).
    - wpietri 7y ago
      I agree entirely with your point, and wanted to follow a small tangent: > they lack self-preservation instinct I suspect that a self-preservation instinct, even in relatively dumb animals, is enormously complex. It might seem simple to us because it's so evolved, so hardwired. If I had to bet, I'd bet that a mouse's self-preservation instinct is more complex than the first-to-market Level 5 self-driving car will be. And that's good, because I definitely don't want automated vehicles to have more than a hint of self-preservation wiring.
      - TeMPOraL 7y ago
        That's a good point, and yes, I definitely don't want an automated vehicle to have self-preservation instinct. What we're all after is people-preservation instinct, which is not entirely unrelated to the former. But the broader reason I brought up self-preservation is that in humans, it's not just about not getting killed in a crash. It's also about not having your life ruined by causing it, even if you walk away physically unscathed. A person has personal, deeply visceral stake in driving safely. A self-driving car AI currently doesn't, because it's not a mind; what it has is the reflection of priorities in the company that made it. And as it turns out, companies can kill people and get away with it, so I don't feel confident about how safety is prioritized (see also: this article).
- ummonk 7y ago
  Given that nuclear power is significantly safer than other forms of power, are you asserting that the risks of self-driving cars are more about PR and perception than actual risk?
  - mumblemumble 7y ago
    I'm saying that nuclear power could have been a safer energy option, but, in practice, the whole enterprise has been scuttled by a bunch of regrettably bad decisions that have pretty much destroyed everyone's trust. So now it doesn't really matter if it's safer, because it can no longer realistically be considered an option.
    - keenmaster 7y ago
      If everyone believes that everyone else will be irrational, then they themselves will not throw their support behind nuclear power, rendering their stance on nuclear de facto irrational. As Baby Boomers age, and Millennials/GenZ form a greater percentage of the voting population, we have an opportunity to press the reset button on nuclear. The younger generations don’t really have a solid opinion on the matter, and probably don’t think about it much. Scientists are in the best position to influence the public and push for change. It is astonishing that scientific organizations have been timid about nuclear energy, or outright against it, because of a few people with the most extreme, empirically unjustified stance.
      - kozd 7y ago
        Is it a tautology or just an unfortunate nash equilibrium?
        deleted 7y ago
        [deleted]
        keenmaster 7y ago
        As you know, a nash equilibrium is the point at which no player can benefit from a move away from the equilibrium. However, that's a concept from game theory. What is the game here? Who are the players? Part of my point is that you might think that the game is "scientists vs. politicians, ignoramuses, and fear-mongers," but that's not the case. Sometimes they're all on the same side. Imagine a 4 player split-screen Star Fox, where the scientist leaves their controller and helps the others shoot down their plane because they don't think they can win. If that sounds baffling, it is.
- xiphias2 7y ago
  As long as self driving vehicles are small passanger vehicles, they are not able to kill more than a few people. If you look at the Uber accident, killing 1 person severly harmed the company. The only extremely harmful scenario that I can imagine in self driving is a bad remote mass software update (which is sadly possible even in an otherwise great company) causing lots of accidents at the same time in different vehicles.
- runarberg 7y ago
  > First, the AI hype train. Yes! I really wished we would call this something more descriptive of the boring statistics involved. Artificial Intelligence is just too cool sounding name not to get excited about it. And what people don’t realize is that this is really just a statistical inference model, nothing intelligence. Machine Learning is better, but it is still not descriptive enough. I think if we called it something like Computational Reinforcement Modelling or Iterative Weight Inference people might stop having the idea that there are machines making “smart” choices involved and finally seeing that really we are just inferring based some computations on provided data. Really, “Machine Learning” deserves no more hype than the boring sounding Kalman Filter.
  - perl4ever 7y ago
    ...I like the Kalman Filter even if I might not be quite up to programming one. Every time I drive a modern car with a shitty automatic transmission that glitches and stutters and fails to shift right, I think "Is this thing using a Kalman filter, or what fundamentally wrong thing was done with the programming here?" I wish someone would reveal to me whether the algorithm is not the panacea I wish it were, or else what a lousy programmer does when they don't know about it.
graedus 7y ago
It's scary not because this specific dataset was used to train Teslas that are on the road today. Rather, because it makes us aware of an entire class of errors that most of us probably hadn't thought about before. I guess you are absolutely certain that training data used in production cars will be free of these issues, but it's not clear why.
- tomnipotent 7y ago
  It does not make use aware of a new class of errors. Labeling issues is nothing new, but plenty of systems trained on them continue to work just fine. This is FUD.
  - jwandborg 7y ago
    Is there any statistical/mathematical tool to completely eradicate or greatly diminish the effects of bad labeling? Is there any reason - other than the combination of pure circumstance and gut feeling of the Data Scientist in charge of saying that it's good enough to deploy - that ~33% insanity in training doesn't become ~33% insanity in the system?
    - GordonS 7y ago
      Maybe we could use deep learning? Oh, wait...
    - tomnipotent 7y ago
      > Is there any statistical/mathematical tool to completely eradicate or greatly diminish the effects of bad labeling Yes, it's called statistics and probability theory.
      - jwandborg 7y ago
        That's correct. I know what goes in and what comes out, not what happens in the middle. How does ~33% insanity in become < ~33% insanity out? Edit: Parent was edited, was previously (paraphrased) > I'm guessing you have no technical understanding of how this works
drcross 7y ago
Thank you. I can't believe the grandstand here.
TeMPOraL 7y ago
> No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? I mean, this sounds like the machine learning equivalent of "no serious business is pulling random bits of code from StackOverflow or whatever tutorial popped up first in the search results", and yet I think everyone who worked with software for more than few years can confirm that companies absolutely do it, and a lot. (Tangentially, PHP got a bad rap in large part because of this - people not knowing how to do things correctly copying code from tutorials written by people who didn't know it either; these days, JS ecosystem is getting a bad rap for having such bad code wrapped in neat and easy-to-install NPM packages.)
- TomVDB 7y ago
  Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? Fly-by-night self-driving startups will at some point run into issues with the NHTSA. George Hotz self-driving car project was shut down as soon as he announced that he'd start selling some prototype. States require permits just to be allowed to test self-driving prototypes on the open road. I don't know what the requirements are, but they're probably subject to some kind of review that's more than just filling in a form.
  - garaetjjte 7y ago
    It doesn't look like it was shut down? http://comma.ai http://comma.ai
    - Mathnerd314 7y ago
      > In the days that followed, Hotz abandoned his plans to sell Comma One directly. Instead, he embarked on a regulatory workaround: this current path of selling just the hardware and open-sourcing the code behind it, meaning it is available for free. He says often, “We are not selling a consumer product.”
  - 7y ago
deleted 7y ago
[deleted]
rocauc 7y ago
The scary part isn't necessarily this dataset, but that unlabeled data causes a silent decrease in model performance -- which can be esp important for underrepresented classes.

mumblemumble 7y ago

It's scary when you consider two other factors: First, the AI hype train. People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically, that it is intelligent. Many enthusiastic people, and also many policymakers, don't fully realize the extent to which machine learning is constrained by both the quality and nature of its training data, and the capabilities of the larger, non-intelligent, software framework that it's being plugged into. Second, that we have one case study - the post-mortem analysis of the fatal pedestrian collision in Arizona - that strongly indicates that commercial products are not free of the sorts problems being highlighted here, and that, unlike what others have suggested, misclassification problems aren't necessarily an issue that's isolated to individual frames and that will come out in the wash when the software is dealing with a stream of frames. Me, I think that self-driving cars are probably a lot like nuclear power. In theory, yes, it is a great idea. In practice, there are a lot of little details that one must get right, and there seem to be a whole lot of opportunities for flaky engineering decisions and incompetent public policy, both enabled by insufficiently-tempered optimism, to scuttle the whole thing.

tomnipotent 7y ago
I'm not sure what point you're trying to make, but it has nothing to do with some mislabeled examples. Every system using supervised training assets has labeling issues, and the world hasn't ended yet. Take a look at Google Translate. This is only "scary" if you're ignorant of the problem. The OP is selling something, and it's in their best interest to spread FUD to sell it.
- semi-extrinsic 7y ago
  Nobody gets killed by Google Translate etc. messing up, so no sweat, throw together something that works 99% or the time. But for safety critical systems it's six nines reliability or GTFO.
  - thaumasiotes 7y ago
    > Nobody gets killed by Google Translate etc. messing up, so no sweat It's not so much that "nobody gets killed by Google translate messing up" as "when people get killed by Google Translate messing up, I can't tell".
    - perl4ever 7y ago
      I think Google Maps is more relevant to self driving. Google Maps has become, in my opinion, the best way to navigate, far superior to any other GPS-enabled option I know of. But the better it gets, the more obvious it is that you can't rely on it totally. If self-driving was feasible tomorrow or in the next decade, navigation would be much better. And if anyone could do it, Google would. Think of the trope about idiots driving into lakes or off the road or whatever because they were following navigation. And now think about unconditionally trusting Google Maps or anything else, for say, a year. You know it's not up to that standard. I think you can debate whether it needs to be an order of magnitude better, or multiple orders of magnitude better.
  - tomnipotent 7y ago
    Again, how does that have anything to do with a public data set? The amount of pure BS FUD in this topic is astounding.
- gugagore 7y ago
  That's probably the least charitable interpretation possible. No idea who the OP is, in any case. It's scary that the only way we know how to build something that detects pedestrians in an image with any kind of reliability is to use training data. You: All training data has labeling issues. Me: Training data is the only way we know how to build some aspects of systems. Other people here: Some of these systems are safety-critical. I doesn't feel like FUD to be concerned that we need to have strategies for mitigating quality issues in training data. That if you don't have anything like that, then you cannot be in the market of building a safety-critical system. As you have identified, labeling issues are always there. Kind of how human error is always there. Okay, so now that you believe that, you have to find ways to mitigate it. But when I hang around people who do self-driving cars, they don't have a good answer to how to mitigate this risk. Some of them don't even really believe in it (which is different than being ignorant of the problem), because they argue with enough data things will get good enough. It's all really sloppy. Instead of just saying "This is FUD", why not tell us why you feel so reassured that this is not a problem? So far, you said "this always has been a problem and always will be a problem", "the world hasn't ended yet" and "Google translate". What? re: The world hasn't ended yet: okay, but we also haven't been building self-driving cars
Florin_Andrei 7y ago
> People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically that it is intelligent. I mean, the exact same argument could be made about people. Just because you belong to Homo Sapiens doesn't imply you're an intelligent being at all times - e.g. drink a bottle of vodka and the sapiens part is gone.
- TeMPOraL 7y ago
  The thing with people is that we all share the same brain architecture - we mostly think alike. As a species, we have a hundred thousand plus years of experience dealing with each other; as a civilization, a couple thousand. We've explored most corner cases, designed our infrastructure around those, and built systems protecting us from outliers. For instance, people deemed too unpredictable will not be given driving licenses (and extremely unpredictable people tend to get locked up). And DUI is severely punished; while the execution isn't perfect, the threat of consequences goes a long way towards reducing it. That's the bar an AI system has to meet today. And one big hurdle is that, viewed as minds, AI systems are completely unlike our own. They're extremely unpredictable. On top of that, they lack self-preservation instinct (this may be indirectly a side effect of how we structure companies - the level of effort put into an AI system is a reflection of how much people working on it care about getting it right vs. getting it to market).
  - wpietri 7y ago
    I agree entirely with your point, and wanted to follow a small tangent: > they lack self-preservation instinct I suspect that a self-preservation instinct, even in relatively dumb animals, is enormously complex. It might seem simple to us because it's so evolved, so hardwired. If I had to bet, I'd bet that a mouse's self-preservation instinct is more complex than the first-to-market Level 5 self-driving car will be. And that's good, because I definitely don't want automated vehicles to have more than a hint of self-preservation wiring.
    - TeMPOraL 7y ago
      That's a good point, and yes, I definitely don't want an automated vehicle to have self-preservation instinct. What we're all after is people-preservation instinct, which is not entirely unrelated to the former. But the broader reason I brought up self-preservation is that in humans, it's not just about not getting killed in a crash. It's also about not having your life ruined by causing it, even if you walk away physically unscathed. A person has personal, deeply visceral stake in driving safely. A self-driving car AI currently doesn't, because it's not a mind; what it has is the reflection of priorities in the company that made it. And as it turns out, companies can kill people and get away with it, so I don't feel confident about how safety is prioritized (see also: this article).
ummonk 7y ago
Given that nuclear power is significantly safer than other forms of power, are you asserting that the risks of self-driving cars are more about PR and perception than actual risk?
- mumblemumble 7y ago
  I'm saying that nuclear power could have been a safer energy option, but, in practice, the whole enterprise has been scuttled by a bunch of regrettably bad decisions that have pretty much destroyed everyone's trust. So now it doesn't really matter if it's safer, because it can no longer realistically be considered an option.
  - keenmaster 7y ago
    If everyone believes that everyone else will be irrational, then they themselves will not throw their support behind nuclear power, rendering their stance on nuclear de facto irrational. As Baby Boomers age, and Millennials/GenZ form a greater percentage of the voting population, we have an opportunity to press the reset button on nuclear. The younger generations don’t really have a solid opinion on the matter, and probably don’t think about it much. Scientists are in the best position to influence the public and push for change. It is astonishing that scientific organizations have been timid about nuclear energy, or outright against it, because of a few people with the most extreme, empirically unjustified stance.
    - kozd 7y ago
      Is it a tautology or just an unfortunate nash equilibrium?
      - deleted 7y ago
        [deleted]
      - keenmaster 7y ago
        As you know, a nash equilibrium is the point at which no player can benefit from a move away from the equilibrium. However, that's a concept from game theory. What is the game here? Who are the players? Part of my point is that you might think that the game is "scientists vs. politicians, ignoramuses, and fear-mongers," but that's not the case. Sometimes they're all on the same side. Imagine a 4 player split-screen Star Fox, where the scientist leaves their controller and helps the others shoot down their plane because they don't think they can win. If that sounds baffling, it is.
xiphias2 7y ago
As long as self driving vehicles are small passanger vehicles, they are not able to kill more than a few people. If you look at the Uber accident, killing 1 person severly harmed the company. The only extremely harmful scenario that I can imagine in self driving is a bad remote mass software update (which is sadly possible even in an otherwise great company) causing lots of accidents at the same time in different vehicles.
runarberg 7y ago
> First, the AI hype train. Yes! I really wished we would call this something more descriptive of the boring statistics involved. Artificial Intelligence is just too cool sounding name not to get excited about it. And what people don’t realize is that this is really just a statistical inference model, nothing intelligence. Machine Learning is better, but it is still not descriptive enough. I think if we called it something like Computational Reinforcement Modelling or Iterative Weight Inference people might stop having the idea that there are machines making “smart” choices involved and finally seeing that really we are just inferring based some computations on provided data. Really, “Machine Learning” deserves no more hype than the boring sounding Kalman Filter.
- perl4ever 7y ago
  ...I like the Kalman Filter even if I might not be quite up to programming one. Every time I drive a modern car with a shitty automatic transmission that glitches and stutters and fails to shift right, I think "Is this thing using a Kalman filter, or what fundamentally wrong thing was done with the programming here?" I wish someone would reveal to me whether the algorithm is not the panacea I wish it were, or else what a lousy programmer does when they don't know about it.

graedus 7y ago

It's scary not because this specific dataset was used to train Teslas that are on the road today. Rather, because it makes us aware of an entire class of errors that most of us probably hadn't thought about before. I guess you are absolutely certain that training data used in production cars will be free of these issues, but it's not clear why.

tomnipotent 7y ago
It does not make use aware of a new class of errors. Labeling issues is nothing new, but plenty of systems trained on them continue to work just fine. This is FUD.
- jwandborg 7y ago
  Is there any statistical/mathematical tool to completely eradicate or greatly diminish the effects of bad labeling? Is there any reason - other than the combination of pure circumstance and gut feeling of the Data Scientist in charge of saying that it's good enough to deploy - that ~33% insanity in training doesn't become ~33% insanity in the system?
  - GordonS 7y ago
    Maybe we could use deep learning? Oh, wait...
  - tomnipotent 7y ago
    > Is there any statistical/mathematical tool to completely eradicate or greatly diminish the effects of bad labeling Yes, it's called statistics and probability theory.
    - jwandborg 7y ago
      That's correct. I know what goes in and what comes out, not what happens in the middle. How does ~33% insanity in become < ~33% insanity out? Edit: Parent was edited, was previously (paraphrased) > I'm guessing you have no technical understanding of how this works

drcross 7y ago

Thank you. I can't believe the grandstand here.

TeMPOraL 7y ago

> No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? I mean, this sounds like the machine learning equivalent of "no serious business is pulling random bits of code from StackOverflow or whatever tutorial popped up first in the search results", and yet I think everyone who worked with software for more than few years can confirm that companies absolutely do it, and a lot. (Tangentially, PHP got a bad rap in large part because of this - people not knowing how to do things correctly copying code from tutorials written by people who didn't know it either; these days, JS ecosystem is getting a bad rap for having such bad code wrapped in neat and easy-to-install NPM packages.)

TomVDB 7y ago
Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? Fly-by-night self-driving startups will at some point run into issues with the NHTSA. George Hotz self-driving car project was shut down as soon as he announced that he'd start selling some prototype. States require permits just to be allowed to test self-driving prototypes on the open road. I don't know what the requirements are, but they're probably subject to some kind of review that's more than just filling in a form.
- garaetjjte 7y ago
  It doesn't look like it was shut down? http://comma.ai http://comma.ai
  - Mathnerd314 7y ago
    > In the days that followed, Hotz abandoned his plans to sell Comma One directly. Instead, he embarked on a regulatory workaround: this current path of selling just the hardware and open-sourcing the code behind it, meaning it is available for free. He says often, “We are not selling a consumer product.”
- 7y ago

deleted 7y ago

[deleted]

rocauc 7y ago

The scary part isn't necessarily this dataset, but that unlabeled data causes a silent decrease in model performance -- which can be esp important for underrepresented classes.