6 ms·

> It was the way the pilots described the unexpected behavior of the RL model. And you chose to quote it, and to make the point that it is not the sort of thin

> It was the way the pilots described the unexpected behavior of the RL model. And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD. And then you had the effrontery to chide dcow for responding to your claim, as if they were taking the discussion in this absurd direction. You've chosen to repeat yourself here, so I will say again that there is considerable irony in what you are doing - irony in the sense of statements being made that display a lack of self-awareness.

cycomanic 3y ago
> > It was the way the pilots described the unexpected behavior of the RL model. > And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD. At some point there were numerous reports of cars using driving assist steering into the dividers at highway offramps. I believe there were in fact some some real accidents that happened this way. That definitely was unexpected behaviour and I would argue one could even qualify it as the car acting like it had a death wish so I don't think the OP statements can be qualified as hyperbolic FUD. Now you're going to argue that these were early incarnations of non AV systems (despite the name), but I think they do illustrate how the systems can behave in unpredictable (and dangerous) ways when they encounter novel situations. That's why I commend waymo for not following the hype and keeping the environment they operate very restricted.
- mannykannot 3y ago
  > Now you're going to argue that these were early incarnations... Well, yes I am, because, as you realize, it is a valid point! I will also add that, I have, myself, made the point that we have to be careful because complex systems have bizarre failure modes, so just because, in everyday circumstances, these systems may appear to function 'sensibly', we cannot simply trust them to be 'sensible' outside of the scope of testing. I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads. We are discussing the publication of some results of Waymo's extensive testing, and I stand by what I wrote in my first response here: none of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. I agree with you that Waymo is taking the right approach here (and, FWIW, I regard Tesla's two-faced stance as unethical.) I doubt I would have objected to Bumby's invocation of a death wish in response to the past events of which you speak, but in response to Waymo's test results and the points I made about them, I think 'hyperbolic FUD' is justified. At some point, a rational person has to make accommodations when things have changed and arguments based on old data lose their relevance.
  - bumby 3y ago
    >I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads. This bypasses the entirety of the trust argument originally stated by leveraging the very point (statistics) it cautioned against. (As stated elsewhere: "So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.") >none of these three cases Regardless if you think the decision should be based on statistics alone, my further point is that an n=3 sample size is not adequate to make strong claims. Add to it that Waymo only reports the data they want to[1], there is a reason to be careful about making any claims. Here's how that scenario plays out when we rely on organizations to self-report safety data in a competitive environment, from my experience. Even though it may start with the best of intentions, cost and schedule pressure builds. Now items that would have otherwise been reported as safety incidents are now classified with vague terms like "test anomalies" and essentially buried. They will still report safety metrics, but now it's, at best, incomplete and misleading. Until some event that's egregious enough forces the company to be more transparent. [1] https://spectrum.ieee.org/have-selfdriving-cars-stopped-gett https://spectrum.ieee.org/have-selfdriving-cars-stopped-gett...
    - mannykannot 3y ago
      It is absurd of you to suppose that Waymo's testing has yielded just three data points on the safety of its cars... Just suppose there had been no incidents incidents at all - then they would clearly have to be banned from the road, as we would have absolutely no data pertaining to their safety!
      - bumby 3y ago
        Again, missed the point. We shouldn't make strong claims about self-curated data when there's an incentive to make that data look safer than it actually was. We know they aren't fully transparent. We also know that other well-known and well-funded AV developers have very bad practices that are highlighted when they are forced by regulators to be transparent. While that isn't a smoking gun against Waymo, it should give us pause and make one question a naive perspective in favor of a skeptical one.
bumby 3y ago
I'll help you out here, since there still appears to be some difficulty. I've replaced your word "it" to make the point as clear as possible. >"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads" Or to put it different terms, the sometimes unpredictable behavior resulting from RL may be a feature on the battlefield but a bug on public roadways. I still stand by that point. And, yes, that means dcow also missed that point. There's nothing hypothetical about bringing about real-world case studies of autonomous behavior based on RL models. We've been through this so many times now that I'm coming to the conclusion you may be arguing in bad faith or you get incredibly distracted by certain terms that it inhibits reading comprehension.
- mannykannot 3y ago
  >I've replaced your word "it" to make the point as clear as possible. >"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads" Ah, so we're playing the "I didn't say what I just said" game now. Here's what you actually wrote: "That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road." Anyone with the slightest familiarity with language will find no credibility in your claim that in this comment, the thing that is being called "possibly good in war" but "not so good on a public road" is anything other than the one explicitly-mentioned aspect of the system that you have specifically chosen to present as an example. Furthermore it was completely reasonable for dcow to respond "modern AVs are not driving like they have a death wish by any stretch of the imagination" after you chose to make the above statement. With your latest response, you continue to put more weight on an anecdote about this unrelated system than you do on the extensive empirical evidence from testing the actual system that is the subject of this article.
  - bumby 3y ago
    Ignoring the no true Scotsman-ism of your post, what's odd is that you are telling someone who actually wrote it what was intended. You've made it completely clear you misunderstood it. I've pointed out exactly where you made the mistake. Yet you can't seem to bring yourself to admit that just maybe your biases made you infer more than what was actually said. The "explicitly mentioned aspect" is the unpredictable behavior. You can tell that, not just from the wording, but from the fact that has been the consistent throughline of the entire sub-thread. And not to beat a dead horse, my secondary point has consistently been that we should not put too much emphasis on self-curated data when there is a bad incentive to embellish it. Yet here you are.

cycomanic 3y ago

> > It was the way the pilots described the unexpected behavior of the RL model. > And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD. At some point there were numerous reports of cars using driving assist steering into the dividers at highway offramps. I believe there were in fact some some real accidents that happened this way. That definitely was unexpected behaviour and I would argue one could even qualify it as the car acting like it had a death wish so I don't think the OP statements can be qualified as hyperbolic FUD. Now you're going to argue that these were early incarnations of non AV systems (despite the name), but I think they do illustrate how the systems can behave in unpredictable (and dangerous) ways when they encounter novel situations. That's why I commend waymo for not following the hype and keeping the environment they operate very restricted.

mannykannot 3y ago
> Now you're going to argue that these were early incarnations... Well, yes I am, because, as you realize, it is a valid point! I will also add that, I have, myself, made the point that we have to be careful because complex systems have bizarre failure modes, so just because, in everyday circumstances, these systems may appear to function 'sensibly', we cannot simply trust them to be 'sensible' outside of the scope of testing. I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads. We are discussing the publication of some results of Waymo's extensive testing, and I stand by what I wrote in my first response here: none of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. I agree with you that Waymo is taking the right approach here (and, FWIW, I regard Tesla's two-faced stance as unethical.) I doubt I would have objected to Bumby's invocation of a death wish in response to the past events of which you speak, but in response to Waymo's test results and the points I made about them, I think 'hyperbolic FUD' is justified. At some point, a rational person has to make accommodations when things have changed and arguments based on old data lose their relevance.
- bumby 3y ago
  >I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads. This bypasses the entirety of the trust argument originally stated by leveraging the very point (statistics) it cautioned against. (As stated elsewhere: "So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.") >none of these three cases Regardless if you think the decision should be based on statistics alone, my further point is that an n=3 sample size is not adequate to make strong claims. Add to it that Waymo only reports the data they want to[1], there is a reason to be careful about making any claims. Here's how that scenario plays out when we rely on organizations to self-report safety data in a competitive environment, from my experience. Even though it may start with the best of intentions, cost and schedule pressure builds. Now items that would have otherwise been reported as safety incidents are now classified with vague terms like "test anomalies" and essentially buried. They will still report safety metrics, but now it's, at best, incomplete and misleading. Until some event that's egregious enough forces the company to be more transparent. [1] https://spectrum.ieee.org/have-selfdriving-cars-stopped-gett https://spectrum.ieee.org/have-selfdriving-cars-stopped-gett...
  - mannykannot 3y ago
    It is absurd of you to suppose that Waymo's testing has yielded just three data points on the safety of its cars... Just suppose there had been no incidents incidents at all - then they would clearly have to be banned from the road, as we would have absolutely no data pertaining to their safety!
    - bumby 3y ago
      Again, missed the point. We shouldn't make strong claims about self-curated data when there's an incentive to make that data look safer than it actually was. We know they aren't fully transparent. We also know that other well-known and well-funded AV developers have very bad practices that are highlighted when they are forced by regulators to be transparent. While that isn't a smoking gun against Waymo, it should give us pause and make one question a naive perspective in favor of a skeptical one.

bumby 3y ago

I'll help you out here, since there still appears to be some difficulty. I've replaced your word "it" to make the point as clear as possible. >"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads" Or to put it different terms, the sometimes unpredictable behavior resulting from RL may be a feature on the battlefield but a bug on public roadways. I still stand by that point. And, yes, that means dcow also missed that point. There's nothing hypothetical about bringing about real-world case studies of autonomous behavior based on RL models. We've been through this so many times now that I'm coming to the conclusion you may be arguing in bad faith or you get incredibly distracted by certain terms that it inhibits reading comprehension.

mannykannot 3y ago
>I've replaced your word "it" to make the point as clear as possible. >"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads" Ah, so we're playing the "I didn't say what I just said" game now. Here's what you actually wrote: "That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road." Anyone with the slightest familiarity with language will find no credibility in your claim that in this comment, the thing that is being called "possibly good in war" but "not so good on a public road" is anything other than the one explicitly-mentioned aspect of the system that you have specifically chosen to present as an example. Furthermore it was completely reasonable for dcow to respond "modern AVs are not driving like they have a death wish by any stretch of the imagination" after you chose to make the above statement. With your latest response, you continue to put more weight on an anecdote about this unrelated system than you do on the extensive empirical evidence from testing the actual system that is the subject of this article.
- bumby 3y ago
  Ignoring the no true Scotsman-ism of your post, what's odd is that you are telling someone who actually wrote it what was intended. You've made it completely clear you misunderstood it. I've pointed out exactly where you made the mistake. Yet you can't seem to bring yourself to admit that just maybe your biases made you infer more than what was actually said. The "explicitly mentioned aspect" is the unpredictable behavior. You can tell that, not just from the wording, but from the fact that has been the consistent throughline of the entire sub-thread. And not to beat a dead horse, my secondary point has consistently been that we should not put too much emphasis on self-curated data when there is a bad incentive to embellish it. Yet here you are.