4 ms·

Interesting, but is it just me or does it seem overly jumpy?

by yarg 7y ago

jfengel 7y ago
The "vision fps" runs around 12 to 20, so yeah, that's gonna seem kinda jumpy. The display fps is slightly higher, which contributes to a juddering feeling.
- Erlich_Bachman 7y ago
  I think the more interesting/dangerous jumping is when the perception engine fails to consistently classify/find objects in the scene, between adjacent frames, not the specific fps of the process. This video seems to display some of such inter-frame jumpiness.
  - mitchellgoffpc 7y ago
    Inter-frame jumpy-ness can be smoothed out slightly by the control algorithms, on the assumption of object permanence, as long as the underlying vision NN doesn’t miss things too often.
    - deleted 7y ago
      [deleted]
Traster 7y ago
One of the real challenges of these systems is that even a very good system only needs to lose confidence in some object for a very short period of time for the system to lose confidence in its navigation and return control to the user. The easy way to get around that issue is to categorize objects and then just ignore them out of the context you expect them to appear in -- which is how Tesla managed to kill a pedestrian walking their bicycle across the road.
- anchpop 7y ago
  > which is how Tesla managed to kill a pedestrian walking their bicycle across the road. That was Uber, just so you know
- zionic 7y ago
  Wasn't that uber?
- IceSentry 7y ago
  That event was done by Uber and if the human backup driver was paying attention, as they were supposed to because it was a test run, it wouldn't have happen. Yes the software was at fault of doing what you are saying, but the incident was a test run and the driver should have caught it and report to the engineer, not play on their phone and not pay attention.
  - Traster 7y ago
    Arguing that it happened because the human back up wasn't paying attention isn't a great defense of an autonomous vehicle. The issue isn't that the bug happened, the issue is that that particular bug only comes from doing something fundamentally bad - hardcoding the situation.
deleted 7y ago
[deleted]
mitchellgoffpc 7y ago
I’m kind of impressed by how smooth it is, actually. If you watch videos of state-of-the-art object localization NNs, they tend to be EXTREMELY jumpy. These neural nets usually operate on only a single frame at a time, at least in the lower layers, so their predictions tend to jump around a lot from frame to frame (especially when the camera is moving!)
- yarg 7y ago
  It worries me in general though, and I think that a higher level of consistently in the results of the parts of the image that don't significantly change between frames seems like a goal worth pursuing. I also think that a deeper understanding of the mechanisms and techniques required to reduce jitter might offer some insights into ways of handling adversarial images.
  - yarg 7y ago
    And I do mean insights and not a potential solution. I think it's an issue related to the handling of the spatial discontinuities introduced by the conversion of an effectively continuous reality into a representation consisting of a large number of discrete elements. I think a more in depth understanding of how to navigate organically introduced discontinuities could provide a baseline against which we can look to combat the maliciously introduced ones. It most likely won't be enough - since the problem source is an adaptive and intelligent adversary.

Interesting, but is it just me or does it seem overly jumpy?

by yarg 7y ago

Interesting, but is it just me or does it seem overly jumpy?

The "vision fps" runs around 12 to 20, so yeah, that's gonna seem kinda jumpy. The display fps is slightly higher, which contributes to a juddering feeling.

Erlich_Bachman 7y ago
I think the more interesting/dangerous jumping is when the perception engine fails to consistently classify/find objects in the scene, between adjacent frames, not the specific fps of the process. This video seems to display some of such inter-frame jumpiness.
- mitchellgoffpc 7y ago
  Inter-frame jumpy-ness can be smoothed out slightly by the control algorithms, on the assumption of object permanence, as long as the underlying vision NN doesn’t miss things too often.
  - deleted 7y ago
    [deleted]

Traster 7y ago

One of the real challenges of these systems is that even a very good system only needs to lose confidence in some object for a very short period of time for the system to lose confidence in its navigation and return control to the user. The easy way to get around that issue is to categorize objects and then just ignore them out of the context you expect them to appear in -- which is how Tesla managed to kill a pedestrian walking their bicycle across the road.

anchpop 7y ago
> which is how Tesla managed to kill a pedestrian walking their bicycle across the road. That was Uber, just so you know
zionic 7y ago
Wasn't that uber?
IceSentry 7y ago
That event was done by Uber and if the human backup driver was paying attention, as they were supposed to because it was a test run, it wouldn't have happen. Yes the software was at fault of doing what you are saying, but the incident was a test run and the driver should have caught it and report to the engineer, not play on their phone and not pay attention.
- Traster 7y ago
  Arguing that it happened because the human back up wasn't paying attention isn't a great defense of an autonomous vehicle. The issue isn't that the bug happened, the issue is that that particular bug only comes from doing something fundamentally bad - hardcoding the situation.

deleted 7y ago

[deleted]

mitchellgoffpc 7y ago

I’m kind of impressed by how smooth it is, actually. If you watch videos of state-of-the-art object localization NNs, they tend to be EXTREMELY jumpy. These neural nets usually operate on only a single frame at a time, at least in the lower layers, so their predictions tend to jump around a lot from frame to frame (especially when the camera is moving!)

yarg 7y ago
It worries me in general though, and I think that a higher level of consistently in the results of the parts of the image that don't significantly change between frames seems like a goal worth pursuing. I also think that a deeper understanding of the mechanisms and techniques required to reduce jitter might offer some insights into ways of handling adversarial images.
- yarg 7y ago
  And I do mean insights and not a potential solution. I think it's an issue related to the handling of the spatial discontinuities introduced by the conversion of an effectively continuous reality into a representation consisting of a large number of discrete elements. I think a more in depth understanding of how to navigate organically introduced discontinuities could provide a baseline against which we can look to combat the maliciously introduced ones. It most likely won't be enough - since the problem source is an adaptive and intelligent adversary.