5 ms·

I believe this is a key component to true self driving cars. Being able to imagine correctly what is behind stuff is a required skill for safe driving.

majidmir 6y ago
There's no understanding happening here. As I understand it only provides a way to render images from novel views without any 3d/2d segmentation.
- faeyanpiraat 6y ago
  No understanding yet, but it's a step in that general direction.
- visarga 6y ago
  This neural net could create a realistic 3D environment where AI agents could train and explore. The lack of embodiment is the main thing missing from reinforcement learning agents to close the gap to humans. The environment itself can be controllable by embeddings, spanning any conceivable situation. Of course, ideally, if this architecture could be made much more efficient.
Uberphallus 6y ago
Basing life-or-death decisions on the computer vision interpretation of a synthetically produced image of what may or may not be behind an object sounds like a recipe for disaster, as if the current state of the field wasn't disaster enough. Just thinking of the amount of adversarial "speedrunner" type of scenarios gives me a headache.
- drmacak 6y ago
  On the other hand thats what people(brain) do all the time.
  - purplecats 6y ago
    Potentially explaining the disastrous results
  - Uberphallus 6y ago
    Your brain takes decisions based on synthetically produced images? Then maybe you shouldn't drive on LSD ;)
    - AnthonBerg 6y ago
      The contents of your visual field exist precisely in your head and nowhere else. They are synthesized from limited sensory data. It so happens that your brain also generates the sensation of the content of your visual field being located in the world around you. These captured and resynthesized visual data happen to be mostly accurate but also wildly interpreted and altered, for instance the majestic and wondrous and inexplicable addition of the sensation of color.
      - Uberphallus 6y ago
        Keyword: precisely Your brain fills a lot of gaps, and your senses have a lot of noise, but it's still closer to reality than it isn't, particularly for the fovea. Still, your brain doesn't make up what's behind a building violating our reliance on straight lines of light (unless you use sensorial "enhancement" drugs), or make up detailed images of what's in your peripheral vision (again, a known effect of many psychedelics), then presents it to you as truth, that's rather imagination and it uses a very different part of the brain, though connected to the visual cortex through the V1 area. I'm sure it can have some uses (AI racing, anyone?), but most self-driving car decisions are ruled by what has been seen (signs/lights) and what can be seen, because that's how we arranged our streets and roads to be safe for us.
        visarga 6y ago
        > your brain doesn't make up what's behind a building The brain imagines something based on prior experience. Otherwise you'd be unable to navigate complex environments.
jonplackett 6y ago
Everyone questioning this - if much rather a car have some idea what the things it can’t see might be. It doesn’t have to give as much weight to them as things it can see for certain. But this kind of predictive knowledge is what makes humans much better than AI in a lot of areas. It’s what we normally call ‘common sense’

majidmir 6y ago

There's no understanding happening here. As I understand it only provides a way to render images from novel views without any 3d/2d segmentation.

faeyanpiraat 6y ago
No understanding yet, but it's a step in that general direction.
visarga 6y ago
This neural net could create a realistic 3D environment where AI agents could train and explore. The lack of embodiment is the main thing missing from reinforcement learning agents to close the gap to humans. The environment itself can be controllable by embeddings, spanning any conceivable situation. Of course, ideally, if this architecture could be made much more efficient.

Uberphallus 6y ago

Basing life-or-death decisions on the computer vision interpretation of a synthetically produced image of what may or may not be behind an object sounds like a recipe for disaster, as if the current state of the field wasn't disaster enough. Just thinking of the amount of adversarial "speedrunner" type of scenarios gives me a headache.

drmacak 6y ago
On the other hand thats what people(brain) do all the time.
- purplecats 6y ago
  Potentially explaining the disastrous results
- Uberphallus 6y ago
  Your brain takes decisions based on synthetically produced images? Then maybe you shouldn't drive on LSD ;)
  - AnthonBerg 6y ago
    The contents of your visual field exist precisely in your head and nowhere else. They are synthesized from limited sensory data. It so happens that your brain also generates the sensation of the content of your visual field being located in the world around you. These captured and resynthesized visual data happen to be mostly accurate but also wildly interpreted and altered, for instance the majestic and wondrous and inexplicable addition of the sensation of color.
    - Uberphallus 6y ago
      Keyword: precisely Your brain fills a lot of gaps, and your senses have a lot of noise, but it's still closer to reality than it isn't, particularly for the fovea. Still, your brain doesn't make up what's behind a building violating our reliance on straight lines of light (unless you use sensorial "enhancement" drugs), or make up detailed images of what's in your peripheral vision (again, a known effect of many psychedelics), then presents it to you as truth, that's rather imagination and it uses a very different part of the brain, though connected to the visual cortex through the V1 area. I'm sure it can have some uses (AI racing, anyone?), but most self-driving car decisions are ruled by what has been seen (signs/lights) and what can be seen, because that's how we arranged our streets and roads to be safe for us.
      - visarga 6y ago
        > your brain doesn't make up what's behind a building The brain imagines something based on prior experience. Otherwise you'd be unable to navigate complex environments.

jonplackett 6y ago

Everyone questioning this - if much rather a car have some idea what the things it can’t see might be. It doesn’t have to give as much weight to them as things it can see for certain. But this kind of predictive knowledge is what makes humans much better than AI in a lot of areas. It’s what we normally call ‘common sense’