4 ms·

This feels much more like how biological systems operate. I wonder if neural networks trained on this kind of data would work any differently than ones trained

by jcims 5y ago

This feels much more like how biological systems operate. I wonder if neural networks trained on this kind of data would work any differently than ones trained on standard frame-by-frame camera feeds.

igouy 5y ago
First paragraph -- "The pixels in event-based cameras—a technology inspired by animal vision systems…"
nynx 5y ago
I was just thinking the same thing. This would would only work for biologically-plausible neuron variants, where neurons train constantly when being used. Maybe something like HTM (hierarchical temporal memory) neurons would work.
gumby 5y ago
There is a stealthy startup that at least originally was doing machine vision based on the human early visual system. You and I don’t see anything like pixels; our vision system pulls out relatively high level features like parallel lines, motion etc and starts working with that in addition to other optical data, muscle feedback on your lens shape etc. This is why many animals, including humans, freeze when frightened or when they perceive a risk: stationary objects are simply harder for most animals to pick out. The theory is (or was — I don’t know what they are doing today) is that you can see things interesting to humans and reason about them in ways interesting to humans. Not part of that company: there are some wild but interesting theories related to this kind of thing. We pretty clearly have face-finding hardware (though not in the early visual system I presume). Do animals have this? Cats and dogs look at human faces. I have read a theory that reading may hijack some subsystem originally used for identifying tracks (footprints). It does appear that recognizing 2D pictures is learnt, not innate, while 1990s/2000 ML went the other way. Some anthropomorphically-inspired design could yield systems that are more comprehensible and more useful. I wouldn't make a fetish of it (cars don't run faster than a human does) but machines should be better adapted to humans rather than the other way around (e.g. cars need special places to move around in so can only help humans when that is possible).
- ngcc_hk 5y ago
  Is the basic input still “pixel” and have we dropped these and maintain our neural network in … Yes it seems we do not see and think in pixel, but are we really not think in pixel even though we are not aware of them ?
  - gumby 5y ago
    The concept of ‘pixel’ is only about 45 years old, and wasn’t even a concept in the age of type. If you look at the anatomy of the retina or the physiology of the early visual system you’ll see that the concept of “pixel” doesn’t even really make sense in that context.
    - franga2000 5y ago
      Doesn't it? My understanding is basic, but as far as I know we have discrete detector cells that respond to light hitting them. We have ones that respond to all light as well as different ones that respond to only certain wavelength ranges. They aren't aligned in a matrix, sure, but that's not what makes a pixel a pixel. We don't see the pixels because our consciousness works at a higher level of abstraction. You won't find pixel a few layers deep into a CNN either, but that doesn't mean they don't exist, the "neurons" at that layer just don't see them because they have no use for them. That's always been my understanding at least...
termy 5y ago
I did my masters research in this field. I utilized a spiking neural network to classify images that I converted to spike trains. They are hard to train and as there was no sparse method to hold the images in the GPU the memory constraints were significant.

by jcims 5y ago

igouy 5y ago

First paragraph -- "The pixels in event-based cameras—a technology inspired by animal vision systems…"

nynx 5y ago

I was just thinking the same thing. This would would only work for biologically-plausible neuron variants, where neurons train constantly when being used. Maybe something like HTM (hierarchical temporal memory) neurons would work.

gumby 5y ago

There is a stealthy startup that at least originally was doing machine vision based on the human early visual system. You and I don’t see anything like pixels; our vision system pulls out relatively high level features like parallel lines, motion etc and starts working with that in addition to other optical data, muscle feedback on your lens shape etc. This is why many animals, including humans, freeze when frightened or when they perceive a risk: stationary objects are simply harder for most animals to pick out. The theory is (or was — I don’t know what they are doing today) is that you can see things interesting to humans and reason about them in ways interesting to humans. Not part of that company: there are some wild but interesting theories related to this kind of thing. We pretty clearly have face-finding hardware (though not in the early visual system I presume). Do animals have this? Cats and dogs look at human faces. I have read a theory that reading may hijack some subsystem originally used for identifying tracks (footprints). It does appear that recognizing 2D pictures is learnt, not innate, while 1990s/2000 ML went the other way. Some anthropomorphically-inspired design could yield systems that are more comprehensible and more useful. I wouldn't make a fetish of it (cars don't run faster than a human does) but machines should be better adapted to humans rather than the other way around (e.g. cars need special places to move around in so can only help humans when that is possible).

ngcc_hk 5y ago
Is the basic input still “pixel” and have we dropped these and maintain our neural network in … Yes it seems we do not see and think in pixel, but are we really not think in pixel even though we are not aware of them ?
- gumby 5y ago
  The concept of ‘pixel’ is only about 45 years old, and wasn’t even a concept in the age of type. If you look at the anatomy of the retina or the physiology of the early visual system you’ll see that the concept of “pixel” doesn’t even really make sense in that context.
  - franga2000 5y ago
    Doesn't it? My understanding is basic, but as far as I know we have discrete detector cells that respond to light hitting them. We have ones that respond to all light as well as different ones that respond to only certain wavelength ranges. They aren't aligned in a matrix, sure, but that's not what makes a pixel a pixel. We don't see the pixels because our consciousness works at a higher level of abstraction. You won't find pixel a few layers deep into a CNN either, but that doesn't mean they don't exist, the "neurons" at that layer just don't see them because they have no use for them. That's always been my understanding at least...

termy 5y ago

I did my masters research in this field. I utilized a spiking neural network to classify images that I converted to spike trains. They are hard to train and as there was no sparse method to hold the images in the GPU the memory constraints were significant.