4 ms·

We propose an alternative approach to construct an artificial learning system, which naturally learns in an unsupervised manner. Its mathematical prototype is a

We propose an alternative approach to construct an artificial learning system, which naturally learns in an unsupervised manner. Its mathematical prototype is a dynamical system, which automatically shapes its vector field in response to the input signal. The vector field converges to a gradient of a multi-dimensional probability density distribution of the input process, taken with negative sign. The most probable patterns are represented by the stable fixed points, whose basins of attraction are formed automatically. The performance of this system is illustrated with musical signals. Could anyone eli5 this?

jamessb 11y ago
They estimated how likely particular notes were, using a kind of averaging procedure that corresponding to simulating an elastic foam. They used a fourier transform to identify the notes in a piece of music: they then simulated the physical procedure of adding a stone to the foam at a location corresponding to the frequency, and time corresponding to when the note was played. The final shape of the foam then gives a density estimate of the data: the more often a particular note was played, the deeper is the dip in the foam at that location ('the foam evolves into a time-averaged density of the input'). Once you've done this, you have a curved surface. You can imagine a ball being released at some point and rolling around: it could get stuck in the bottom of a well ('a stable fixed point'), which corresponds to the 'most probable patterns' in the training data. Edit to add: "The vector field converges to a gradient of a multi-dimensional probability density distribution of the input process, taken with negative sign" You can think of the "vector field" of the dynamical system describing the movement of the ball as a bunch of arrows, giving the direction the ball would move if released from each possible starting point. If you trace these arrows end-to-end, you can see the trajectories that the ball could take. Since the ball will roll downhill, the vector field of the system is (minus) the gradient of the potential energy/height. Also, because of how we formed the surface, its height 'converged to the probability density of the input process' (the more frequent an input was, the lower the corresponding dip in the foam). Thus, the vector field of the system converges to the gradient of the probability density of the input.
dsfsdfd 11y ago
It's almost exactly a rubber sheet with balls placed upon the surface. The weight of the balls creates depressions in the sheet. This is it really, except for two extra things: 1:When the balls are removed, the sheet slowly returns to flat. 2: The sheet isn't 2 dimensional but has as many dimensions as the input vector.

jamessb 11y ago

They estimated how likely particular notes were, using a kind of averaging procedure that corresponding to simulating an elastic foam. They used a fourier transform to identify the notes in a piece of music: they then simulated the physical procedure of adding a stone to the foam at a location corresponding to the frequency, and time corresponding to when the note was played. The final shape of the foam then gives a density estimate of the data: the more often a particular note was played, the deeper is the dip in the foam at that location ('the foam evolves into a time-averaged density of the input'). Once you've done this, you have a curved surface. You can imagine a ball being released at some point and rolling around: it could get stuck in the bottom of a well ('a stable fixed point'), which corresponds to the 'most probable patterns' in the training data. Edit to add: "The vector field converges to a gradient of a multi-dimensional probability density distribution of the input process, taken with negative sign" You can think of the "vector field" of the dynamical system describing the movement of the ball as a bunch of arrows, giving the direction the ball would move if released from each possible starting point. If you trace these arrows end-to-end, you can see the trajectories that the ball could take. Since the ball will roll downhill, the vector field of the system is (minus) the gradient of the potential energy/height. Also, because of how we formed the surface, its height 'converged to the probability density of the input process' (the more frequent an input was, the lower the corresponding dip in the foam). Thus, the vector field of the system converges to the gradient of the probability density of the input.

dsfsdfd 11y ago

It's almost exactly a rubber sheet with balls placed upon the surface. The weight of the balls creates depressions in the sheet. This is it really, except for two extra things: 1:When the balls are removed, the sheet slowly returns to flat. 2: The sheet isn't 2 dimensional but has as many dimensions as the input vector.