4 ms·
This works for images because images can be decomposed into different scales: in classical image processing into image pyramids, and in this case using deep con
by versteegen 10y ago
This works for images because images can be decomposed into different scales: in classical image processing into image pyramids, and in this case using deep convolutional neural networks which capture progressively more complex features and larger scales. So you decompose an image, keeping the high level features fixed, and modify the image (using gradient ascent) to make the low level features match those from a sample of the artistic style.
So to use the same algorithm for music you would have to decompose audio in a similar meaningful way. There has also been a lot of success in speech recognition with CNNs lately, but I don't know what the situation is with modelling music.
- amelius 10y ago> So to use the same algorithm for music you would have to decompose audio in a similar meaningful way. Well, in music, scaling could be compared to increasing/decreasing frequency. We all know that a song which is transposed by e.g. an octave still sounds the same (albeit lower/higher). So I think the concept translates well from images.
- versteegen 10y agoThat doesn't sound like what I meant. The point of image pyramids is that they separate fine details (e.g. style) from coarse details (form). A note transposed down an octave is still exactly the same type of object; it's not more abstract so frequency is not an analog to scale in images. What you want is a progression of abstractions, e.g. note, chord, melody... but one where 1) each is largely orthogonal so that they can be separated and recombined with a lot of flexibility (definitely not true of notes and melodies) 2) it be computed straightforwardly, preferably by a differentiable function 3) can separate style and form.
- GFK_of_xmaspast 10y agoMutiresolution analysis (forex wavelets) is perfectly appropriate for time series data tho.