3 ms·
For analog pedals, I'd expect better results from circuit modeling. Given the wide (and very non-linear) range of settings of a typical pedal, as well as inter
by musicale 5y ago
For analog pedals, I'd expect better results from circuit modeling.
Given the wide (and very non-linear) range of settings of a typical pedal, as well as interaction (impedance, etc.) with a real guitar and amplifier, it seems like it would be a pain to get all of the training data.
For a digital pedal, running the actual (e.g. Eventide) DSP code is just going to be better than some ML approximation.
On the other hand, I've been a bit dissatisfied with amplifier and cabinet models based on traditional DSP and physical modeling approaches, so maybe neural networks could fill in some of the gaps.
- sillysaurusx 5y agoML dilettante here! I wouldn’t discount ML. The nonlinearities are the bread and butter of modern ML models. In fact, two linear layers without a nonlinearity inbetween is equivalent to one big linear layer. So nonlinearities are required. To put it another way, I would gladly bet any reasonable sum of money that in a double blind test, the listener wouldn’t be able to tell the difference from a genuine guitar pedal. (Not necessarily this pedal, but I suspect ML will model the effects more than adequately for human hearing precision.) FWIW, I say this as someone who used to argue that graphics programmers were doing gamedev all wrong because they weren’t modeling light, they were approximating light. ML models were the way out. I also think much of the problem is that ML devs often don’t have traditional signal processing experience, so they haven’t been modeling signals in quite the right way. (I’m trying to rectify that a bit with my FFT tutorials: https://twitter.com/theshawwn/status/1398796224921321472?s=21 https://twitter.com/theshawwn/status/1398796224921321472?s=2...) It remains to be seen, but Fourier space has recently been making strides in ML, and it’s likely much easier for a model to approximate a nonlinear waveform in frequency space than as a raw waveform. To put it another way, if human speech is getting to the point where ML models can trick people, what are the chances that a future model won’t be able to do it for guitars?
- ushakov 5y agoactually, the researchers of aalto university (now at neural dsp), who pioneered the guitar ml technology, were working on speech initially and did this one as a side project source: https://m.youtube.com/watch?v=WLTzbEKTxhk https://m.youtube.com/watch?v=WLTzbEKTxhk
- nyanpasu64 5y agoI've heard images are better modeled in DCT space (which isn't based on complex numbers) because it's better at energy compaction than FFT, and also because it doesn't assume that the image is periodic. Also some people think that the FFT is insufficient, even for audio, because it doesn't model time-domain hearing perception. Some people say that wavelets are better at modeling images than purely frequency-domain transforms, because they take spatiality more into account. From what I've heard, wavelets work well for modeling human vision (in fact convolutional neural network input kernels tend to converge to Gabor filters, which I don't know howw they differ from Gabor wavelets) and noise reduction, but have fallen flat for image/video compression codec design.
- sillysaurusx 5y agoAll excellent points, and I think you should DM me on twitter to chat about this more. (I hope you will!) DCT is on my radar. But there are several serious limitations that I think are overlooked. For example, convolution is no longer a simple component-wise multiplication. This seems, to me, a big deal. Complex numbers are tricky to model, but I think most people have given up too easily, or haven't been creative enough in how they're modeling them. Some of my (outdated) ideas: https://gist.github.com/shawwn/c6865fccafac5066e1c7bab6727814e0 https://gist.github.com/shawwn/c6865fccafac5066e1c7bab672781... In other words, you're probably right, but I'm focused solely on FFTs on the (very low) chance that people have overlooked something that will work well.
- nyanpasu64 5y agoSorry I don't work on neural networks much, and have my plate too full with other projects (and my DSP is a bit rusty?) to hold a conversation on this right now. And I don't use Twitter much either. Maybe we can talk later? Not sure.
- sillysaurusx 5y agoNo worries :) it was just an offer. It surprised me how much you knew about the domain. Good luck with your projects!
- musicale 5y ago> The nonlinearities are the bread and butter of modern ML models I guess I didn't get my point across. What I meant was that pedal settings tend to be non-linear with multiple sweet spots (which often depend on the guitar and amp) so you shouldn't just do a linear range from 1-10^N (where N is the number of knobs) for training data, as someone else had suggested. Moreover, there are also dependencies on the impedance chain, gain structure, feedback, reflections, etc., which seem well-suited to circuit and physical modeling. Digital pedals, as I note, are largely software anyway so it doesn't make sense to me to try to model them with ML any more than it does to model Microsoft Word using ML (though I'm sure someone has tried.) In general ML seems most useful when you don't have good analytical models - but in the case of circuits and software we have very good analytical models.
- sillysaurusx 5y agoThat's fair, and true! But one interesting thing about ML models is that they're often much more performant. For example, it's relatively expensive to evaluate analog circuits digitally. An ML model that can do it on a raspi with no delay and no quality loss is interesting, to me at least.
- ushakov 5y agoin disbelief? See "4.4. Listening Tests" in the paper the technology is based on https://acris.aalto.fi/ws/portalfiles/portal/41964332/Real_time_guitar_amplifier_emulation.pdf https://acris.aalto.fi/ws/portalfiles/portal/41964332/Real_t... those are MUSHRA tests, meaning only skilled listeners are allowed to participate https://en.wikipedia.org/wiki/MUSHRA https://en.wikipedia.org/wiki/MUSHRA TLDR: the resulting neural networks for given amplifier models were rated as “excellent” by the listeners, some even outperformed the reference!
- texasbigdata 5y agoNeural DSP sounds pretty good to me, and they’re new. Source: https://youtu.be/MhnkGMBn1tw https://youtu.be/MhnkGMBn1tw
- alexjplant 5y ago> On the other hand, I've been a bit dissatisfied with amplifier and cabinet models based on traditional DSP and physical modeling approaches, Have you tried the Fractal stuff? I've been using it since the first generation (consistently for live use since the Axe-Fx II days) and they've been ahead of the competition since their inception. At this point I'd venture to say that the majority of their amp models sound indistinguishable from the real thing with no advanced parameter tweaking. That being said when I was between Fractal units about a year ago I spent a brief amount of time with the Pod Go and was immensely impressed with how much they were able to pack into a $500 floor unit. Most of the amps and drives still felt a bit caricature-y but were still very usable - a far cry from the Pod Bean days. It's truly a great time to be a musician.