5 ms·

any pedal? That seems far-fetched - one trivial counter-example would be a looper as that requires modal input and state. I also wonder if it can handle complex

any pedal? That seems far-fetched - one trivial counter-example would be a looper as that requires modal input and state. I also wonder if it can handle complex multi-tap delays. Can anyone give a rough idea of the actual limitations? I would guess that there is a limit to how non-local the effects it can manage are.

taneq 5y ago
If you want to be pedantic it probably can't emulate a guitar pedal that you can tap a representation of a Turing machine into and it will only make a sound if the machine halts.
- shmageggy 5y ago
  You can't create such a pedal in the first place, so...
  - jdbernard 5y ago
    Yep. Disprove the sweeping generalization by counterexample.
  - tiborsaas 5y ago
    The joke is that some pedals can be really complex with menu systems and all kinds of tricky jazz.
  - marcodiego 5y ago
    You definitely can. You just cannot prove it will always make a sound.
  - jcelerier 5y ago
    why couldn't you ? the code would just be extern char* turing_machine; int main() { process_turing_machine(turing_machine); beep(); }
- gberger 5y ago
  I think I've read something about this in a story involving Achilles and a tortoise.
ushakov 5y ago
you could model amplifiers, distortion effects, phasers and flangers, according to CoreAudioML project, which makes this possible https://github.com/Alec-Wright/CoreAudioML https://github.com/Alec-Wright/CoreAudioML
- duped 5y ago
  You can model most useful nonlinear functions with neural networks, this is unsurprising. You can also use Volterra series. You can even estimate/measure the Volterra kernel then train a NN to model it instead of dealing with the computational complexity of generalized convolution for nonlinear dynamic systems. The hard part is that there are some fundamental limitations to deal with. The biggest is aliasing - distortion effects in particular deal with enormous amounts of distortion (> 100% THD) which creates spectrums far outside the range of hearing. Digital audio systems need to have high orders of oversampling to prevent audible aliasing (8-16x is not unheard of!). After aliasing is memory. It's too early in the morning for me to do math but I'm almost certain you can't model a looper with a causal NN that has less internal state memory than the length of your loop. Doing so is dumb anyway, since loopers are pretty trivial and their biggest cost is memory. Same goes for digital delay and modulation effects, the algorithms are not expensive.
  - marcan_42 5y ago
    Now I wonder if a NN would be able to learn a nonlinear effect without aliasing, even if run at the original sample rate. Oversampling and filtering are, after all, things that could become part of the model too. Perhaps it can learn to approximate them with less CPU cost than doing it for real.
    - duped 5y ago
      Oversampling requires producing more output information than input information. It would be incredible for a NN to realize a system that could do this without requiring more memory and CPU cycles than a good oversampling algorithm, which can be derived analytically with various definitions of "optimal." A 3rd or 4th order polynomial interpolator is pretty darn good and doesn't need a NN to find the coefficients.
JD557 5y ago
They claim to be using a LSTM, and I believe that any RNN-like architecture should (in theory) be able to learn a loop pedal. If you aren't familiar with RNNs, think about it like a NN that instead of learning a input -> output function, learns a (input, state) -> (output, newState) function
- lunixbochs 5y ago
  Only if the entire loop audio fits into the RNN's hidden state? Or if you use some kind of external memory mechanism that gives the network access to previous audio.
  - titzer 5y ago
    This is the kind of thing where augmenting the NN with an actual raw audio buffer would allow it to mix in some of the signal from the past quite easily.
  - jeeceebees 5y ago
    LSTM stands for Long Short Term Memory. It's a recurrent network that learns what and how long things should be kept in its internal state buffer. It doesn't have a fixed state size because it's just learning a nonlinear function that takes an input and a state to an output and a new state. Obviously it can't model all possible, infinite length recurrences, but it can definitely do a pretty good job of approximating long term recurrence relations in complex signals.
    - lunixbochs 5y ago
      I don't think that assessment is quite right. The hidden size is fixed - the second argument to Pytorch's nn.LSTM constructor is "hidden_size – The number of features in the hidden state h". A call to `y, hidden = layer.forward(x)` (where x has a batch size of 1, and an arbitrary length) produces two hidden states of dimensions `(1, 1, hidden_size)`, where hidden_size is the exact number you passed to the LSTM constructor. Those two states represent the long term and short term memory features. You would need to have an LSTM with hidden_size large enough to store the samples (or a compressed representation) of your entire loop. Not to mention you'd run into other issues with handling the logic around variable length loops based on a pedal toggle.
keyth72 5y ago
..any pedal It can’t do time based effects such as delay, reverb, flange, chorus, etc. The LSTM can model distortion, overdrive, compression to an extent, and amp circuits (including vacuum tubes).
- keyth72 5y ago
  But I will say that there is a project out there for time varying effects, but haven’t dug into it yet. I believe it’s a different method than the LSTM https://github.com/Alec-Wright/NeuralTimeVaryFX https://github.com/Alec-Wright/NeuralTimeVaryFX

taneq 5y ago

If you want to be pedantic it probably can't emulate a guitar pedal that you can tap a representation of a Turing machine into and it will only make a sound if the machine halts.

shmageggy 5y ago
You can't create such a pedal in the first place, so...
- jdbernard 5y ago
  Yep. Disprove the sweeping generalization by counterexample.
- tiborsaas 5y ago
  The joke is that some pedals can be really complex with menu systems and all kinds of tricky jazz.
- marcodiego 5y ago
  You definitely can. You just cannot prove it will always make a sound.
- jcelerier 5y ago
  why couldn't you ? the code would just be extern char* turing_machine; int main() { process_turing_machine(turing_machine); beep(); }
gberger 5y ago
I think I've read something about this in a story involving Achilles and a tortoise.

ushakov 5y ago

you could model amplifiers, distortion effects, phasers and flangers, according to CoreAudioML project, which makes this possible https://github.com/Alec-Wright/CoreAudioML https://github.com/Alec-Wright/CoreAudioML

duped 5y ago
You can model most useful nonlinear functions with neural networks, this is unsurprising. You can also use Volterra series. You can even estimate/measure the Volterra kernel then train a NN to model it instead of dealing with the computational complexity of generalized convolution for nonlinear dynamic systems. The hard part is that there are some fundamental limitations to deal with. The biggest is aliasing - distortion effects in particular deal with enormous amounts of distortion (> 100% THD) which creates spectrums far outside the range of hearing. Digital audio systems need to have high orders of oversampling to prevent audible aliasing (8-16x is not unheard of!). After aliasing is memory. It's too early in the morning for me to do math but I'm almost certain you can't model a looper with a causal NN that has less internal state memory than the length of your loop. Doing so is dumb anyway, since loopers are pretty trivial and their biggest cost is memory. Same goes for digital delay and modulation effects, the algorithms are not expensive.
- marcan_42 5y ago
  Now I wonder if a NN would be able to learn a nonlinear effect without aliasing, even if run at the original sample rate. Oversampling and filtering are, after all, things that could become part of the model too. Perhaps it can learn to approximate them with less CPU cost than doing it for real.
  - duped 5y ago
    Oversampling requires producing more output information than input information. It would be incredible for a NN to realize a system that could do this without requiring more memory and CPU cycles than a good oversampling algorithm, which can be derived analytically with various definitions of "optimal." A 3rd or 4th order polynomial interpolator is pretty darn good and doesn't need a NN to find the coefficients.

JD557 5y ago

They claim to be using a LSTM, and I believe that any RNN-like architecture should (in theory) be able to learn a loop pedal. If you aren't familiar with RNNs, think about it like a NN that instead of learning a input -> output function, learns a (input, state) -> (output, newState) function

lunixbochs 5y ago
Only if the entire loop audio fits into the RNN's hidden state? Or if you use some kind of external memory mechanism that gives the network access to previous audio.
- titzer 5y ago
  This is the kind of thing where augmenting the NN with an actual raw audio buffer would allow it to mix in some of the signal from the past quite easily.
- jeeceebees 5y ago
  LSTM stands for Long Short Term Memory. It's a recurrent network that learns what and how long things should be kept in its internal state buffer. It doesn't have a fixed state size because it's just learning a nonlinear function that takes an input and a state to an output and a new state. Obviously it can't model all possible, infinite length recurrences, but it can definitely do a pretty good job of approximating long term recurrence relations in complex signals.
  - lunixbochs 5y ago
    I don't think that assessment is quite right. The hidden size is fixed - the second argument to Pytorch's nn.LSTM constructor is "hidden_size – The number of features in the hidden state h". A call to `y, hidden = layer.forward(x)` (where x has a batch size of 1, and an arbitrary length) produces two hidden states of dimensions `(1, 1, hidden_size)`, where hidden_size is the exact number you passed to the LSTM constructor. Those two states represent the long term and short term memory features. You would need to have an LSTM with hidden_size large enough to store the samples (or a compressed representation) of your entire loop. Not to mention you'd run into other issues with handling the logic around variable length loops based on a pedal toggle.

keyth72 5y ago

..any pedal It can’t do time based effects such as delay, reverb, flange, chorus, etc. The LSTM can model distortion, overdrive, compression to an extent, and amp circuits (including vacuum tubes).

keyth72 5y ago
But I will say that there is a project out there for time varying effects, but haven’t dug into it yet. I believe it’s a different method than the LSTM https://github.com/Alec-Wright/NeuralTimeVaryFX https://github.com/Alec-Wright/NeuralTimeVaryFX