3 ms·
Not sure about Neural DSP or reverbs in general, but real-time neural network based DSP seems very possible. The open source Neural amp modeler[1] would be a go
by landonxjames 2y ago
Not sure about Neural DSP or reverbs in general, but real-time neural network based DSP seems very possible. The open source Neural amp modeler[1] would be a good place to start diving in.
[1] https://www.neuralampmodeler.com/the-code https://www.neuralampmodeler.com/the-code
- prashp 2y agoI have tried NAM but with limited success in modeling some time-based effects (e.g. octave shifting). However, I have not tried to model reverb effects.
- intalentive 2y agoTo handle time-based effects you need a custom architecture. https://www.research.ed.ac.uk/en/publications/neural-modelling-of-periodically-modulated-time-varying-effects https://www.research.ed.ac.uk/en/publications/neural-modelli... Don’t use NAM. Learn PyTorch.
- prashp 2y agoNAM uses pytorch for its NN implementation?
- intalentive 2y agoIt is 100% possible and there are a slew of tricks you can use to get big performance boosts with negligible cost to accuracy.
- prashp 2y agoDo you know what the tricks are?
- intalentive 2y ago1. Don’t use LSTMs (4 vector-matrix multiplies) or GRUs (3 multiplies). Use a fixed Hippo matrix to update state. Just 1 multiply and since it’s fixed you can unroll during training, much faster than backprop through time. 2. Write SIMD intrinsics by hand. None of the libraries are as fast. 3. Don’t use sigmoid or tanh functions as your nonlinear activation. Instead approximate them with the softsign function which is much cheaper. Depends on exact architecture, but these optimizations have yielded 10-30x improvement for single threaded CPU real time audio applications. When GPU audio matures all this may be unnecessary.