15 ms·

Neural networks emulate any guitar pedal for $120

286 points by ushakov 5y ago

duped 5y ago
The cool bit about this to me is that $120 in hardware is a the one-off prototyping cost. As a product it could be made much cheaper. Another benefit to a software defined pedal is that it can express sounds that cannot be replicated in analog. Emulation is boring. Train it to do stuff that I can't buy in a pedal!
- ushakov 5y ago
  but the coolest thing is that you can share and download models of one-of-a-kind amps or effect chains, which cannot be bought
  - TrackerFF 5y ago
    That's the business model of Kemper (https://www.kemper-amps.com/ https://www.kemper-amps.com/)
    - ushakov 5y ago
      yes, except you don't need to purchase a $1000+ device this thing runs on consumer-grade hardware
  - duped 5y ago
    Line6 was doing this 10-15 years ago and it wasn't that cool then. Eventide has some tools for it today tool, there's also the Owl, various teensy projects, and of course Kemper.
    - leviathant 5y ago
      I've had a Line 6 Guitar Port for about 20 years. It was $99, and emulates amps, cabinets, pedals, console channels, rack effects, etc., admittedly offloading the work into a computer that (at the time) cost about a grand. Are they perfect emulations? No. Did anyone notice that in my music? No. Would I take that rig on stage? Also no. It did help me find sounds I liked though, and over the years I've bought hardware equivalents of some of my favorite emulations, and I've bought hardware that goes beyond anything Line 6 can do. As to whether Line 6 is cool or not, NIN/Trent Reznor toured with Line 6 rigs to sold out arenas 20 years ago, and today the live rigs are managed by the FOH using software emulations that can be automated or managed from on-stage controllers. Maybe you don't think that's cool, but the important takeaway is that you should use the tool that's right for you. I think it's good to see people tinkering with new ways to reduce the cost around modulating audio signals in interesting ways.
EamonnMR 5y ago
There's no link to the vst. It would be neat if it was something you could just drop into your DAW and try to train it in wacky effects chains. The artifacts in it's sound might be really neat.
- ushakov 5y ago
  check out https://github.com/GuitarML https://github.com/GuitarML
ushakov 5y ago
For context, here is the discussion of a previous iteration of the technology on HN https://news.ycombinator.com/item?id=24740266 https://news.ycombinator.com/item?id=24740266
krueger71 5y ago
I'm guessing a Hi-Z input before this contraption could improve the tone and do the pickups of the guitar or bass justice. A dedicated buffer pedal or just a pedal with buffer bypass perhaps?
- sirtaj 5y ago
  Yes I was surprised to see the guitar signal going in directly. A simple DI box would improve the sound noticeably.
- DarrisMackelroy 5y ago
  Most definitely. A radial reamp DI would improve the tone on this rig noticeably. Even if you didn’t use reamp specific hardware, using 2 direct boxes to match impedance on the input and output of the Pi would help.
zabil 5y ago
This is really cool. I wonder if this can be used to simulate pianos, rhodes and wurlitzer sounds.
- nonsapreiche 5y ago
  I wonder too, but I think the result will not be better than the samples from which you train it.
  - nonsapreiche 5y ago
    but could be trained to simulate the sympathetic resonance in chords.. https://en.wikipedia.org/wiki/Sympathetic_resonance https://en.wikipedia.org/wiki/Sympathetic_resonance
  - ratww 5y ago
    Now that you mentioned samples: I wonder if neural networks will be able to help with polyphonic note detection, so we can trigger MIDI samples using an off-the-shelf guitar or other instruments. There have been a few recent advancements lately (Boss SY-1), but even the supposedly "ideal" solutions, that require a new polyphonic pickup, are not good at all. I have a Fishman Triple Play and a plugin whose name I forgot, and tracking is frankly terrible.
    - nwatson 5y ago
      In which Pat Metheny tries to replace his whole band with guitar-triggered control of real, non-guitar instruments (keyboards, drums, etc) ... "Orchestrion" ... https://youtu.be/KsYEOUKS4Yk https://youtu.be/KsYEOUKS4Yk
      - ratww 5y ago
        Yeah, it's honestly not that good. I think Metheny uses an Axon. Even with those you need to be very careful with your phrasing, you can't be too fast, you lose a lot of dynamic range and a lot of expressivity, sometimes notes just die, the latency is high... Here's what Metheny said himself: "But the guitar‑to‑MIDI part has always been a problem. It's a question of physics. On input, I sort of have to rush. But I know how to rush. I play ahead." It's fun for lots of things, and you can make lots of cool music, but it's still very limited to certain styles, dynamic ranges, phrasing, tempo, speeds...
IAmGraydon 5y ago
The video won’t play for me, but does this allow tweaking of parameters? The problem with most projects like this one is that it replicates the sound of a certain pedal with certain settings. That’s a major problem for most guitarists as playing with the knobs live is a big part of the attraction to pedals.
- ushakov 5y ago
  not quite there yet, but on possibility is train multiple models and switch between them on the fly another is add more params to the models to take the knob states into the account when doing predictions (which can impact the performance)
- FrankyFire 5y ago
  No, it doesn't. But to be honest, this would be next level. When looking at this, I'm thinking of alternatives and I can only name 2: 1 - Kemper Profiler 2 - NeuralDSP Both of them are above 1000€/$. We are talking about 10x the price of this thing. Add some Multi-FX Pedal (like Line6 HX-Stomp) where you put this in the FX-Loop and you end up with something equally good for still half the price. And in general, its not about what is better, digital or analog. It's about the use-case. In the studio or when noodling around with the knobs when practicing: Real Amps and Pedals. But on stage, you don't play with the settings of your pedals or you want presets. This is where you go digital. No one will notice the slightly different sound there anyway.
  - IAmGraydon 5y ago
    I played out live for a decade and absolutely used the knobs on my pedals live. Most musicians I knew with a pedalboard did the same. That said, if you use a preset based effects setup like rackmount gear, I could see how this would be cool.
    - FrankyFire 5y ago
      Yeah, true. But you would only change it between songs, I guess, and only to the values you always use for that song, right? - This is what I would capture and use as a preset, preset change triggered by MIDI from the MultiFX unit. I know some who change their settings during playing, but I wouldn't consider this as standard. Also, you can still add a real Pedal to a MultiFX Board if you really want to change its settings live. On the contrary, automatically changing the settings controlled by your DAW and triggered via MIDI into your MultiFX unit is also pretty common. You don't have to hassle with your Effects at all and can concentrate on playing, what you should, as you need to be absolutely on point.
andybak 5y ago
any pedal? That seems far-fetched - one trivial counter-example would be a looper as that requires modal input and state. I also wonder if it can handle complex multi-tap delays. Can anyone give a rough idea of the actual limitations? I would guess that there is a limit to how non-local the effects it can manage are.
- taneq 5y ago
  If you want to be pedantic it probably can't emulate a guitar pedal that you can tap a representation of a Turing machine into and it will only make a sound if the machine halts.
  - shmageggy 5y ago
    You can't create such a pedal in the first place, so...
    - jdbernard 5y ago
      Yep. Disprove the sweeping generalization by counterexample.
    - tiborsaas 5y ago
      The joke is that some pedals can be really complex with menu systems and all kinds of tricky jazz.
    - marcodiego 5y ago
      You definitely can. You just cannot prove it will always make a sound.
    - jcelerier 5y ago
      why couldn't you ? the code would just be extern char* turing_machine; int main() { process_turing_machine(turing_machine); beep(); }
  - gberger 5y ago
    I think I've read something about this in a story involving Achilles and a tortoise.
notum 5y ago
As much as I'm torn with my feelings about nVidia as a company, especially recently with their attempts to artificially limit hardware you own, I must admit Jetson family is incredibly capable and very well executed. I've been using Jetsons as RasPi replacements wherever I can, they are not only more capable but also much more reliable than RPI4s, in my not so limited experience.
- Bancakes 5y ago
  Extremely expensive. Or overpriced?
  - notum 5y ago
    I wouldn't say that. 2GB dev kit is $50 depending on where you buy them. That's much more bang for the buck, even if you opt out using the tensor cores they were designed for.
    - 1996 5y ago
      They sorely lack m2 for high throughput storage. A board with a m2 port costs an extra $300 and only exists since last year: https://antmicro.com/blog/2020/04/updated-jetson-nano-xavier-nx-baseboard-m2/ https://antmicro.com/blog/2020/04/updated-jetson-nano-xavier... It should have been standard and by default from the beginning.
      - notum 5y ago
        M2 port would have been awesome! I believe the comment stating Jetsons were expensive was made in comparison with the RPI4, which also lacks an m2 slot. I really do find Jetons much more affordable when performance is taken into account, again, Tensor aside. They can also boot from USB3 as of recently, which boosts storage access speed tremendously.
Applejinx 5y ago
…at MP3 quality. This is fine if you like that sort of thing. I would note that latency is very important here: it's not going to be nice to play through if it's incurring any significant latency.
- montroser 5y ago
  Yes, latency is huge. Modern digital audio stacks in consumer OSes are still completely terrible at this. Not that it's an easy problem to solve. But it's pretty hard to beat elections flowing through an analog circuit, when in the digital side you have to: convert analog to digital, run through the kernel to get to user space, run the bits through the RNN, send back to kernel space, convert to analog and finally send to an output. In order to be competitive with a 1980s guitar pedal, you have to do all of that in under ~10ms latency, and that's just really hard still. Even though we carry around these super computers in our pockets these days, there are still some things left where analog still beats the pants.
  - hashkb 5y ago
    Also, the competition is not against vintage analog gear. Modern analog gear is having a true Renaissance, and this community can afford to support the Wampler's of the world. I don't understand the modelling camp at all, their stuff just doesn't sound good, nor is there any joy in working with it.
    - Applejinx 5y ago
      I feel the same but it's hell convincing the reductionists that they haven't got 'the thing, and the whole of the thing' in their little emulation. There's also another element: if you have, say, a vintage Fender Champ and a Klon (or whatever) it's because you mean to project different expressions through your string handling and note-playing. At that level you've made a best effort to produce the most emotionally transparent and responsive signal chain, which you will then not think about once you've got it turned on and tweaked: ALL the settings are liable to sound 'good' and respond for you. The modeling approach is so often "This is exactly that, but better, because here are twelve other Fender Champs and models of Klon to choose from!" and when the first claim isn't as true as we would like, and the second is a distraction and time-sink, that's not great. I can tell when I've chosen wrongly in my music-making tools, because I flat-out stop making music. Even in a dilettantish way: it just stops being a thing. That's a concern.
fermigier 5y ago
Uli Behringer in a box ;)
ushakov 5y ago
have i mentioned yet that training only takes ~30 minutes and you only need ~3 minutes of data?
- jtriangle 5y ago
  Could it work on a live amp like the Kemper or a quad cortex? And can the models be exported and shared? Because if those two things were squared away, I could see this being an extremely viable project.
  - ushakov 5y ago
    yes! for training the model you’d need a separate computer/server though
deleted 5y ago
[deleted]
TrackerFF 5y ago
Sure, but how good will it sound? High quality AD/DA converters alone will cost.
hashkb 5y ago
Can any skeptics weigh in? To me, this kind of tech is centuries away from fooling tone hounds.
- Applejinx 5y ago
  Not much need to (or benefit in doing so). If you're somebody like AnalogMan, building pedals with careful selection of NOS transistors wired point-to-point, this is so utterly not a threat to you. I've seen entirely analog circuits designed and built with the PCB parts and techniques used in these devices, that are miles away from the quality of good pedals, without even being digital or modeling at all. I'm sure the problem circuits measure completely fine, and then you A/B them with a truly great pedal and it's chalk and cheese. If you try to argue the point, people committed to the digital modeling model get quite fierce, so tone hounds learn to just roll their eyes and not engage. You can also think of it as learning to rely on secret weapons.
  - 52-6F-62 5y ago
    > You can also think of it as learning to rely on secret weapons Case in point: the old Neve desks
- PaulDavisThe1st 5y ago
  "tone hounds"? There are many hundreds to many thousands of pedals. Everyone has their own favorites. There is no "right" in this world. There's just as much chance that whatever "error" a modelled pedal may embody is preferred by its users as there is that it will be deemed worse.
- jtriangle 5y ago
  Andersons has a few YouTube videos where the Kemper profiler fools tone hounds enough that you can infer that it's already good enough. The NeuralDSP hardware is of at least similar quality, maybe better in some instances. Sure if you run a null test these will fail, but in real life it's really up to how honest the tone hound in question is, and if you can trap them into being honest. If you're talking product viability, something like the Pepsi challenge, double blind testing, would probably be effective marketing.
magicalhippo 5y ago
I recently watched a video[1] by Rhett Shull where he compared Neural Capture of a Quad Cortex[2] to real pedals. I'm not a musician, but even I could easily tell the difference a lot of the time. It didn't sound bad, but there were clearly aspects the Neural Capture failed to, well, capture. Still, early days. [1]: https://www.youtube.com/watch?v=4SSOJxZHolc https://www.youtube.com/watch?v=4SSOJxZHolc [2]: https://neuraldsp.com/quad-cortex https://neuraldsp.com/quad-cortex
- ushakov 5y ago
  GuitarML have a similar video as well https://www.youtube.com/watch?v=xEOFz3UcDyA https://www.youtube.com/watch?v=xEOFz3UcDyA
  - magicalhippo 5y ago
    Similar-ish but clearly very different. I noticed that in that video he was using the WaveNet model, though apparently the statefull LSTM model performs better[1]. Though even for that you can hear[2] a fairly clear difference. [1]: https://towardsdatascience.com/neural-networks-for-real-time-audio-raspberry-pi-guitar-pedal-bded4b6b7f31 https://towardsdatascience.com/neural-networks-for-real-time... [2]: https://towardsdatascience.com/neural-networks-for-real-time-audio-stateful-lstm-b534babeae5d https://towardsdatascience.com/neural-networks-for-real-time...
marcodiego 5y ago
Isn't any reasonably good DSP able to do same? Or is it learning automatically to mimic the effect?
martinvol 5y ago
Wow, fantastic. Amazing work!
kazinator 5y ago
Emulate 20 transistors using hundreds of millions of transistors!
- jtriangle 5y ago
  20 especially shitty transistors to be fair.
- BookPage 5y ago
  Thanks, this made me laugh out loud.
wintermutestwin 5y ago
It is my understanding that capture based emulation of a given sound seems like a better idea than it really is. Say you capture a given amp or pedal sound - even if you get the latency down to acceptable #s, what if you want/need to turn a knob? Tweaking knobs is an essential part of the process of dialing in a sound relative to your guitar and output environment. Contrast with the Fractalaudio approach of modeling each component of a device. Fractal's AxeFx is the gold standard and any geek would gush over the HW and SW engineering. The best part is that the company owner keeps improving his algos and pushing out updates for free. This device costs the equivalent of a good amp head and is loaded with more amps and effects than any of the competition. Sorry if this sounded like an ad, but I am always surprised how little airtime this amazing product gets in hacker circles.
- kazinator 5y ago
  The knobs can be included in the emulation. So that is to say, you model the unit as something having an audio input, and several controllers, such as knobs or switches. You then capture the behavior for all combinations of controller values, as part of the model. ("All combinations", for a potentiometer, might mean stepping it from 0 to 11 in increments of 1.) The emulation ased on the model runs on a piece of hardware which has some generic controllers on it: some rotary encoders, a couple of switches and whatnot. These get assigned to the parameters of the model. You could have a MIDI input on it, and use MIDI controllers, which would be cool. There are MIDI foot controllers that you can tilt with your foot to vary a parameter. The different models could be assigned to MIDI program numbers. You could change the patch number with the foot controller, and vary the parameters with it also. The foot controller might have, say, only two pedals, so you have to assign which ones you want: if the patch has five parameters, you have to fix the values of three of them and map the two important ones to the foot pedals you have. For the others, you can bend down and tweak the knobs on the unit itself.
- ushakov 5y ago
  the problem with this approach is you need to have knowledge about the circuit to make it happen, not only that, but you also need people, who can reconstruct the circuit digitally this approach is not scalable (that's why the high cost) the ML approach doesn't require any knowledge about the system (black-box) to produce the result
archsurface 5y ago
I bet not the miku.
carlob 5y ago
I wonder how it deals with dynamic range. I've always been told that one shouldn't run an electric guitar directly through a stereo because the peaks might fry it. So I would expect that whatever comes out of the RPi doesn't have much dynamic range anyway so computer speakers would be ok, but sound would be kinda meh. Am I missing something?
duosonic 5y ago
Honest question: if you're a musician, what is the appeal of digital modeling? Is it purely affordability/accessibility, or are you drawn to it because it would create different sonic possibilities that you couldn't get from the original?
- URSpider94 5y ago
  You mean, besides the appeal, what is the appeal? As an artist and a musician, you can have access to close approximations of sounds that would require a warehouse full of amps and dozens of pedals, costing tens of thousands of dollars - all in a Pi project box. That’s not just a little bit of affordability and convenience.
  - duosonic 5y ago
    And I certainly don't mean to sound dismissive about the possibilities that creates! It's a huge factor and I would see it being akin to how the rise of "pro-sumer" home recording equipment played a huge role in underground punk music.
- humbledrone 5y ago
  Speaking for myself, the accessibility and convenience is a huge part. Instead of lugging around a 60 pound fragile finnicky pedalboard, I just throw my Helix in a backpack and am good to go. It has way more pedals in it than I could fit on a real pedal board, and I can also switch presets with the tap of my foot (including rewiring all the connections between pedals, changing their order, swapping pedals, swapping amps, etc). And yeah the sonic possibilities are endless. On a physical pedalboard, it's pretty involved to rewire everything to change the routing, or add/remove pedals, etc. With the digital modeling ones, this all becomes trivial and you can try all kinds of different setups much more quickly, save them and go back later, share them with friends, etc. All that said, it's kind of like asking an acoustic guitarist in the 1950s why they would use an electric guitar. Electric guitars obviously have lots of advantages, but it's not like people stopped playing acoustic guitars. I still think real analog pedals are cool, they're fun to collect, in some cases they sound better, etc. And sometimes you don't need the mega-flexibility -- if you have a 4-pedal setup that does "your tone" maybe that's all you ever need.
musicale 5y ago
For analog pedals, I'd expect better results from circuit modeling. Given the wide (and very non-linear) range of settings of a typical pedal, as well as interaction (impedance, etc.) with a real guitar and amplifier, it seems like it would be a pain to get all of the training data. For a digital pedal, running the actual (e.g. Eventide) DSP code is just going to be better than some ML approximation. On the other hand, I've been a bit dissatisfied with amplifier and cabinet models based on traditional DSP and physical modeling approaches, so maybe neural networks could fill in some of the gaps.
- sillysaurusx 5y ago
  ML dilettante here! I wouldn’t discount ML. The nonlinearities are the bread and butter of modern ML models. In fact, two linear layers without a nonlinearity inbetween is equivalent to one big linear layer. So nonlinearities are required. To put it another way, I would gladly bet any reasonable sum of money that in a double blind test, the listener wouldn’t be able to tell the difference from a genuine guitar pedal. (Not necessarily this pedal, but I suspect ML will model the effects more than adequately for human hearing precision.) FWIW, I say this as someone who used to argue that graphics programmers were doing gamedev all wrong because they weren’t modeling light, they were approximating light. ML models were the way out. I also think much of the problem is that ML devs often don’t have traditional signal processing experience, so they haven’t been modeling signals in quite the right way. (I’m trying to rectify that a bit with my FFT tutorials: https://twitter.com/theshawwn/status/1398796224921321472?s=21 https://twitter.com/theshawwn/status/1398796224921321472?s=2...) It remains to be seen, but Fourier space has recently been making strides in ML, and it’s likely much easier for a model to approximate a nonlinear waveform in frequency space than as a raw waveform. To put it another way, if human speech is getting to the point where ML models can trick people, what are the chances that a future model won’t be able to do it for guitars?
  - ushakov 5y ago
    actually, the researchers of aalto university (now at neural dsp), who pioneered the guitar ml technology, were working on speech initially and did this one as a side project source: https://m.youtube.com/watch?v=WLTzbEKTxhk https://m.youtube.com/watch?v=WLTzbEKTxhk
patfla 5y ago
I stopped reading at the first sentence: "It’s a well-established fact that a guitarist’s acumen can be accurately gauged by the size of their pedal board- the more stompboxes, the better the player." As both a software engineer and a guitarist, I'd say the opposite is true. Or at least truer. You can't do math-rock without a lot of pedals but the hard part is to acquire the chops. A lot of pedals, and production effects generally, quickly become cliches and it's like dropping down a musical black hole. Someone like Hendrix could take a new effect (superset of pedals) and make it work musically brilliantly but most pedal users buy the pedal to get someone else's sound.
- ushakov 5y ago
  don’t take it too literally, i’m pretty sure this was meant as a sarcastic joke by the author :D
  - patfla 5y ago
    Maybe I should have read past the first sentence. The joke's on me.
- awsanswers 5y ago
  I believe that was sarcasm
- pier25 5y ago
  If it wasn't obvious this was a joke, the following paragraph starts with: "Jokes aside..."
- deleted 5y ago
  [deleted]
- u_gene 5y ago
  ..but most pedal users buy the pedal to.. show off in-front of others who don't have that pedal. period. GarageBand has 1000 of pedals, people don't need more pedals, they need to practice more, but they don't understand that. That's why they are buying more pedals while keep dragging about how much they care about green/clean planet.
jimnotgym 5y ago
In my attic there is a dusty box with a multi effects unit from the 90's. The reason I stopped using it is that it is not that the fuzz doesn't react to the guitars volume pot like a Fuzz Face or because the Tubescreamer was a TS9 not an 808. It was because having a single box with all of your effects in is a faff to tinker with. I like the dedicated hardware of my pedals, I like having the right number of knobs. I like being able to turn off the fuzz but keep the delay by stepping on the fuzz switch. I like being able to run the TS before or after the fuzz and to see it happen. I mentally am much more at home with little boxes for each stage, it is like a real world flow chart!
- bozzcl 5y ago
  You can do all that with modern multi-effects, and even more. I'm running a hybrid setup, using a Line 6 Helix in front of a Soldano SLO 30 (and on its effects loop as well). Each effect can be adjusted (with all sorts of knobs, more than 12 for some), moved around and triggered individually or together. You can even use the Helix to toggle channels in the amp, manually or as part of presets. It drastically eased up my workflow to experiment with and mix effects. No more tinkering with cables. No more wondering which pedal is plugged in wrong or doesn't have enough power. No more velcro strips. Plus, selling my hoard of pedals felt nice.
jimnotgym 5y ago
Just a quick note, you could solder together most classic guitar pedals for $20. You could buy reissues of many of them for less than $100. Big Muff, Proco Rat, Klon Centaur (Klones), Tubescreamer, Dynacomp, etc are all available for less than that. Just being clear that this is not a price issue.
- geedy 5y ago
  Each though.
u_gene 5y ago
Yo People!! A friend keeps sending links to guitar pedal discussions to me and I thought I chip in to the conversation. While you can emulate any guitar pedal with AI and you don't have to pay 120$ I'd like to introduce you to some understanding. MUSICIANS love to purchase shit they can show off to others. it's in their ego nature. While 120$ for software might compete with quad pro cortex pedal, you can't show off with 120$ but with a pedal you bought fo 2g you can. That is why BOSS made a big travel box for physical pedals this year, this is what they released at the NAMM2021. How do I know that? I make software for them. GarageBand has 1000s of different pedals already and those are free. most musicians have apple or abelton live or some similar. Also, not to discourage anyone - live music is dying. More and more you see musicians dancing on the stage with their instruments pretending they are playing, while pro-edited backtrack is going through the speakers. If you want to make this world a better place think how to improve housing/farming/healthcare with technology. Things like roof above the head, food on the table will be always in demand. There is no money to be maid with musicians, cause they don't have any. it took me few years to realize, you can learn on my mistakes. If you have some cash and time and thinking if you need to build some software product - kill that, buy some land and build a cabin. In the end you'll have another property. you can sell it or rent it. you can live it to your kids, there is value there. There is no value in your software idea unless people use it like crazy, and they really won't cause they already have 100 pedals and that new pedal bag from boss
n4kana 5y ago
I want to know if anyone ever uses this to create a fleet of classic microphone sounds from a working mic like the SM57. Sennheiser, Neumann, Telefunken, AKG. Hell, you could stack it with preamp filters like Neve. It would bring me great pleasure to see a studio chock full of SM57s masquerading as the best microphones money can buy.
- ushakov 5y ago
  i saw people simulating studio-grade compressors, artifacts included

286 points by ushakov 5y ago

The cool bit about this to me is that $120 in hardware is a the one-off prototyping cost. As a product it could be made much cheaper. Another benefit to a software defined pedal is that it can express sounds that cannot be replicated in analog. Emulation is boring. Train it to do stuff that I can't buy in a pedal!

ushakov 5y ago
but the coolest thing is that you can share and download models of one-of-a-kind amps or effect chains, which cannot be bought
- TrackerFF 5y ago
  That's the business model of Kemper (https://www.kemper-amps.com/ https://www.kemper-amps.com/)
  - ushakov 5y ago
    yes, except you don't need to purchase a $1000+ device this thing runs on consumer-grade hardware
- duped 5y ago
  Line6 was doing this 10-15 years ago and it wasn't that cool then. Eventide has some tools for it today tool, there's also the Owl, various teensy projects, and of course Kemper.
  - leviathant 5y ago
    I've had a Line 6 Guitar Port for about 20 years. It was $99, and emulates amps, cabinets, pedals, console channels, rack effects, etc., admittedly offloading the work into a computer that (at the time) cost about a grand. Are they perfect emulations? No. Did anyone notice that in my music? No. Would I take that rig on stage? Also no. It did help me find sounds I liked though, and over the years I've bought hardware equivalents of some of my favorite emulations, and I've bought hardware that goes beyond anything Line 6 can do. As to whether Line 6 is cool or not, NIN/Trent Reznor toured with Line 6 rigs to sold out arenas 20 years ago, and today the live rigs are managed by the FOH using software emulations that can be automated or managed from on-stage controllers. Maybe you don't think that's cool, but the important takeaway is that you should use the tool that's right for you. I think it's good to see people tinkering with new ways to reduce the cost around modulating audio signals in interesting ways.

EamonnMR 5y ago

There's no link to the vst. It would be neat if it was something you could just drop into your DAW and try to train it in wacky effects chains. The artifacts in it's sound might be really neat.

ushakov 5y ago
check out https://github.com/GuitarML https://github.com/GuitarML

ushakov 5y ago

For context, here is the discussion of a previous iteration of the technology on HN https://news.ycombinator.com/item?id=24740266 https://news.ycombinator.com/item?id=24740266

krueger71 5y ago

I'm guessing a Hi-Z input before this contraption could improve the tone and do the pickups of the guitar or bass justice. A dedicated buffer pedal or just a pedal with buffer bypass perhaps?

sirtaj 5y ago
Yes I was surprised to see the guitar signal going in directly. A simple DI box would improve the sound noticeably.
DarrisMackelroy 5y ago
Most definitely. A radial reamp DI would improve the tone on this rig noticeably. Even if you didn’t use reamp specific hardware, using 2 direct boxes to match impedance on the input and output of the Pi would help.

zabil 5y ago

This is really cool. I wonder if this can be used to simulate pianos, rhodes and wurlitzer sounds.

nonsapreiche 5y ago
I wonder too, but I think the result will not be better than the samples from which you train it.
- nonsapreiche 5y ago
  but could be trained to simulate the sympathetic resonance in chords.. https://en.wikipedia.org/wiki/Sympathetic_resonance https://en.wikipedia.org/wiki/Sympathetic_resonance
- ratww 5y ago
  Now that you mentioned samples: I wonder if neural networks will be able to help with polyphonic note detection, so we can trigger MIDI samples using an off-the-shelf guitar or other instruments. There have been a few recent advancements lately (Boss SY-1), but even the supposedly "ideal" solutions, that require a new polyphonic pickup, are not good at all. I have a Fishman Triple Play and a plugin whose name I forgot, and tracking is frankly terrible.
  - nwatson 5y ago
    In which Pat Metheny tries to replace his whole band with guitar-triggered control of real, non-guitar instruments (keyboards, drums, etc) ... "Orchestrion" ... https://youtu.be/KsYEOUKS4Yk https://youtu.be/KsYEOUKS4Yk
    - ratww 5y ago
      Yeah, it's honestly not that good. I think Metheny uses an Axon. Even with those you need to be very careful with your phrasing, you can't be too fast, you lose a lot of dynamic range and a lot of expressivity, sometimes notes just die, the latency is high... Here's what Metheny said himself: "But the guitar‑to‑MIDI part has always been a problem. It's a question of physics. On input, I sort of have to rush. But I know how to rush. I play ahead." It's fun for lots of things, and you can make lots of cool music, but it's still very limited to certain styles, dynamic ranges, phrasing, tempo, speeds...

IAmGraydon 5y ago

The video won’t play for me, but does this allow tweaking of parameters? The problem with most projects like this one is that it replicates the sound of a certain pedal with certain settings. That’s a major problem for most guitarists as playing with the knobs live is a big part of the attraction to pedals.

ushakov 5y ago
not quite there yet, but on possibility is train multiple models and switch between them on the fly another is add more params to the models to take the knob states into the account when doing predictions (which can impact the performance)
FrankyFire 5y ago
No, it doesn't. But to be honest, this would be next level. When looking at this, I'm thinking of alternatives and I can only name 2: 1 - Kemper Profiler 2 - NeuralDSP Both of them are above 1000€/$. We are talking about 10x the price of this thing. Add some Multi-FX Pedal (like Line6 HX-Stomp) where you put this in the FX-Loop and you end up with something equally good for still half the price. And in general, its not about what is better, digital or analog. It's about the use-case. In the studio or when noodling around with the knobs when practicing: Real Amps and Pedals. But on stage, you don't play with the settings of your pedals or you want presets. This is where you go digital. No one will notice the slightly different sound there anyway.
- IAmGraydon 5y ago
  I played out live for a decade and absolutely used the knobs on my pedals live. Most musicians I knew with a pedalboard did the same. That said, if you use a preset based effects setup like rackmount gear, I could see how this would be cool.
  - FrankyFire 5y ago
    Yeah, true. But you would only change it between songs, I guess, and only to the values you always use for that song, right? - This is what I would capture and use as a preset, preset change triggered by MIDI from the MultiFX unit. I know some who change their settings during playing, but I wouldn't consider this as standard. Also, you can still add a real Pedal to a MultiFX Board if you really want to change its settings live. On the contrary, automatically changing the settings controlled by your DAW and triggered via MIDI into your MultiFX unit is also pretty common. You don't have to hassle with your Effects at all and can concentrate on playing, what you should, as you need to be absolutely on point.

andybak 5y ago

any pedal? That seems far-fetched - one trivial counter-example would be a looper as that requires modal input and state. I also wonder if it can handle complex multi-tap delays. Can anyone give a rough idea of the actual limitations? I would guess that there is a limit to how non-local the effects it can manage are.

taneq 5y ago
If you want to be pedantic it probably can't emulate a guitar pedal that you can tap a representation of a Turing machine into and it will only make a sound if the machine halts.
- shmageggy 5y ago
  You can't create such a pedal in the first place, so...
  - jdbernard 5y ago
    Yep. Disprove the sweeping generalization by counterexample.
  - tiborsaas 5y ago
    The joke is that some pedals can be really complex with menu systems and all kinds of tricky jazz.
  - marcodiego 5y ago
    You definitely can. You just cannot prove it will always make a sound.
  - jcelerier 5y ago
    why couldn't you ? the code would just be extern char* turing_machine; int main() { process_turing_machine(turing_machine); beep(); }
- gberger 5y ago
  I think I've read something about this in a story involving Achilles and a tortoise.

notum 5y ago

As much as I'm torn with my feelings about nVidia as a company, especially recently with their attempts to artificially limit hardware you own, I must admit Jetson family is incredibly capable and very well executed. I've been using Jetsons as RasPi replacements wherever I can, they are not only more capable but also much more reliable than RPI4s, in my not so limited experience.

Bancakes 5y ago
Extremely expensive. Or overpriced?
- notum 5y ago
  I wouldn't say that. 2GB dev kit is $50 depending on where you buy them. That's much more bang for the buck, even if you opt out using the tensor cores they were designed for.
  - 1996 5y ago
    They sorely lack m2 for high throughput storage. A board with a m2 port costs an extra $300 and only exists since last year: https://antmicro.com/blog/2020/04/updated-jetson-nano-xavier-nx-baseboard-m2/ https://antmicro.com/blog/2020/04/updated-jetson-nano-xavier... It should have been standard and by default from the beginning.
    - notum 5y ago
      M2 port would have been awesome! I believe the comment stating Jetsons were expensive was made in comparison with the RPI4, which also lacks an m2 slot. I really do find Jetons much more affordable when performance is taken into account, again, Tensor aside. They can also boot from USB3 as of recently, which boosts storage access speed tremendously.

Applejinx 5y ago

…at MP3 quality. This is fine if you like that sort of thing. I would note that latency is very important here: it's not going to be nice to play through if it's incurring any significant latency.

montroser 5y ago
Yes, latency is huge. Modern digital audio stacks in consumer OSes are still completely terrible at this. Not that it's an easy problem to solve. But it's pretty hard to beat elections flowing through an analog circuit, when in the digital side you have to: convert analog to digital, run through the kernel to get to user space, run the bits through the RNN, send back to kernel space, convert to analog and finally send to an output. In order to be competitive with a 1980s guitar pedal, you have to do all of that in under ~10ms latency, and that's just really hard still. Even though we carry around these super computers in our pockets these days, there are still some things left where analog still beats the pants.
- hashkb 5y ago
  Also, the competition is not against vintage analog gear. Modern analog gear is having a true Renaissance, and this community can afford to support the Wampler's of the world. I don't understand the modelling camp at all, their stuff just doesn't sound good, nor is there any joy in working with it.
  - Applejinx 5y ago
    I feel the same but it's hell convincing the reductionists that they haven't got 'the thing, and the whole of the thing' in their little emulation. There's also another element: if you have, say, a vintage Fender Champ and a Klon (or whatever) it's because you mean to project different expressions through your string handling and note-playing. At that level you've made a best effort to produce the most emotionally transparent and responsive signal chain, which you will then not think about once you've got it turned on and tweaked: ALL the settings are liable to sound 'good' and respond for you. The modeling approach is so often "This is exactly that, but better, because here are twelve other Fender Champs and models of Klon to choose from!" and when the first claim isn't as true as we would like, and the second is a distraction and time-sink, that's not great. I can tell when I've chosen wrongly in my music-making tools, because I flat-out stop making music. Even in a dilettantish way: it just stops being a thing. That's a concern.

fermigier 5y ago

Uli Behringer in a box ;)

ushakov 5y ago

have i mentioned yet that training only takes ~30 minutes and you only need ~3 minutes of data?

jtriangle 5y ago
Could it work on a live amp like the Kemper or a quad cortex? And can the models be exported and shared? Because if those two things were squared away, I could see this being an extremely viable project.
- ushakov 5y ago
  yes! for training the model you’d need a separate computer/server though

deleted 5y ago

[deleted]

TrackerFF 5y ago

Sure, but how good will it sound? High quality AD/DA converters alone will cost.

hashkb 5y ago

Can any skeptics weigh in? To me, this kind of tech is centuries away from fooling tone hounds.

Applejinx 5y ago
Not much need to (or benefit in doing so). If you're somebody like AnalogMan, building pedals with careful selection of NOS transistors wired point-to-point, this is so utterly not a threat to you. I've seen entirely analog circuits designed and built with the PCB parts and techniques used in these devices, that are miles away from the quality of good pedals, without even being digital or modeling at all. I'm sure the problem circuits measure completely fine, and then you A/B them with a truly great pedal and it's chalk and cheese. If you try to argue the point, people committed to the digital modeling model get quite fierce, so tone hounds learn to just roll their eyes and not engage. You can also think of it as learning to rely on secret weapons.
- 52-6F-62 5y ago
  > You can also think of it as learning to rely on secret weapons Case in point: the old Neve desks
PaulDavisThe1st 5y ago
"tone hounds"? There are many hundreds to many thousands of pedals. Everyone has their own favorites. There is no "right" in this world. There's just as much chance that whatever "error" a modelled pedal may embody is preferred by its users as there is that it will be deemed worse.
jtriangle 5y ago
Andersons has a few YouTube videos where the Kemper profiler fools tone hounds enough that you can infer that it's already good enough. The NeuralDSP hardware is of at least similar quality, maybe better in some instances. Sure if you run a null test these will fail, but in real life it's really up to how honest the tone hound in question is, and if you can trap them into being honest. If you're talking product viability, something like the Pepsi challenge, double blind testing, would probably be effective marketing.

magicalhippo 5y ago

I recently watched a video[1] by Rhett Shull where he compared Neural Capture of a Quad Cortex[2] to real pedals. I'm not a musician, but even I could easily tell the difference a lot of the time. It didn't sound bad, but there were clearly aspects the Neural Capture failed to, well, capture. Still, early days. [1]: https://www.youtube.com/watch?v=4SSOJxZHolc https://www.youtube.com/watch?v=4SSOJxZHolc [2]: https://neuraldsp.com/quad-cortex https://neuraldsp.com/quad-cortex

ushakov 5y ago
GuitarML have a similar video as well https://www.youtube.com/watch?v=xEOFz3UcDyA https://www.youtube.com/watch?v=xEOFz3UcDyA
- magicalhippo 5y ago
  Similar-ish but clearly very different. I noticed that in that video he was using the WaveNet model, though apparently the statefull LSTM model performs better[1]. Though even for that you can hear[2] a fairly clear difference. [1]: https://towardsdatascience.com/neural-networks-for-real-time-audio-raspberry-pi-guitar-pedal-bded4b6b7f31 https://towardsdatascience.com/neural-networks-for-real-time... [2]: https://towardsdatascience.com/neural-networks-for-real-time-audio-stateful-lstm-b534babeae5d https://towardsdatascience.com/neural-networks-for-real-time...

marcodiego 5y ago

Isn't any reasonably good DSP able to do same? Or is it learning automatically to mimic the effect?

martinvol 5y ago

Wow, fantastic. Amazing work!

kazinator 5y ago

Emulate 20 transistors using hundreds of millions of transistors!

jtriangle 5y ago
20 especially shitty transistors to be fair.
BookPage 5y ago
Thanks, this made me laugh out loud.

wintermutestwin 5y ago

It is my understanding that capture based emulation of a given sound seems like a better idea than it really is. Say you capture a given amp or pedal sound - even if you get the latency down to acceptable #s, what if you want/need to turn a knob? Tweaking knobs is an essential part of the process of dialing in a sound relative to your guitar and output environment. Contrast with the Fractalaudio approach of modeling each component of a device. Fractal's AxeFx is the gold standard and any geek would gush over the HW and SW engineering. The best part is that the company owner keeps improving his algos and pushing out updates for free. This device costs the equivalent of a good amp head and is loaded with more amps and effects than any of the competition. Sorry if this sounded like an ad, but I am always surprised how little airtime this amazing product gets in hacker circles.

kazinator 5y ago
The knobs can be included in the emulation. So that is to say, you model the unit as something having an audio input, and several controllers, such as knobs or switches. You then capture the behavior for all combinations of controller values, as part of the model. ("All combinations", for a potentiometer, might mean stepping it from 0 to 11 in increments of 1.) The emulation ased on the model runs on a piece of hardware which has some generic controllers on it: some rotary encoders, a couple of switches and whatnot. These get assigned to the parameters of the model. You could have a MIDI input on it, and use MIDI controllers, which would be cool. There are MIDI foot controllers that you can tilt with your foot to vary a parameter. The different models could be assigned to MIDI program numbers. You could change the patch number with the foot controller, and vary the parameters with it also. The foot controller might have, say, only two pedals, so you have to assign which ones you want: if the patch has five parameters, you have to fix the values of three of them and map the two important ones to the foot pedals you have. For the others, you can bend down and tweak the knobs on the unit itself.
ushakov 5y ago
the problem with this approach is you need to have knowledge about the circuit to make it happen, not only that, but you also need people, who can reconstruct the circuit digitally this approach is not scalable (that's why the high cost) the ML approach doesn't require any knowledge about the system (black-box) to produce the result

archsurface 5y ago

I bet not the miku.

carlob 5y ago

I wonder how it deals with dynamic range. I've always been told that one shouldn't run an electric guitar directly through a stereo because the peaks might fry it. So I would expect that whatever comes out of the RPi doesn't have much dynamic range anyway so computer speakers would be ok, but sound would be kinda meh. Am I missing something?

duosonic 5y ago

Honest question: if you're a musician, what is the appeal of digital modeling? Is it purely affordability/accessibility, or are you drawn to it because it would create different sonic possibilities that you couldn't get from the original?

URSpider94 5y ago
You mean, besides the appeal, what is the appeal? As an artist and a musician, you can have access to close approximations of sounds that would require a warehouse full of amps and dozens of pedals, costing tens of thousands of dollars - all in a Pi project box. That’s not just a little bit of affordability and convenience.
- duosonic 5y ago
  And I certainly don't mean to sound dismissive about the possibilities that creates! It's a huge factor and I would see it being akin to how the rise of "pro-sumer" home recording equipment played a huge role in underground punk music.
humbledrone 5y ago
Speaking for myself, the accessibility and convenience is a huge part. Instead of lugging around a 60 pound fragile finnicky pedalboard, I just throw my Helix in a backpack and am good to go. It has way more pedals in it than I could fit on a real pedal board, and I can also switch presets with the tap of my foot (including rewiring all the connections between pedals, changing their order, swapping pedals, swapping amps, etc). And yeah the sonic possibilities are endless. On a physical pedalboard, it's pretty involved to rewire everything to change the routing, or add/remove pedals, etc. With the digital modeling ones, this all becomes trivial and you can try all kinds of different setups much more quickly, save them and go back later, share them with friends, etc. All that said, it's kind of like asking an acoustic guitarist in the 1950s why they would use an electric guitar. Electric guitars obviously have lots of advantages, but it's not like people stopped playing acoustic guitars. I still think real analog pedals are cool, they're fun to collect, in some cases they sound better, etc. And sometimes you don't need the mega-flexibility -- if you have a 4-pedal setup that does "your tone" maybe that's all you ever need.

musicale 5y ago

For analog pedals, I'd expect better results from circuit modeling. Given the wide (and very non-linear) range of settings of a typical pedal, as well as interaction (impedance, etc.) with a real guitar and amplifier, it seems like it would be a pain to get all of the training data. For a digital pedal, running the actual (e.g. Eventide) DSP code is just going to be better than some ML approximation. On the other hand, I've been a bit dissatisfied with amplifier and cabinet models based on traditional DSP and physical modeling approaches, so maybe neural networks could fill in some of the gaps.

sillysaurusx 5y ago
ML dilettante here! I wouldn’t discount ML. The nonlinearities are the bread and butter of modern ML models. In fact, two linear layers without a nonlinearity inbetween is equivalent to one big linear layer. So nonlinearities are required. To put it another way, I would gladly bet any reasonable sum of money that in a double blind test, the listener wouldn’t be able to tell the difference from a genuine guitar pedal. (Not necessarily this pedal, but I suspect ML will model the effects more than adequately for human hearing precision.) FWIW, I say this as someone who used to argue that graphics programmers were doing gamedev all wrong because they weren’t modeling light, they were approximating light. ML models were the way out. I also think much of the problem is that ML devs often don’t have traditional signal processing experience, so they haven’t been modeling signals in quite the right way. (I’m trying to rectify that a bit with my FFT tutorials: https://twitter.com/theshawwn/status/1398796224921321472?s=21 https://twitter.com/theshawwn/status/1398796224921321472?s=2...) It remains to be seen, but Fourier space has recently been making strides in ML, and it’s likely much easier for a model to approximate a nonlinear waveform in frequency space than as a raw waveform. To put it another way, if human speech is getting to the point where ML models can trick people, what are the chances that a future model won’t be able to do it for guitars?
- ushakov 5y ago
  actually, the researchers of aalto university (now at neural dsp), who pioneered the guitar ml technology, were working on speech initially and did this one as a side project source: https://m.youtube.com/watch?v=WLTzbEKTxhk https://m.youtube.com/watch?v=WLTzbEKTxhk

patfla 5y ago

I stopped reading at the first sentence: "It’s a well-established fact that a guitarist’s acumen can be accurately gauged by the size of their pedal board- the more stompboxes, the better the player." As both a software engineer and a guitarist, I'd say the opposite is true. Or at least truer. You can't do math-rock without a lot of pedals but the hard part is to acquire the chops. A lot of pedals, and production effects generally, quickly become cliches and it's like dropping down a musical black hole. Someone like Hendrix could take a new effect (superset of pedals) and make it work musically brilliantly but most pedal users buy the pedal to get someone else's sound.

ushakov 5y ago
don’t take it too literally, i’m pretty sure this was meant as a sarcastic joke by the author :D
- patfla 5y ago
  Maybe I should have read past the first sentence. The joke's on me.
awsanswers 5y ago
I believe that was sarcasm
pier25 5y ago
If it wasn't obvious this was a joke, the following paragraph starts with: "Jokes aside..."
deleted 5y ago
[deleted]
u_gene 5y ago
..but most pedal users buy the pedal to.. show off in-front of others who don't have that pedal. period. GarageBand has 1000 of pedals, people don't need more pedals, they need to practice more, but they don't understand that. That's why they are buying more pedals while keep dragging about how much they care about green/clean planet.

jimnotgym 5y ago

In my attic there is a dusty box with a multi effects unit from the 90's. The reason I stopped using it is that it is not that the fuzz doesn't react to the guitars volume pot like a Fuzz Face or because the Tubescreamer was a TS9 not an 808. It was because having a single box with all of your effects in is a faff to tinker with. I like the dedicated hardware of my pedals, I like having the right number of knobs. I like being able to turn off the fuzz but keep the delay by stepping on the fuzz switch. I like being able to run the TS before or after the fuzz and to see it happen. I mentally am much more at home with little boxes for each stage, it is like a real world flow chart!

bozzcl 5y ago
You can do all that with modern multi-effects, and even more. I'm running a hybrid setup, using a Line 6 Helix in front of a Soldano SLO 30 (and on its effects loop as well). Each effect can be adjusted (with all sorts of knobs, more than 12 for some), moved around and triggered individually or together. You can even use the Helix to toggle channels in the amp, manually or as part of presets. It drastically eased up my workflow to experiment with and mix effects. No more tinkering with cables. No more wondering which pedal is plugged in wrong or doesn't have enough power. No more velcro strips. Plus, selling my hoard of pedals felt nice.

jimnotgym 5y ago

Just a quick note, you could solder together most classic guitar pedals for $20. You could buy reissues of many of them for less than $100. Big Muff, Proco Rat, Klon Centaur (Klones), Tubescreamer, Dynacomp, etc are all available for less than that. Just being clear that this is not a price issue.

geedy 5y ago
Each though.

u_gene 5y ago

Yo People!! A friend keeps sending links to guitar pedal discussions to me and I thought I chip in to the conversation. While you can emulate any guitar pedal with AI and you don't have to pay 120$ I'd like to introduce you to some understanding. MUSICIANS love to purchase shit they can show off to others. it's in their ego nature. While 120$ for software might compete with quad pro cortex pedal, you can't show off with 120$ but with a pedal you bought fo 2g you can. That is why BOSS made a big travel box for physical pedals this year, this is what they released at the NAMM2021. How do I know that? I make software for them. GarageBand has 1000s of different pedals already and those are free. most musicians have apple or abelton live or some similar. Also, not to discourage anyone - live music is dying. More and more you see musicians dancing on the stage with their instruments pretending they are playing, while pro-edited backtrack is going through the speakers. If you want to make this world a better place think how to improve housing/farming/healthcare with technology. Things like roof above the head, food on the table will be always in demand. There is no money to be maid with musicians, cause they don't have any. it took me few years to realize, you can learn on my mistakes. If you have some cash and time and thinking if you need to build some software product - kill that, buy some land and build a cabin. In the end you'll have another property. you can sell it or rent it. you can live it to your kids, there is value there. There is no value in your software idea unless people use it like crazy, and they really won't cause they already have 100 pedals and that new pedal bag from boss

n4kana 5y ago

I want to know if anyone ever uses this to create a fleet of classic microphone sounds from a working mic like the SM57. Sennheiser, Neumann, Telefunken, AKG. Hell, you could stack it with preamp filters like Neve. It would bring me great pleasure to see a studio chock full of SM57s masquerading as the best microphones money can buy.

ushakov 5y ago
i saw people simulating studio-grade compressors, artifacts included