3 ms·

Making synthesizers emulate the sound of real instruments was once a major goal of synthesis, but that has mostly faded outside of the rarefied world of physica

Making synthesizers emulate the sound of real instruments was once a major goal of synthesis, but that has mostly faded outside of the rarefied world of physical modeling. Electronic musicians remain interested in sounding "warm" and "organic" (or the opposite), but it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet. But when I was learning FM synthesis (one of the harder ones), I found it incredibly useful to go through the exercise of trying to make a synthesizer sound like a convincing gong, or reed instrument, or guitar string. In fact, I don't think FM really clicked until I did that.

TheOtherHobbes 3y ago
This isn't about accurate recreations of acoustic instruments, it's about capturing some of the features of acoustic instruments in non-imitative patches to make them sound less sterile and simplistic. A lot of (non-MIDI) computer music is either drones, granular clouds, or percussive blips. There's an under-explored area of articulation and musical detail between those sounds and warm, mushy analog. The DX7 was a good first take at that. But today you can have hundreds of oscillators stacked and patched in any configuration you want instead of just six.
- bravura 3y ago
  You can actually use thousands of oscillators, which is what Magenta did with their differentiable DSP (DDSP) approach to modeling acoustic instruments including singing: https://magenta.tensorflow.org/ddsp-vst https://magenta.tensorflow.org/ddsp-vst https://magenta.tensorflow.org/ddsp https://magenta.tensorflow.org/ddsp I doesn't sound realistic but it is expressive in an unusual way.
- Blackthorn 3y ago
  Feels like some better instruments can help with this, over the basic keyboard. Things with polyphonic aftertouch, or electronic wind instruments, or more advanced surfaces like the Linnstrument or Continuum.
MrScruff 3y ago
Agree with you there on both counts. Attempting to match physical instruments is a great learning exercise, but not usually that interesting creatively. However patches that live someone in the middle ground between naturalistic and synthetic are often the most interesting IMO.
bravura 3y ago
"it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet" Perhaps that's because it's quite difficult to do that with synthesizers? Because once you start using samples, you loss the ability to play and modulate without it sounding like a sample? I guess my counterpoint would be the massive popularity of Omnisphere, which combines a sophisticated sample-based synthesizer with good filters and a few other types of synthesis. Or would you consider this "organic" but not "realistic/acoustic"?
- bongodongobob 3y ago
  High end realistic patches allow you to move mics around the virtual space, change mics, move musicians, control airflow, bowing speed and pressure etc. They sound REALLY good these days, almost indistinguishable from the real thing. I think this article is more geared towards pure synthesis when you are using very few primary harmonics. Like stacked sine waves or something. They are impossible to eq because the frequencies are too pure and unnatural so you have to dirty them up.
- jandrewrogers 3y ago
  Many synthesizers do not use samples. Physical modeling oscillators have been available in good synthesis engines for a long time. These generate sounds based on a physics model of real acoustic instrument designs and materials, dynamically reacting like real instruments. These used to be dedicated synthesizers but are now buried in a long list of oscillator models available on general purpose synthesizers. I used to own a Korg Z1, which had one of the first physical modeling synthesis engines. While it could produce relatively realistic acoustic sounds, that is not what people ended up using these types of synthesizers for. Unlike a physical acoustic instrument, where the basic geometry, material properties, and other parameters are fixed, most parameters of the synthesized physical models could be dynamically modulated and manipulated like any other synthesis parameter. Modulating the fundamental physics properties of an acoustic instrument at audio rates produces some really interesting timbres and effects that are not reproducible using any other type of synthesis or (obviously) acoustic instruments. Consequently, no one ended up using them for realistic acoustic sounds; it was much more interesting to use the synthesis engine to do physically impossible manipulation of the acoustic model to generate novel sounds. These days almost everything in a synthesizer engine is based on modeling due to the inexorable increase in available computing power. Nonetheless, there is still considerable convenience and economy in using acoustic instruments or samples for many purposes. Just because we can create incredibly detailed and realistic physical models doesn't mean it is worth the effort and they often have terrible UI. I could see this as being something where AI could do a lot. Omnisphere is a good example of where the term "sample-based" isn't entirely accurate (ignoring its modeled oscillators). While it does have a large and excellent sample library, many of the oscillator engines use the samples as spectral feedstock instead of as a sound to be played per se. The raw sample is not identifiable even though it imparts a characteristic quality on how the oscillator sounds. (Omnisphere is also massively popular because it is an excellent synthesis engine with an unusually good ease-of-use to power ratio. Still one of my all-time favorites.)
zigzag312 3y ago
Synplant 2 can now do that automatically. It uses AI to create synth patches that emulate the input sample.
- djmips 3y ago
  Can it create FM synth patches in particular? Like a patch for a DX7.
  - ta988 3y ago
    Aphex Twin tool for that: https://magpi.raspberrypi.com/articles/aphex-twin-midimutant https://magpi.raspberrypi.com/articles/aphex-twin-midimutant
  - zigzag312 3y ago
    Not for DX7. It creates patches for it's own synth (2 oscillators, FM, envelopes, LFO, and effects). You can then look at how they are build and edit the parameters.
  - chaosprint 3y ago
    I have a project that aims to do the similar task but with a different approach: https://github.com/chaosprint/RaveForce https://github.com/chaosprint/RaveForce not finished yet, so pr is welcomed!
- InCityDreams 3y ago
  Sample an elephant, synplant to synthesize it. As a non-singer synplant has allowed me to be on my tracks...you could never tell. Also, amseveral nembers of my friends and family are a) knocked out by the simple fun of synplant, and hearing/ looking at what ai believes constitutes their voices.
pierrec 3y ago
Just to give an example of "the rarefied field of physical modeling": SampleModeling/AudioModeling/SWAM instruments enjoy some popularity, I'd say they were the go-to demo when MPE controllers had a burst in popularity a couple of years ago. Most of them use a hybrid sampling-synthesis approach, and some of them use almost entirely synthesis. Astoundingly realistic when used correctly. Personally, I'd say physical modeling still has a lot of unexplored potential, but that doesn't seem to be a popular opinion.
LegitShady 3y ago
i think its because sample based virtual instruments, while enormous in hard drive space, do a good job of approximating many classical instruments and have advanced enough to be useful without being difficult to implement. I could learn a new synth or I could use the Spitfire Labs BBC Symphony Orchestra pack and spend a lot of takes playing with macro wheels and maybe some adjusting after to try to improve articulation to something more natural sounding. It would by a bear live but in the studio its easy.| The synth would offer more customization but the sample one is good enough for most things, especially in a small/home studio setting.
- tgv 3y ago
  Synth strings can be great, but in their own aesthetics. I'd argue the sample pack is much, much more realistic and rich sounding than any synth patch you can come up with. There's no easily programmable synth that can emulate the mixture of a dozen or so strings vibrating at slightly different pitches. Creating a single, somewhat realistic violin is almost impossible outside of very large physical models (there's a good model in Reaktor by Chet Singer). Even dedicated physical modelling synths fail at that task.
djmips 3y ago
I'm trying to make FM synthesis mimic a particular sound but I find it very difficult to search for something that even approximates the sound I'm looking to emulate (an elephant's roar). Do you have any tips? Edit - from the comments I found this paper which looks promising https://ccrma.stanford.edu/~cc/vox/smac2013som/upho2013.pdf https://ccrma.stanford.edu/~cc/vox/smac2013som/upho2013.pdf Which also leads me to this paper by Chowning himeself from 1980 titled 'Computer synthesis of the singing voice' which uses FM synthesis. https://www.speech.kth.se/music/publications/kma/papers/kma29-ocr.pdf https://www.speech.kth.se/music/publications/kma/papers/kma2...
kzrdude 3y ago
I think Roland's digital piano sounds are entirely simulated/modelled, they don't use sampling like for example Yamaha does for their digital pianos. However, is that a continuation of synthesizers or not?

TheOtherHobbes 3y ago

This isn't about accurate recreations of acoustic instruments, it's about capturing some of the features of acoustic instruments in non-imitative patches to make them sound less sterile and simplistic. A lot of (non-MIDI) computer music is either drones, granular clouds, or percussive blips. There's an under-explored area of articulation and musical detail between those sounds and warm, mushy analog. The DX7 was a good first take at that. But today you can have hundreds of oscillators stacked and patched in any configuration you want instead of just six.

bravura 3y ago
You can actually use thousands of oscillators, which is what Magenta did with their differentiable DSP (DDSP) approach to modeling acoustic instruments including singing: https://magenta.tensorflow.org/ddsp-vst https://magenta.tensorflow.org/ddsp-vst https://magenta.tensorflow.org/ddsp https://magenta.tensorflow.org/ddsp I doesn't sound realistic but it is expressive in an unusual way.
Blackthorn 3y ago
Feels like some better instruments can help with this, over the basic keyboard. Things with polyphonic aftertouch, or electronic wind instruments, or more advanced surfaces like the Linnstrument or Continuum.

MrScruff 3y ago

Agree with you there on both counts. Attempting to match physical instruments is a great learning exercise, but not usually that interesting creatively. However patches that live someone in the middle ground between naturalistic and synthetic are often the most interesting IMO.

bravura 3y ago

"it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet" Perhaps that's because it's quite difficult to do that with synthesizers? Because once you start using samples, you loss the ability to play and modulate without it sounding like a sample? I guess my counterpoint would be the massive popularity of Omnisphere, which combines a sophisticated sample-based synthesizer with good filters and a few other types of synthesis. Or would you consider this "organic" but not "realistic/acoustic"?

bongodongobob 3y ago
High end realistic patches allow you to move mics around the virtual space, change mics, move musicians, control airflow, bowing speed and pressure etc. They sound REALLY good these days, almost indistinguishable from the real thing. I think this article is more geared towards pure synthesis when you are using very few primary harmonics. Like stacked sine waves or something. They are impossible to eq because the frequencies are too pure and unnatural so you have to dirty them up.
jandrewrogers 3y ago
Many synthesizers do not use samples. Physical modeling oscillators have been available in good synthesis engines for a long time. These generate sounds based on a physics model of real acoustic instrument designs and materials, dynamically reacting like real instruments. These used to be dedicated synthesizers but are now buried in a long list of oscillator models available on general purpose synthesizers. I used to own a Korg Z1, which had one of the first physical modeling synthesis engines. While it could produce relatively realistic acoustic sounds, that is not what people ended up using these types of synthesizers for. Unlike a physical acoustic instrument, where the basic geometry, material properties, and other parameters are fixed, most parameters of the synthesized physical models could be dynamically modulated and manipulated like any other synthesis parameter. Modulating the fundamental physics properties of an acoustic instrument at audio rates produces some really interesting timbres and effects that are not reproducible using any other type of synthesis or (obviously) acoustic instruments. Consequently, no one ended up using them for realistic acoustic sounds; it was much more interesting to use the synthesis engine to do physically impossible manipulation of the acoustic model to generate novel sounds. These days almost everything in a synthesizer engine is based on modeling due to the inexorable increase in available computing power. Nonetheless, there is still considerable convenience and economy in using acoustic instruments or samples for many purposes. Just because we can create incredibly detailed and realistic physical models doesn't mean it is worth the effort and they often have terrible UI. I could see this as being something where AI could do a lot. Omnisphere is a good example of where the term "sample-based" isn't entirely accurate (ignoring its modeled oscillators). While it does have a large and excellent sample library, many of the oscillator engines use the samples as spectral feedstock instead of as a sound to be played per se. The raw sample is not identifiable even though it imparts a characteristic quality on how the oscillator sounds. (Omnisphere is also massively popular because it is an excellent synthesis engine with an unusually good ease-of-use to power ratio. Still one of my all-time favorites.)

zigzag312 3y ago

Synplant 2 can now do that automatically. It uses AI to create synth patches that emulate the input sample.

djmips 3y ago
Can it create FM synth patches in particular? Like a patch for a DX7.
- ta988 3y ago
  Aphex Twin tool for that: https://magpi.raspberrypi.com/articles/aphex-twin-midimutant https://magpi.raspberrypi.com/articles/aphex-twin-midimutant
- zigzag312 3y ago
  Not for DX7. It creates patches for it's own synth (2 oscillators, FM, envelopes, LFO, and effects). You can then look at how they are build and edit the parameters.
- chaosprint 3y ago
  I have a project that aims to do the similar task but with a different approach: https://github.com/chaosprint/RaveForce https://github.com/chaosprint/RaveForce not finished yet, so pr is welcomed!
InCityDreams 3y ago
Sample an elephant, synplant to synthesize it. As a non-singer synplant has allowed me to be on my tracks...you could never tell. Also, amseveral nembers of my friends and family are a) knocked out by the simple fun of synplant, and hearing/ looking at what ai believes constitutes their voices.

pierrec 3y ago

Just to give an example of "the rarefied field of physical modeling": SampleModeling/AudioModeling/SWAM instruments enjoy some popularity, I'd say they were the go-to demo when MPE controllers had a burst in popularity a couple of years ago. Most of them use a hybrid sampling-synthesis approach, and some of them use almost entirely synthesis. Astoundingly realistic when used correctly. Personally, I'd say physical modeling still has a lot of unexplored potential, but that doesn't seem to be a popular opinion.

LegitShady 3y ago

i think its because sample based virtual instruments, while enormous in hard drive space, do a good job of approximating many classical instruments and have advanced enough to be useful without being difficult to implement. I could learn a new synth or I could use the Spitfire Labs BBC Symphony Orchestra pack and spend a lot of takes playing with macro wheels and maybe some adjusting after to try to improve articulation to something more natural sounding. It would by a bear live but in the studio its easy.| The synth would offer more customization but the sample one is good enough for most things, especially in a small/home studio setting.

tgv 3y ago
Synth strings can be great, but in their own aesthetics. I'd argue the sample pack is much, much more realistic and rich sounding than any synth patch you can come up with. There's no easily programmable synth that can emulate the mixture of a dozen or so strings vibrating at slightly different pitches. Creating a single, somewhat realistic violin is almost impossible outside of very large physical models (there's a good model in Reaktor by Chet Singer). Even dedicated physical modelling synths fail at that task.

djmips 3y ago

I'm trying to make FM synthesis mimic a particular sound but I find it very difficult to search for something that even approximates the sound I'm looking to emulate (an elephant's roar). Do you have any tips? Edit - from the comments I found this paper which looks promising https://ccrma.stanford.edu/~cc/vox/smac2013som/upho2013.pdf https://ccrma.stanford.edu/~cc/vox/smac2013som/upho2013.pdf Which also leads me to this paper by Chowning himeself from 1980 titled 'Computer synthesis of the singing voice' which uses FM synthesis. https://www.speech.kth.se/music/publications/kma/papers/kma29-ocr.pdf https://www.speech.kth.se/music/publications/kma/papers/kma2...

kzrdude 3y ago

I think Roland's digital piano sounds are entirely simulated/modelled, they don't use sampling like for example Yamaha does for their digital pianos. However, is that a continuation of synthesizers or not?