5 ms·

Manipulate audio with a simple Python library

164 points by coppolaemilio 12y ago

r0muald 12y ago
> last_5_seconds = song[5000:] I think it should be song[-5000:] EDIT it was already reported https://github.com/jiaaro/pydub/issues/65 https://github.com/jiaaro/pydub/issues/65
- jiaaro 12y ago
  oh damn, updated the readme but not the dot-com. Fixed. I really need to automate that :/
meowface 12y ago
This looks really cool. I'm a Python developer but have a little experience with audio/music processing. Is there some other software or DSL that can manipulate audio files with a high-level syntax like this?
- deleted 12y ago
  [deleted]
- acbart 12y ago
  It reminds me of VirtualDub's scripting language, which is meant for videos but could do audio, if I recall correctly. It's been about half a decade since I used it, though...
  - meowface 12y ago
    What do professional music producers and mixers use? I imagine GUI-type mixing software is still the most popular, but I feel like there must be some experimental musicians out there producing music by fuzzing different parameters in some kind of scripting or configuration language. If there actually aren't, I kind of feel like there should be.
    - erikschoster 12y ago
      There are many DSLs for music! Here are a few: https://en.wikipedia.org/wiki/Audio_programming_language https://en.wikipedia.org/wiki/Audio_programming_language Many of which are inspired by or even directly based on Max Matthew's work: https://en.wikipedia.org/wiki/MUSIC-N https://en.wikipedia.org/wiki/MUSIC-N
      - meowface 12y ago
        Thanks, these are really cool. These were the kinds of things I had in mind. I'm tempted to experiment with one myself, but my music skills and knowledge are non-existent. And I'm pretty sure those are much more essential than programming skills when using these languages.
        jholman 12y ago
        That doesn't sound like a reason to avoid experimenting. If you're tempted to experiment, jump in! Be bold!
    - one-more-minute 12y ago
      I've seen some really cool stuff being done with Overtone: http://overtone.github.io/ http://overtone.github.io/
      - meowface 12y ago
        This is awesome, thanks. edit: Just found this example song apparently coded live in Overtone: https://soundcloud.com/meta-ex/spiked-with-recursive-dreams https://soundcloud.com/meta-ex/spiked-with-recursive-dreams Quite amazing.
    - zwegner 12y ago
      I have a simple Python library[0] for direct audio synthesis, with oscillators, filters, various other effects, MIDI input, plus some basic building blocks for algorithmic music construction. It pretty much needs PyPy to provide samples fast enough for real-time audio, but even PyPy's too slow sometimes. Not too much documentation right now, and there's barely any example usage. Think it's about time to remedy that! I have plans for an open-source album created with it, but it's just an idea at this point. [0] https://github.com/zwegner/undulance https://github.com/zwegner/undulance
- xj9 12y ago
  I don't know if this fits the bill exactly, but I've been dying to have time to play around with Overtone[1] and Emacs Live[2]! [1]: http://overtone.github.io http://overtone.github.io [2]: http://overtone.github.io/emacs-live/ http://overtone.github.io/emacs-live/
- crucialfelix 12y ago
  http://supercollider.github.io/ http://supercollider.github.io/ is one of the most widely used.
- rhizome31 12y ago
  There's ChucK http://chuck.cs.princeton.edu/ http://chuck.cs.princeton.edu/ and also for Python there's Pyo http://ajaxsoundstudio.com/software/pyo/ http://ajaxsoundstudio.com/software/pyo/
goldfeld 12y ago
I'd be crazy for a library (any language) to distill a sound file into discrete MIDI notes (or any notation), with configurable threshold levels. Clustering those into different tracks by some unsupervised learning would be a dream.
- lifeisstillgood 12y ago
  This comment is mostly why I have never tread in the feared world of multimedia. I simply don't understand what it means. I mean I can guess at trying to pull the original "tracks" "laid down" out from a recording but - is that possible, common but not FOSS? I simply don't know where to start and have not had the incentive to discover. It's probably as laughable to others as say someone turnin up and saying "yeah SQL - I sort of understand it's to do with tables right?" But audio and video seem like closed worlds of programming. There seems to be no gateway from here to there. So something like this is v exciting - it might be the gateway drug.
  - jfb 12y ago
    Audio and video really aren't that closed; they just require a commitment to learning things that are pretty different than what a typical programmer does; and they get math heavy. But they're really, really fun to play with, when you knuckle down, and there's a fairly gentle learning curve from understanding formats and containers to samples to statistical signal processing and psychoacoustic and visual models. And being good at it means writing your own ticket, if you're a careerist.
    - lifeisstillgood 12y ago
      Is there a recognised / common syllabus for that gentle learning curve - it might make a worthwhile hobby ... But it's definitely a "one day" for the moment.
      - chipsy 12y ago
        I don't know of a defined syllabus but here is what I did: 1. Write some basic synthesis code(naive oscillator and volume envelope) and some way of sequencing it. Write little toy trackers and some procedural audio with this technique. 2. Try and fail a few times to write dataflow engine for audio like Pure Data. (this is kind of a big project and it turns out, you really don't need it to experiment.) 3. Write Standard MIDI Format playback system for a one-voice PC beeper emulation. This turns out to be a gateway drug for me learning more in depth because all you have to do is add "just one more" feature and every MIDI file you play sounds a little better. 4. Expand MIDI playback and synthesizer in tandem. End up with polyphonic .WAV sampler, then Soundfont playback. Learn everything about DSP badly, and gradually correct misconceptions. (DSP material can be tricky since the math concepts map to illegible code, and a lot of the resources are for general engineering applications instead of audio.) 5. Rewrite things and work on custom subtractive synthesizer, after getting everything wrong a few times over in the first synthesis engine. Still do not know to write nice sounding IIR filters; steal a good free implementation instead. And that is where I stand today. I know enough to engineer complex signal chains, how some of the common formats are structured, and some tricks for improving sound quality and optimizing CPU time; what I miss is the background for writing original signal processing algorithms, which gets really specialized(there are some people who devote themselves entirely to reverbs, for example). These algorithms and their sound characteristics are effectively trade secrets, so the opaqueness of the field is not just a matter of the problems being hard - DSP just hasn't become as commodified as other software.
Joeboy 12y ago
A couple of things that people who like this might also be interested in: Coursera.org is currently running a course called "Audio Signal Processing for Music Applications" which I believe uses python. It's in its second week, so you have time to catch up. https://www.coursera.org/course/audio https://www.coursera.org/course/audio http://aubio.org/ http://aubio.org/ is a library that does note onset detection, pitch detection, beat / tempo tracking and various other things. It has python bindings.
urs2102 12y ago
What is the best way to break apart an MP3 file (similar to ID3Lib) so you can see how it's encoded and edit the artist name, song name, add an image, change the encoding format, etc? Is there a good python library for re-encoding an MP3?
- jiaaro 12y ago
  pydub does it, but it shells out to ffmpeg, not sure if that's acceptable
- jpasden 12y ago
  This might do what you're looking for: http://eyed3.nicfit.net/ http://eyed3.nicfit.net/
emillon 12y ago
I've been similarly frustrated with DAWs too. In the last few weeks I had a go at writing a DSL that shells out to SoX for audio manipulation. This way I don't have to manipulate audio samples myself. It's way less polished than pydub but here it is if anyone is interested: https://github.com/emillon/tmc https://github.com/emillon/tmc Example: https://github.com/emillon/tmc/blob/master/Music/TMC/Example.hs https://github.com/emillon/tmc/blob/master/Music/TMC/Example... At first it was a pretext to play with free monads, a way of building EDSLs. But right now I'm not sure it's not just a complication. Though, having an intermediate representation before executing the SoX commands makes it possible to write an optimizer (for example, collapsing two audio shifts).
- jiaaro 12y ago
  I was thinking of using sox as well - turns out a lot a lot of the audioop module in the Stdlib comes directly from sox :) relevant comment in the audioop source: https://github.com/python-git/python/blob/master/Modules/audioop.c#L27 https://github.com/python-git/python/blob/master/Modules/aud...
- conradfr 12y ago
  I know it's not available on Linux and not free but have you tried Reaper ? It has lots of scripting / coding capabilities.
  - emillon 12y ago
    Interesting, I didn't know about these capabilities, thanks!
prezjordan 12y ago
Cool idea! I like the overloaded operators.
- mobiuscog 12y ago
  I would if they were consistent, but having some to change volume and some to change repeats is somewhat jarring.
  - jiaaro 12y ago
    I almost used a decibel type but in practice, when you write my_sound + 6 it feels pretty natural to think about that as "add 6dB" and my_sound * 3 as "my_sound 3 times" Also, since those are the only odd overloads, it is quite easy to learn (at least I think so, but I'm the author =P )
stevetjoa 12y ago
Thanks for making this. Relevant plug: I have a Vagrant box [1] and GitHub repo [2] containing IPython notebooks that we use in a workshop on music information retrieval. (Caution: the IPython notebooks are under heavy development, i.e. incomplete. If you have any feedback, please create a GitHub issue.) I just added pydub to the latest version of the Vagrant box [3]. The nice thing about this setup is that, regardless of your host OS, everybody has the same development environment so you don't have to go through the pains of installing numpy, scipy, scikit-learn, essentia, and more. [1] https://vagrantcloud.com/stevetjoa/boxes/stanford-mir https://vagrantcloud.com/stevetjoa/boxes/stanford-mir [2] https://github.com/stevetjoa/stanford-mir https://github.com/stevetjoa/stanford-mir [3] https://vagrantcloud.com/stevetjoa/boxes/stanford-mir/versions/7 https://vagrantcloud.com/stevetjoa/boxes/stanford-mir/versio...
- jiaaro 12y ago
  You're welcome :) Looks like a cool project!
tdicola 12y ago
Nice! I love little focused Python libraries like this. Will have to keep it in mind if I ever do audio stuff in the future.
bravura 12y ago
Cool project. The echonest remix API has similar capabilities, and can quantize into granularities of measures, beats, tatums. http://echonest.github.io/remix/ http://echonest.github.io/remix/
jestinjoy1 12y ago
Suggestions on some cool academic projects that could use this library?
DarkIye 12y ago
I started writing a sample-level audio suite in Java, for musical purposes: https://github.com/williamberg/audioDCB https://github.com/williamberg/audioDCB which I'll probably discard because Java is not the right tool for the job. Inspecting pydub/pydub/pyaudioop.py , I see there are methods for working on the sample level. I'll make a mental note to come back to this project when I get back on the computer music bandwagon again.

164 points by coppolaemilio 12y ago

> last_5_seconds = song[5000:] I think it should be song[-5000:] EDIT it was already reported https://github.com/jiaaro/pydub/issues/65 https://github.com/jiaaro/pydub/issues/65

jiaaro 12y ago
oh damn, updated the readme but not the dot-com. Fixed. I really need to automate that :/

meowface 12y ago

This looks really cool. I'm a Python developer but have a little experience with audio/music processing. Is there some other software or DSL that can manipulate audio files with a high-level syntax like this?

deleted 12y ago
[deleted]
acbart 12y ago
It reminds me of VirtualDub's scripting language, which is meant for videos but could do audio, if I recall correctly. It's been about half a decade since I used it, though...
- meowface 12y ago
  What do professional music producers and mixers use? I imagine GUI-type mixing software is still the most popular, but I feel like there must be some experimental musicians out there producing music by fuzzing different parameters in some kind of scripting or configuration language. If there actually aren't, I kind of feel like there should be.
  - erikschoster 12y ago
    There are many DSLs for music! Here are a few: https://en.wikipedia.org/wiki/Audio_programming_language https://en.wikipedia.org/wiki/Audio_programming_language Many of which are inspired by or even directly based on Max Matthew's work: https://en.wikipedia.org/wiki/MUSIC-N https://en.wikipedia.org/wiki/MUSIC-N
    - meowface 12y ago
      Thanks, these are really cool. These were the kinds of things I had in mind. I'm tempted to experiment with one myself, but my music skills and knowledge are non-existent. And I'm pretty sure those are much more essential than programming skills when using these languages.
      - jholman 12y ago
        That doesn't sound like a reason to avoid experimenting. If you're tempted to experiment, jump in! Be bold!
  - one-more-minute 12y ago
    I've seen some really cool stuff being done with Overtone: http://overtone.github.io/ http://overtone.github.io/
    - meowface 12y ago
      This is awesome, thanks. edit: Just found this example song apparently coded live in Overtone: https://soundcloud.com/meta-ex/spiked-with-recursive-dreams https://soundcloud.com/meta-ex/spiked-with-recursive-dreams Quite amazing.
  - zwegner 12y ago
    I have a simple Python library[0] for direct audio synthesis, with oscillators, filters, various other effects, MIDI input, plus some basic building blocks for algorithmic music construction. It pretty much needs PyPy to provide samples fast enough for real-time audio, but even PyPy's too slow sometimes. Not too much documentation right now, and there's barely any example usage. Think it's about time to remedy that! I have plans for an open-source album created with it, but it's just an idea at this point. [0] https://github.com/zwegner/undulance https://github.com/zwegner/undulance
xj9 12y ago
I don't know if this fits the bill exactly, but I've been dying to have time to play around with Overtone[1] and Emacs Live[2]! [1]: http://overtone.github.io http://overtone.github.io [2]: http://overtone.github.io/emacs-live/ http://overtone.github.io/emacs-live/
crucialfelix 12y ago
http://supercollider.github.io/ http://supercollider.github.io/ is one of the most widely used.
rhizome31 12y ago
There's ChucK http://chuck.cs.princeton.edu/ http://chuck.cs.princeton.edu/ and also for Python there's Pyo http://ajaxsoundstudio.com/software/pyo/ http://ajaxsoundstudio.com/software/pyo/

goldfeld 12y ago

I'd be crazy for a library (any language) to distill a sound file into discrete MIDI notes (or any notation), with configurable threshold levels. Clustering those into different tracks by some unsupervised learning would be a dream.

lifeisstillgood 12y ago
This comment is mostly why I have never tread in the feared world of multimedia. I simply don't understand what it means. I mean I can guess at trying to pull the original "tracks" "laid down" out from a recording but - is that possible, common but not FOSS? I simply don't know where to start and have not had the incentive to discover. It's probably as laughable to others as say someone turnin up and saying "yeah SQL - I sort of understand it's to do with tables right?" But audio and video seem like closed worlds of programming. There seems to be no gateway from here to there. So something like this is v exciting - it might be the gateway drug.
- jfb 12y ago
  Audio and video really aren't that closed; they just require a commitment to learning things that are pretty different than what a typical programmer does; and they get math heavy. But they're really, really fun to play with, when you knuckle down, and there's a fairly gentle learning curve from understanding formats and containers to samples to statistical signal processing and psychoacoustic and visual models. And being good at it means writing your own ticket, if you're a careerist.
  - lifeisstillgood 12y ago
    Is there a recognised / common syllabus for that gentle learning curve - it might make a worthwhile hobby ... But it's definitely a "one day" for the moment.
    - chipsy 12y ago
      I don't know of a defined syllabus but here is what I did: 1. Write some basic synthesis code(naive oscillator and volume envelope) and some way of sequencing it. Write little toy trackers and some procedural audio with this technique. 2. Try and fail a few times to write dataflow engine for audio like Pure Data. (this is kind of a big project and it turns out, you really don't need it to experiment.) 3. Write Standard MIDI Format playback system for a one-voice PC beeper emulation. This turns out to be a gateway drug for me learning more in depth because all you have to do is add "just one more" feature and every MIDI file you play sounds a little better. 4. Expand MIDI playback and synthesizer in tandem. End up with polyphonic .WAV sampler, then Soundfont playback. Learn everything about DSP badly, and gradually correct misconceptions. (DSP material can be tricky since the math concepts map to illegible code, and a lot of the resources are for general engineering applications instead of audio.) 5. Rewrite things and work on custom subtractive synthesizer, after getting everything wrong a few times over in the first synthesis engine. Still do not know to write nice sounding IIR filters; steal a good free implementation instead. And that is where I stand today. I know enough to engineer complex signal chains, how some of the common formats are structured, and some tricks for improving sound quality and optimizing CPU time; what I miss is the background for writing original signal processing algorithms, which gets really specialized(there are some people who devote themselves entirely to reverbs, for example). These algorithms and their sound characteristics are effectively trade secrets, so the opaqueness of the field is not just a matter of the problems being hard - DSP just hasn't become as commodified as other software.

Joeboy 12y ago

A couple of things that people who like this might also be interested in: Coursera.org is currently running a course called "Audio Signal Processing for Music Applications" which I believe uses python. It's in its second week, so you have time to catch up. https://www.coursera.org/course/audio https://www.coursera.org/course/audio http://aubio.org/ http://aubio.org/ is a library that does note onset detection, pitch detection, beat / tempo tracking and various other things. It has python bindings.

urs2102 12y ago

What is the best way to break apart an MP3 file (similar to ID3Lib) so you can see how it's encoded and edit the artist name, song name, add an image, change the encoding format, etc? Is there a good python library for re-encoding an MP3?

jiaaro 12y ago
pydub does it, but it shells out to ffmpeg, not sure if that's acceptable
jpasden 12y ago
This might do what you're looking for: http://eyed3.nicfit.net/ http://eyed3.nicfit.net/

emillon 12y ago

I've been similarly frustrated with DAWs too. In the last few weeks I had a go at writing a DSL that shells out to SoX for audio manipulation. This way I don't have to manipulate audio samples myself. It's way less polished than pydub but here it is if anyone is interested: https://github.com/emillon/tmc https://github.com/emillon/tmc Example: https://github.com/emillon/tmc/blob/master/Music/TMC/Example.hs https://github.com/emillon/tmc/blob/master/Music/TMC/Example... At first it was a pretext to play with free monads, a way of building EDSLs. But right now I'm not sure it's not just a complication. Though, having an intermediate representation before executing the SoX commands makes it possible to write an optimizer (for example, collapsing two audio shifts).

jiaaro 12y ago
I was thinking of using sox as well - turns out a lot a lot of the audioop module in the Stdlib comes directly from sox :) relevant comment in the audioop source: https://github.com/python-git/python/blob/master/Modules/audioop.c#L27 https://github.com/python-git/python/blob/master/Modules/aud...
conradfr 12y ago
I know it's not available on Linux and not free but have you tried Reaper ? It has lots of scripting / coding capabilities.
- emillon 12y ago
  Interesting, I didn't know about these capabilities, thanks!

prezjordan 12y ago

Cool idea! I like the overloaded operators.

mobiuscog 12y ago
I would if they were consistent, but having some to change volume and some to change repeats is somewhat jarring.
- jiaaro 12y ago
  I almost used a decibel type but in practice, when you write my_sound + 6 it feels pretty natural to think about that as "add 6dB" and my_sound * 3 as "my_sound 3 times" Also, since those are the only odd overloads, it is quite easy to learn (at least I think so, but I'm the author =P )

stevetjoa 12y ago

Thanks for making this. Relevant plug: I have a Vagrant box [1] and GitHub repo [2] containing IPython notebooks that we use in a workshop on music information retrieval. (Caution: the IPython notebooks are under heavy development, i.e. incomplete. If you have any feedback, please create a GitHub issue.) I just added pydub to the latest version of the Vagrant box [3]. The nice thing about this setup is that, regardless of your host OS, everybody has the same development environment so you don't have to go through the pains of installing numpy, scipy, scikit-learn, essentia, and more. [1] https://vagrantcloud.com/stevetjoa/boxes/stanford-mir https://vagrantcloud.com/stevetjoa/boxes/stanford-mir [2] https://github.com/stevetjoa/stanford-mir https://github.com/stevetjoa/stanford-mir [3] https://vagrantcloud.com/stevetjoa/boxes/stanford-mir/versions/7 https://vagrantcloud.com/stevetjoa/boxes/stanford-mir/versio...

jiaaro 12y ago
You're welcome :) Looks like a cool project!

tdicola 12y ago

Nice! I love little focused Python libraries like this. Will have to keep it in mind if I ever do audio stuff in the future.

bravura 12y ago

Cool project. The echonest remix API has similar capabilities, and can quantize into granularities of measures, beats, tatums. http://echonest.github.io/remix/ http://echonest.github.io/remix/

jestinjoy1 12y ago

Suggestions on some cool academic projects that could use this library?

DarkIye 12y ago

I started writing a sample-level audio suite in Java, for musical purposes: https://github.com/williamberg/audioDCB https://github.com/williamberg/audioDCB which I'll probably discard because Java is not the right tool for the job. Inspecting pydub/pydub/pyaudioop.py , I see there are methods for working on the sample level. I'll make a mental note to come back to this project when I get back on the computer music bandwagon again.