10 ms·

Can someone explain to me how FFmpeg seems to be the only open-source software to do even just basic functionality with audio. I was looking at getting the sou

Can someone explain to me how FFmpeg seems to be the only open-source software to do even just basic functionality with audio. I was looking at getting the sound wave graph for a piece of audio a while ago, and not only was FFmpeg the only option I found to be able to do it, it was amazingly fast and also free.

ArgyleSound 6y ago
why duplicate something that works really well?
- LaLaLand122 6y ago
  - You don't trust so much complex logic, taking untrusted input, written in C and want to rewrite it in Rust. - You want to code it all again using an API that doesn't expect to get its input from a blocking read() function. - ... I think the main reason there isn't any alternative is that it supports soooo many formats that the task seems impossible to anybody thinking about it.
  - Erlich_Bachman 6y ago
    > - You want to code it all again using an API that doesn't expect to get its input from a blocking read() function. In which real world situation/scenario is this a problem? It is hard to think of one, but I am probably missing something? In any case, if that was a real show-stopper, it would probably be much wiser to go with a fork that would modify that one thing, instead of re-writing the whole project.
    - toast0 6y ago
      I could see it being an issue if you were doing a bunch of streaming transcodes, and wanted that in an event loop instead of blocking... but a) you're probably going to want to control the number of simultaneous streams to a low enough number that you could just fork b) the responsible thing to do when decoding streams with ffmpeg is to disable all formats except your whitelisted format, but still sandbox the heck out of it, because there's been a lot of CVEs where a crafted input allows remote code execution Sandboxing is going to be much more complete if the ffmpeg process is only dealing with one input fd, one output fd (maybe an error reportint fd), and no network or filesystem access --- you don't want a decoder error to influence media you're encoding/decoding for another user.
KennyFromIT 6y ago
What ever happened to Facebook's (or was it Netflix?) technology to create a new unit of time measurement to help align audio and video files? I believe it was called a "flick"...
- aspenmayer 6y ago
  Looks like it's archived? https://github.com/facebookarchive/Flicks https://github.com/facebookarchive/Flicks via https://gizmodo.com/facebook-invented-a-new-unit-of-time-1822330819 https://gizmodo.com/facebook-invented-a-new-unit-of-time-182...
- kevin_thibedeau 6y ago
  Doesn't support NTSC. Better to force everyone to deal with irrational sampling rates than pretend they don't exist.
  - zerocrates 6y ago
    The NTSC framerates are weird, but they are rational, in the mathematical sense.
pitaj 6y ago
I don't know much, but isn't Audacity a thing?
- mattacular 6y ago
  Audacity is a great GUI for working with audio files. I would think it has a way to export a graph of the wave that it shows you when you open up an audio file. You can install an FFMPEG plugin for Audacity if you need broader support of audio formats (either import or export).
  - 72deluxe 6y ago
    Try https://github.com/bbc/audiowaveform https://github.com/bbc/audiowaveform for waveforms
  - voltagex_ 6y ago
    At least on Windows, Audacity only supports an ancient version of ffmpeg. I don't know why.
qppo 6y ago
I mean there's plenty of open source stuff for audio. Visualization/oscilloscope tools included. But that kind of visualization isn't what I'd call "basic" functionality, since it's functionally useless for the vast majority of audio applications.
coldpie 6y ago
In addition to what others have said, there's also gstreamer and its suite of plugins. I find gstreamer a bit easier to work with, although both are very complex pieces of software and each have their own quirks. If you're looking for audio production work, there's Ardour, although I haven't used it myself. http://ardour.org/ http://ardour.org/
- ufo 6y ago
  Does gstreamer have a command-line interface in addition to the libraries?
  - otherjason 6y ago
    Indeed it does; it's about as complex as ffmpeg, and in my opinion has a somewhat more intuitive interface for building up complicated pipelines of processing steps: https://gstreamer.freedesktop.org/documentation/tools/index.html?gi-language=c https://gstreamer.freedesktop.org/documentation/tools/index....
  - aljarry 6y ago
    There is a "test tool" for gstreamer pipelines: gst-launch, but generally it's encouraged to run the gstreamer more as a library instead. Example, if you have gstreamer libs installed: gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
    - oplav 6y ago
      You can use gst_parse_launch to create a pipeline using the launch syntax. I've found this helpful to prototype with gst-launch-1.0 and then pull into a separate program down the road. I found it to be pretty hairy trying to create and link all the individual elements manually in complex pipelines. https://gstreamer.freedesktop.org/documentation/gstreamer/gstparse.html?gi-language=c#gst_parse_launch https://gstreamer.freedesktop.org/documentation/gstreamer/gs...
klodolph 6y ago
SoX is amazing. http://sox.sourceforge.net/ http://sox.sourceforge.net/ It’s not perfect but it’s way easier to use for audio stuff than FFmpeg is. I have a bunch of scripts I reuse that do basic stuff like high-pass, normalize, automatically trim audio files, add fade-in or fade-out, downmix to mono, and then resample / dither to the right depth and size. It also will spit out spectrograms. Generally when I need to record a ton of sound clips, I chop the audio up and rename it in a GUI editor similar to Audacity, and then do all the processing in SoX. I might also do a bunch of work in a DAW beforehand.
- hiram112 6y ago
  I hadn't even thought about SOX 'till your comment in about 10 years. And looking at the page, there hasn't been a new release since 2015. From what I recall, it only worked on wav files back in the day, but now it supports OGG. But a lot has changed in even 5 years - does it even support MP3, as patents expired since then?
  - klodolph 6y ago
    I just use SoX for processing audio data, and then pass the result to LAME if I want an MP3. Each format has so many different options for encoding and metadata anyway. It’s not like video, where the sheer amount of data discourages you from working uncompressed. Sure, there hasn’t been a new release since 2015… but would that be necessary? It’s not missing any features I want.
  - JNRowe 6y ago
    > From what I recall, it only worked on wav files back in the day It depends on your build, but on my system it supports: 8svx aif aifc aiff aiffc al amb amr-nb amr-wb anb au avr awb caf cdda cdr cvs cvsd cvu dat dvms f32 f4 f64 f8 fap flac fssd gsm gsrt hcom htk ima ircam la lpc lpc10 lu mat mat4 mat5 maud mp2 mp3 nist ogg paf prc pvf raw s1 s16 s2 s24 s3 s32 s4 s8 sb sd2 sds sf sl sln smp snd sndfile sndr sndt sou sox sph sw txw u1 u16 u2 u24 u3 u32 u4 u8 ub ul uw vms voc vorbis vox w64 wav wavpcm wv wve xa xi. You can check your own with `sox --help`. On Debian mp3 support requires `libsox-fmt-mp3`.
  - 6y ago
deleted 6y ago
[deleted]
DonnyV 6y ago
What about Reaper? Its a full fledged DAW. http://reaper.fm/ http://reaper.fm/
- encom 6y ago
  Reaper is proprietary.
ivan_ah 6y ago
In case that's helpful, here is the code we use to generate waveforms using: .mp3 -(1)-> .wav -> numpy -> matplotlib: https://github.com/learningequality/pressurecooker/blob/master/pressurecooker/images.py#L138-L201 https://github.com/learningequality/pressurecooker/blob/mast... ffmpeg is used for the key step (1) ...
jononor 6y ago
Python with librosa is also quite practical. Can work interactively by doing it in a Jupyter notebook. Some tutorials (from 2015). https://www.youtube.com/watch?v=0ALKGR0I5MA https://www.youtube.com/watch?v=0ALKGR0I5MA https://www.youtube.com/watch?v=MhOdbtPhbLU https://www.youtube.com/watch?v=MhOdbtPhbLU Not affiliated with the project, just use it quite a lot.
johnvaluk 6y ago
My DAW is bash+sox+ecasound because I don't want to be distracted by visuals when working with audio. However, I just started working on a project involving about 15 hours of digital audio recorded under less than ideal circumstances a couple of decades ago and need a reliable way to analyze the data. SoX produces spectrograms that are insufficient for my needs and I've had reliability issues with Audacity. So far, DFasma looks very promising: https://gillesdegottex.gitlab.io/dfasma-website/ https://gillesdegottex.gitlab.io/dfasma-website/
- ttctciyf 6y ago
  Maybe jaaa[0] and japa[1] are worth a look (from Fons Adriaensen's suite of audio tools[2]) 0: https://kokkinizita.linuxaudio.org/linuxaudio/jaaa-pict.html https://kokkinizita.linuxaudio.org/linuxaudio/jaaa-pict.html 1: https://kokkinizita.linuxaudio.org/linuxaudio/japa-pict.html https://kokkinizita.linuxaudio.org/linuxaudio/japa-pict.html 2: https://kokkinizita.linuxaudio.org/linuxaudio/index.html https://kokkinizita.linuxaudio.org/linuxaudio/index.html

ArgyleSound 6y ago

why duplicate something that works really well?

LaLaLand122 6y ago
- You don't trust so much complex logic, taking untrusted input, written in C and want to rewrite it in Rust. - You want to code it all again using an API that doesn't expect to get its input from a blocking read() function. - ... I think the main reason there isn't any alternative is that it supports soooo many formats that the task seems impossible to anybody thinking about it.
- Erlich_Bachman 6y ago
  > - You want to code it all again using an API that doesn't expect to get its input from a blocking read() function. In which real world situation/scenario is this a problem? It is hard to think of one, but I am probably missing something? In any case, if that was a real show-stopper, it would probably be much wiser to go with a fork that would modify that one thing, instead of re-writing the whole project.
  - toast0 6y ago
    I could see it being an issue if you were doing a bunch of streaming transcodes, and wanted that in an event loop instead of blocking... but a) you're probably going to want to control the number of simultaneous streams to a low enough number that you could just fork b) the responsible thing to do when decoding streams with ffmpeg is to disable all formats except your whitelisted format, but still sandbox the heck out of it, because there's been a lot of CVEs where a crafted input allows remote code execution Sandboxing is going to be much more complete if the ffmpeg process is only dealing with one input fd, one output fd (maybe an error reportint fd), and no network or filesystem access --- you don't want a decoder error to influence media you're encoding/decoding for another user.

KennyFromIT 6y ago

What ever happened to Facebook's (or was it Netflix?) technology to create a new unit of time measurement to help align audio and video files? I believe it was called a "flick"...

aspenmayer 6y ago
Looks like it's archived? https://github.com/facebookarchive/Flicks https://github.com/facebookarchive/Flicks via https://gizmodo.com/facebook-invented-a-new-unit-of-time-1822330819 https://gizmodo.com/facebook-invented-a-new-unit-of-time-182...
kevin_thibedeau 6y ago
Doesn't support NTSC. Better to force everyone to deal with irrational sampling rates than pretend they don't exist.
- zerocrates 6y ago
  The NTSC framerates are weird, but they are rational, in the mathematical sense.

pitaj 6y ago

I don't know much, but isn't Audacity a thing?

mattacular 6y ago
Audacity is a great GUI for working with audio files. I would think it has a way to export a graph of the wave that it shows you when you open up an audio file. You can install an FFMPEG plugin for Audacity if you need broader support of audio formats (either import or export).
- 72deluxe 6y ago
  Try https://github.com/bbc/audiowaveform https://github.com/bbc/audiowaveform for waveforms
- voltagex_ 6y ago
  At least on Windows, Audacity only supports an ancient version of ffmpeg. I don't know why.

qppo 6y ago

I mean there's plenty of open source stuff for audio. Visualization/oscilloscope tools included. But that kind of visualization isn't what I'd call "basic" functionality, since it's functionally useless for the vast majority of audio applications.

coldpie 6y ago

In addition to what others have said, there's also gstreamer and its suite of plugins. I find gstreamer a bit easier to work with, although both are very complex pieces of software and each have their own quirks. If you're looking for audio production work, there's Ardour, although I haven't used it myself. http://ardour.org/ http://ardour.org/

ufo 6y ago
Does gstreamer have a command-line interface in addition to the libraries?
- otherjason 6y ago
  Indeed it does; it's about as complex as ffmpeg, and in my opinion has a somewhat more intuitive interface for building up complicated pipelines of processing steps: https://gstreamer.freedesktop.org/documentation/tools/index.html?gi-language=c https://gstreamer.freedesktop.org/documentation/tools/index....
- aljarry 6y ago
  There is a "test tool" for gstreamer pipelines: gst-launch, but generally it's encouraged to run the gstreamer more as a library instead. Example, if you have gstreamer libs installed: gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
  - oplav 6y ago
    You can use gst_parse_launch to create a pipeline using the launch syntax. I've found this helpful to prototype with gst-launch-1.0 and then pull into a separate program down the road. I found it to be pretty hairy trying to create and link all the individual elements manually in complex pipelines. https://gstreamer.freedesktop.org/documentation/gstreamer/gstparse.html?gi-language=c#gst_parse_launch https://gstreamer.freedesktop.org/documentation/gstreamer/gs...

klodolph 6y ago

SoX is amazing. http://sox.sourceforge.net/ http://sox.sourceforge.net/ It’s not perfect but it’s way easier to use for audio stuff than FFmpeg is. I have a bunch of scripts I reuse that do basic stuff like high-pass, normalize, automatically trim audio files, add fade-in or fade-out, downmix to mono, and then resample / dither to the right depth and size. It also will spit out spectrograms. Generally when I need to record a ton of sound clips, I chop the audio up and rename it in a GUI editor similar to Audacity, and then do all the processing in SoX. I might also do a bunch of work in a DAW beforehand.

hiram112 6y ago
I hadn't even thought about SOX 'till your comment in about 10 years. And looking at the page, there hasn't been a new release since 2015. From what I recall, it only worked on wav files back in the day, but now it supports OGG. But a lot has changed in even 5 years - does it even support MP3, as patents expired since then?
- klodolph 6y ago
  I just use SoX for processing audio data, and then pass the result to LAME if I want an MP3. Each format has so many different options for encoding and metadata anyway. It’s not like video, where the sheer amount of data discourages you from working uncompressed. Sure, there hasn’t been a new release since 2015… but would that be necessary? It’s not missing any features I want.
- JNRowe 6y ago
  > From what I recall, it only worked on wav files back in the day It depends on your build, but on my system it supports: 8svx aif aifc aiff aiffc al amb amr-nb amr-wb anb au avr awb caf cdda cdr cvs cvsd cvu dat dvms f32 f4 f64 f8 fap flac fssd gsm gsrt hcom htk ima ircam la lpc lpc10 lu mat mat4 mat5 maud mp2 mp3 nist ogg paf prc pvf raw s1 s16 s2 s24 s3 s32 s4 s8 sb sd2 sds sf sl sln smp snd sndfile sndr sndt sou sox sph sw txw u1 u16 u2 u24 u3 u32 u4 u8 ub ul uw vms voc vorbis vox w64 wav wavpcm wv wve xa xi. You can check your own with `sox --help`. On Debian mp3 support requires `libsox-fmt-mp3`.
- 6y ago

deleted 6y ago

[deleted]

DonnyV 6y ago

What about Reaper? Its a full fledged DAW. http://reaper.fm/ http://reaper.fm/

encom 6y ago
Reaper is proprietary.

ivan_ah 6y ago

In case that's helpful, here is the code we use to generate waveforms using: .mp3 -(1)-> .wav -> numpy -> matplotlib: https://github.com/learningequality/pressurecooker/blob/master/pressurecooker/images.py#L138-L201 https://github.com/learningequality/pressurecooker/blob/mast... ffmpeg is used for the key step (1) ...

jononor 6y ago

Python with librosa is also quite practical. Can work interactively by doing it in a Jupyter notebook. Some tutorials (from 2015). https://www.youtube.com/watch?v=0ALKGR0I5MA https://www.youtube.com/watch?v=0ALKGR0I5MA https://www.youtube.com/watch?v=MhOdbtPhbLU https://www.youtube.com/watch?v=MhOdbtPhbLU Not affiliated with the project, just use it quite a lot.

johnvaluk 6y ago

My DAW is bash+sox+ecasound because I don't want to be distracted by visuals when working with audio. However, I just started working on a project involving about 15 hours of digital audio recorded under less than ideal circumstances a couple of decades ago and need a reliable way to analyze the data. SoX produces spectrograms that are insufficient for my needs and I've had reliability issues with Audacity. So far, DFasma looks very promising: https://gillesdegottex.gitlab.io/dfasma-website/ https://gillesdegottex.gitlab.io/dfasma-website/

ttctciyf 6y ago
Maybe jaaa[0] and japa[1] are worth a look (from Fons Adriaensen's suite of audio tools[2]) 0: https://kokkinizita.linuxaudio.org/linuxaudio/jaaa-pict.html https://kokkinizita.linuxaudio.org/linuxaudio/jaaa-pict.html 1: https://kokkinizita.linuxaudio.org/linuxaudio/japa-pict.html https://kokkinizita.linuxaudio.org/linuxaudio/japa-pict.html 2: https://kokkinizita.linuxaudio.org/linuxaudio/index.html https://kokkinizita.linuxaudio.org/linuxaudio/index.html