4 ms·

Awk for multimedia

154 points by poweredbyptrs 9y ago

egwynn 9y ago
Cool! Reminds me of GStreamer (https://en.wikipedia.org/wiki/GStreamer https://en.wikipedia.org/wiki/GStreamer). It’s definitely like AWK insofar as it’s pipeline-based. But to me the big thing about AWK is that the program doing the transformation is usually super easy to write down in the CLI itself. Either way, sounds fun!
- jcelerier 9y ago
  > But to me the big thing about AWK is that the program doing the transformation is usually super easy to write down in the CLI itself GStreamer is also very easy to use from the CLI: https://gstreamer.freedesktop.org/documentation/tutorials/basic/gstreamer-tools.html
j_s 9y ago
Awesome! Other video tools (besides ffmpeg): https://www.mltframework.org/ https://www.mltframework.org/ (lightweight LGPL C99 perhaps similar to OP) http://www.emmgunn.com/ http://www.emmgunn.com/ (mac VIDEOtoolbox), https://subler.org/ https://subler.org/ (mac mp4) https://sourceforge.net/projects/megui/ https://sourceforge.net/projects/megui/ (windows, recommended for SD sources) https://github.com/stax76/staxrip https://github.com/stax76/staxrip (windows, a better handbrake) Further pointers to similar tools are appreciated! Grabbed several of these from this awesome discussion of a weak article on Handbrake: https://news.ycombinator.com/item?id=13492646 https://news.ycombinator.com/item?id=13492646 (Jan 2017, 59 comments) PS. Also looking for pros/cons/alternatives re: print LUT http://juanmelara.com.au/print-film-emulation-luts-for-download/ http://juanmelara.com.au/print-film-emulation-luts-for-downl... and histogram equalization http://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_equalization.html http://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_eq...
- aikinai 9y ago
  Thanks for sharing all of these tools! Do you happen to know if any of these can write metadata to mp4 or mov files that will be recognized by Mac/iOS Photos? I've been trying to write location, camera, date, etc. to encoded videos, but Photos won't recognize anything except files straight off an iPhone. The closest thing I've found is that ffmpeg can write Quicktime tags and the structure looks identical to me when I view it in exiftool, but Photos still doesn't respect it, so I assume there's some minute difference that breaks Photos' parsing.
  - 0bit 9y ago
    I've used Atomic Parsley[1] and MP4Box[2] successfully for various MOV and MP4 related muxing and metadata handling before. [1] http://atomicparsley.sourceforge.net http://atomicparsley.sourceforge.net [2] https://gpac.wp.imt.fr/mp4box/ https://gpac.wp.imt.fr/mp4box/
  - j_s 9y ago
    Unfortunately I do not have any personal experience with your problem since I purposely avoid Apple Photos (perhaps helpful though not related: I rely on Google+ Photos as my free [lossy] offsite backup). - Date/time? http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7900.msg40307.html#msg40307 http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,790... recommends https://github.com/HayoBaan/matchDateTime https://github.com/HayoBaan/matchDateTime and/or exiftool -a -G -s -time:all [filename] - Lat/long location? works for JPG, only recently for PNG http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,8444.msg43394.html#msg43394 http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,844... - camera? for Lightroom http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,8583.msg44089.html#msg44089 http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,858...
- softwarelimits 9y ago
  Also take a look at https://github.com/Zulko/moviepy https://github.com/Zulko/moviepy
- acj 9y ago
  Natron is worth a look as well: https://natron.fr https://natron.fr Mac/Linux, Python API, mostly usable docs, and the source is available.
- j_s 9y ago
  I left this special-purpose tool out on my initial round-up, but anyone still paying attention might find a use for it: https://github.com/slowmoVideo/slowmoVideo https://github.com/slowmoVideo/slowmoVideo
AdmiralAsshat 9y ago
This seems more like pipe-based processing for multimedia than AWK-based. I don't see what AWK does in relation to this that any good UNIX-y utility that is pipe-friendly is doing (grep, sed, etc).
- anitil 9y ago
  I was completely prepared to gush about how much I love AWK. I suppose that can wait for another day
- agumonkey 9y ago
  more like imagemagick in a way
diggernet 9y ago
How does this compare with AviSynth/AvxSynth?
- ComodoHacker 9y ago
  Looks like a good alternative.
jerkstate 9y ago
this looks pretty cool, reminiscent of the "shake" compositing application for video
- agumonkey 9y ago
  haa the glorious shake. Compositing tools were lovely graph and lazy dataflow gems.
sengork 9y ago
This reminds me of AWK Music http://kmkeen.com/awk-music/ http://kmkeen.com/awk-music/
- Bromskloss 9y ago
  > Using the fifth root gives five intervals for each octave, a pentatonic scale. Why five notes? It is really easy to make something that sounds good in the pentatonic scale and really hard for a pair of notes to be dissonant. Wait a minute! Is the five-note equal-division of the octave really one of those scales that "just sound good"? It's definitely not the same scale as the example he links to.
  - anamexis 9y ago
    I tried this out using overtone, and it doesn't sound too terribly dissonant, but I agree, nothing like the major pentatonic scale linked in the video. (Also, overtone is fun! here's my repl session) (require '[clojure.math.numeric-tower :as math]) (use 'overtone.live) (def freqs (map #(* 440 (math/expt 2 (/ % 5))) (range 0 5))) (map #(demo (sin-osc %)) freqs) (map #(demo (sin-osc %)) (take 2 freqs)) (map #(demo (sin-osc %)) (take 3 freqs)) (map #(demo (sin-osc %)) (drop 2 freqs))
  - boomlinde 9y ago
    The pentatonic scale does not imply equal temperament between the five notes. The "good-sounding" pentatonic scales use very simple harmonic ratios, for example 1, 9/8, 5/4, 3/2 and 5/3 for the major pentatonic scale. In the equally tempered 12-note octave, these roughly correspond to the first, second, third, fifth and sixth. An equal division of an octave into five notes doesn't yield as harmonic results. That said, 5-EDO sounds good in my opinion. Not very harmonic, but somewhat resembles the scales used Gamelan music.
- navaati 9y ago
  For those like me that want to hear it on a modern distribution: instead of redirecting to >/dev/dsp, pipe into |pacat. Works like a charm.
golergka 9y ago
May be I didn't pay it enough time, but this seems to be just pipe-based video processing. I thought that the main feature in awk was to work with meaning in the text itself - so I expected some machine-learning magic to find all dogs and replace them with cats, for example.
- poweredbyptrs 9y ago
  The pipes are only used to communicate higher-level IPC primitives. The real data exchange and processing is bidirectional. This is why it is possible for the last client in the chain to provide the inputs that control the game that was being run by the first client in the chain. There is also audio. The part that I found cool is that the possible data sources aren't just videos, it's emulators, virtual machines, terminals, and so on. "the example here only works with a single pipe-and-filter chain, but there is nothing preventing arbitrary, even dynamic, graphs to be created." (and AWK is a DSL for streaming text processing - it's from the late 70ies.. it's dumb as a rock - there's no high level analysis features)
defined 9y ago
Sorry, but this article has nothing to do with AWK other than mentioning its name. AWK is a pattern-action, implied-loop text processing scripting language that supports user-defined functions and interaction with the OS via pipes and system(). It may have been better titled "Arcan is to multimedia as AWK is to text", or something.

154 points by poweredbyptrs 9y ago

Cool! Reminds me of GStreamer (https://en.wikipedia.org/wiki/GStreamer https://en.wikipedia.org/wiki/GStreamer). It’s definitely like AWK insofar as it’s pipeline-based. But to me the big thing about AWK is that the program doing the transformation is usually super easy to write down in the CLI itself. Either way, sounds fun!

jcelerier 9y ago
> But to me the big thing about AWK is that the program doing the transformation is usually super easy to write down in the CLI itself GStreamer is also very easy to use from the CLI: https://gstreamer.freedesktop.org/documentation/tutorials/basic/gstreamer-tools.html

j_s 9y ago

Awesome! Other video tools (besides ffmpeg): https://www.mltframework.org/ https://www.mltframework.org/ (lightweight LGPL C99 perhaps similar to OP) http://www.emmgunn.com/ http://www.emmgunn.com/ (mac VIDEOtoolbox), https://subler.org/ https://subler.org/ (mac mp4) https://sourceforge.net/projects/megui/ https://sourceforge.net/projects/megui/ (windows, recommended for SD sources) https://github.com/stax76/staxrip https://github.com/stax76/staxrip (windows, a better handbrake) Further pointers to similar tools are appreciated! Grabbed several of these from this awesome discussion of a weak article on Handbrake: https://news.ycombinator.com/item?id=13492646 https://news.ycombinator.com/item?id=13492646 (Jan 2017, 59 comments) PS. Also looking for pros/cons/alternatives re: print LUT http://juanmelara.com.au/print-film-emulation-luts-for-download/ http://juanmelara.com.au/print-film-emulation-luts-for-downl... and histogram equalization http://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_equalization.html http://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_eq...

aikinai 9y ago
Thanks for sharing all of these tools! Do you happen to know if any of these can write metadata to mp4 or mov files that will be recognized by Mac/iOS Photos? I've been trying to write location, camera, date, etc. to encoded videos, but Photos won't recognize anything except files straight off an iPhone. The closest thing I've found is that ffmpeg can write Quicktime tags and the structure looks identical to me when I view it in exiftool, but Photos still doesn't respect it, so I assume there's some minute difference that breaks Photos' parsing.
- 0bit 9y ago
  I've used Atomic Parsley[1] and MP4Box[2] successfully for various MOV and MP4 related muxing and metadata handling before. [1] http://atomicparsley.sourceforge.net http://atomicparsley.sourceforge.net [2] https://gpac.wp.imt.fr/mp4box/ https://gpac.wp.imt.fr/mp4box/
- j_s 9y ago
  Unfortunately I do not have any personal experience with your problem since I purposely avoid Apple Photos (perhaps helpful though not related: I rely on Google+ Photos as my free [lossy] offsite backup). - Date/time? http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7900.msg40307.html#msg40307 http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,790... recommends https://github.com/HayoBaan/matchDateTime https://github.com/HayoBaan/matchDateTime and/or exiftool -a -G -s -time:all [filename] - Lat/long location? works for JPG, only recently for PNG http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,8444.msg43394.html#msg43394 http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,844... - camera? for Lightroom http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,8583.msg44089.html#msg44089 http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,858...
softwarelimits 9y ago
Also take a look at https://github.com/Zulko/moviepy https://github.com/Zulko/moviepy
acj 9y ago
Natron is worth a look as well: https://natron.fr https://natron.fr Mac/Linux, Python API, mostly usable docs, and the source is available.
j_s 9y ago
I left this special-purpose tool out on my initial round-up, but anyone still paying attention might find a use for it: https://github.com/slowmoVideo/slowmoVideo https://github.com/slowmoVideo/slowmoVideo

AdmiralAsshat 9y ago

This seems more like pipe-based processing for multimedia than AWK-based. I don't see what AWK does in relation to this that any good UNIX-y utility that is pipe-friendly is doing (grep, sed, etc).

anitil 9y ago
I was completely prepared to gush about how much I love AWK. I suppose that can wait for another day
agumonkey 9y ago
more like imagemagick in a way

diggernet 9y ago

How does this compare with AviSynth/AvxSynth?

ComodoHacker 9y ago
Looks like a good alternative.

jerkstate 9y ago

this looks pretty cool, reminiscent of the "shake" compositing application for video

agumonkey 9y ago
haa the glorious shake. Compositing tools were lovely graph and lazy dataflow gems.

sengork 9y ago

This reminds me of AWK Music http://kmkeen.com/awk-music/ http://kmkeen.com/awk-music/

Bromskloss 9y ago
> Using the fifth root gives five intervals for each octave, a pentatonic scale. Why five notes? It is really easy to make something that sounds good in the pentatonic scale and really hard for a pair of notes to be dissonant. Wait a minute! Is the five-note equal-division of the octave really one of those scales that "just sound good"? It's definitely not the same scale as the example he links to.
- anamexis 9y ago
  I tried this out using overtone, and it doesn't sound too terribly dissonant, but I agree, nothing like the major pentatonic scale linked in the video. (Also, overtone is fun! here's my repl session) (require '[clojure.math.numeric-tower :as math]) (use 'overtone.live) (def freqs (map #(* 440 (math/expt 2 (/ % 5))) (range 0 5))) (map #(demo (sin-osc %)) freqs) (map #(demo (sin-osc %)) (take 2 freqs)) (map #(demo (sin-osc %)) (take 3 freqs)) (map #(demo (sin-osc %)) (drop 2 freqs))
- boomlinde 9y ago
  The pentatonic scale does not imply equal temperament between the five notes. The "good-sounding" pentatonic scales use very simple harmonic ratios, for example 1, 9/8, 5/4, 3/2 and 5/3 for the major pentatonic scale. In the equally tempered 12-note octave, these roughly correspond to the first, second, third, fifth and sixth. An equal division of an octave into five notes doesn't yield as harmonic results. That said, 5-EDO sounds good in my opinion. Not very harmonic, but somewhat resembles the scales used Gamelan music.
navaati 9y ago
For those like me that want to hear it on a modern distribution: instead of redirecting to >/dev/dsp, pipe into |pacat. Works like a charm.

golergka 9y ago

May be I didn't pay it enough time, but this seems to be just pipe-based video processing. I thought that the main feature in awk was to work with meaning in the text itself - so I expected some machine-learning magic to find all dogs and replace them with cats, for example.

poweredbyptrs 9y ago
The pipes are only used to communicate higher-level IPC primitives. The real data exchange and processing is bidirectional. This is why it is possible for the last client in the chain to provide the inputs that control the game that was being run by the first client in the chain. There is also audio. The part that I found cool is that the possible data sources aren't just videos, it's emulators, virtual machines, terminals, and so on. "the example here only works with a single pipe-and-filter chain, but there is nothing preventing arbitrary, even dynamic, graphs to be created." (and AWK is a DSL for streaming text processing - it's from the late 70ies.. it's dumb as a rock - there's no high level analysis features)

defined 9y ago

Sorry, but this article has nothing to do with AWK other than mentioning its name. AWK is a pattern-action, implied-loop text processing scripting language that supports user-defined functions and interaction with the OS via pipes and system(). It may have been better titled "Arcan is to multimedia as AWK is to text", or something.