8 ms·

What Paris looks like with an echo

95 points by SanderMak 12y ago

Artemis2 12y ago
For glitch art amateurs, I'd advise /r/glitch_art: https://www.reddit.com/r/glitch_art https://www.reddit.com/r/glitch_art.
graeham 12y ago
The effects are pretty cool, but I can't help but wonder how much better it would be if done with a bit of intention and knowledge of what these algorithms are actually doing. "We found that while the raw data makes no music, the waveforms share patterns" Its not that surprising, since music is frequency based while pictures are spatially based. I expect you could get something that sounded like music from a picture if you did some Fourier-transform filtering before with different interval domains (different sizes of 'chunks' being converted to frequencies).
- parksy 12y ago
  These days there are a number of audio synthesizers which do basically what you said, such as the Beepmap plugin that comes stock with FL Studio. Among the first to use the technique artistically, Aphex Twin famously drew his own face in the spectrum of his track "Equation" (https://www.youtube.com/watch?v=wSYAZnQmffg https://www.youtube.com/watch?v=wSYAZnQmffg - skip to 5:20 for the face). Here's a few more, if you're interested: http://twistedsifter.com/2013/01/hidden-images-embedded-into-songs-spectrographs/ http://twistedsifter.com/2013/01/hidden-images-embedded-into...
  - kumarharsh 12y ago
    the Equation song does sound really interesting!
thegeomaster 12y ago
I suspect that better sounds could be made from images if the data was treated as if it were in frequency domain instead of time domain, because then some filtering could make more sense. This could open up some more possibilities, such as even transforming color distribution data into musical notes (I wonder if nice-looking color chords can be transformed into nice-sounding musical chords). This is very interesting, and I love both coding and music, so I guess I'll get on making something like that :)
- stephencanon 12y ago
  You're looking for MetaSynth (http://www.uisoftware.com/MetaSynth/index.php http://www.uisoftware.com/MetaSynth/index.php) or one of a variety of other similar tools.
- nakkiel 12y ago
  Sounds like decent as it is to me. I guess it all depends on your musical culture (not intended as a derogatory remark).
- yread 12y ago
  I'm not sure I understand your comment, but I think that's what is already happening. Consider this: WAV is a sequence of samples - BMP is sequence of RGB values. So you can more or less treat it the same. If you would try to read JPEG (i.e. frequency domain data) as WAV it would be a mess. The "appropriate" format would then be something like MP3 but you would run into issues - MP3 contains more metadata in the streams, JPEG encodes 8x8px 2D images while MP3 encodes ~576 1D samples...
janaagaard 12y ago
Side note: You actually need permission to use pictures of the Eiffel Tower if it is illuminated. In other words "it is no longer legal to publish contemporary photographs of the tower at night without permission in France and some other countries." https://en.wikipedia.org/wiki/Eiffel_Tower#Image_copyright_claims https://en.wikipedia.org/wiki/Eiffel_Tower#Image_copyright_c...
- robin_reala 12y ago
  This happens quite a bit. I took a nice photo of the Asahi building[0] in Tokyo and tried to add it to Wikipedia, only to be told that the gold embellishment on top was considered sculpture under Japanese law and therefore subject to copyright and not permissible on Wikimedia. [0] https://encrypted.google.com/search?tbm=isch&q=asahi%20building&tbs=imgo:1 https://encrypted.google.com/search?tbm=isch&q=asahi%20build...
  - _delirium 12y ago
    In some countries even just regular buildings are considered copyrighted if they have at all nontrivial decoration or design, and photos of them require permission of the person/company that holds the design copyright on the facade. The keyword to find each country's laws is "freedom of panorama", an exception to copyright law that allows photographing public places, even if the photograph includes something copyrighted. Some countries give a blanket exception; others give an exception only for buildings (but not sculptures or other artwork that appears in public, which can complicate photographing a building if it also has sculptures); and those differ on whether it includes only building exteriors or also photographs of the interiors of public lobbies; still others have freedom of panorama but only for non-commercial use; etc. Due to having to sort it out repeatedly, I think Wikimedia has the most up-to-date rundown of each country's laws on the subject: https://commons.wikimedia.org/wiki/Commons:Freedom_of_panorama https://commons.wikimedia.org/wiki/Commons:Freedom_of_panora...
- sailfast 12y ago
  Thank you for enlightening me on this issue. I had no idea this kind of thing was even desirable for sovereign territories, let alone actually in place legally. The lengths some people will go...
thisjepisje 12y ago
I think it would've been more interesting to apply a fourier transformation to the image, convert that to audio, and apply the wah-wah (which is essentially just a low-pass filter) to that.
- mbrock 12y ago
  Why would you need a Fourier transformation to apply a filter?
  - thisjepisje 12y ago
    Because then you'd actually be low-passing the frequencies present in the image, instead of some random bits.
    - mbrock 12y ago
      Huh? If you can apply an FFT and apply a filter in the frequency domain, what's preventing you from applying the convolution in the spatial domain?
  - boomlinde 12y ago
    The filters used in the article are meant to be applied on linear sample streams. A RAW image is not a linear sample stream. Re-encoding the image into a linear sample stream, applying the effect and then decoding the resulting signal to a RAW image again could be truer to the idea of "Paris with an echo".
kimburgess 12y ago
On a similar note, the images to sounds thing has been used by musicians / producers for a while. One of the more high profile users of this is Aphex Twin. [0] http://en.wikipedia.org/wiki/Windowlicker#Hidden_images http://en.wikipedia.org/wiki/Windowlicker#Hidden_images [1] http://www.boiledbeans.net/wp-content/uploads/2011/03/1ac7454c7382aefbd50246fca08c0c6a.png http://www.boiledbeans.net/wp-content/uploads/2011/03/1ac745...
- rch 12y ago
  I was going to post exactly this. The effect in Windowlicker is definitely memorable.
ris 12y ago
Oh jesus I can't believe this made the washington post. It's basically a bunch of people not-really-understanding DSP. It's a highly known and highly explored field.
- skrebbel 12y ago
  Yep. It's basically fractal art all over again. "Press random buttons until something looks vaguely interesting"
pdkl95 12y ago
http://www.warmplace.ru/soft/pixivisor/ http://www.warmplace.ru/soft/pixivisor/ If you like this, I suggest checking out PixiVisor. It's an experimental tool by NightRadio (Alexander Zolotov), who also made the amazing modular synth + tracker "SunVox". PixiVisor takes a short video loop (~animated-GIF or sequence of images), modulates the frames (progressive-scan) into audio. A 2nd instance of the app (usually on another device) demodulates the audio back into similar video frames automagically. "Similar"? Well... after picking up the multi-path room echoes and the "colorful" transfer functions of the DAC, amp, speaker, mic, ADC, etc, there is substantial distortion. This is intentional, as you can vary the signal considerably and the receiver app still reliably demodulates it back into video. This hit the "wait, isn't that impossible?!" stage for me in this demo video: https://www.youtube.com/watch?v=HkdLAWPsONQ https://www.youtube.com/watch?v=HkdLAWPsONQ Using two mono channels of an audio mixer to mix video live is already insane. Then the effect insert is turned up, and various types of reverb get added to the video frames in a way that perfectly matches what you hear. Also, this video, of live room echoes and hand waves that clearly result in changing reflections/blocking across many frequencies. Multi-path is also very visible. https://www.youtube.com/watch?v=mF5C93cDocA https://www.youtube.com/watch?v=mF5C93cDocA It's even multi-platform! (Windows, Linux, OSX, Android, iOS) /* sigh, now I'm going to be goofing off with SunVox instead of writing RSpec */
- fuzzix 12y ago
  > by NightRadio (Alexander Zolotov), who also made the amazing modular synth + tracker "SunVox". Heh, first thing I thought when reading the article was SunVox. PixiVisor is closer in effect to what the article describes, but I was reminded of the "SunVox: Sampler eats JPEG images" video: https://www.youtube.com/watch?v=ColyxxBzeiE https://www.youtube.com/watch?v=ColyxxBzeiE
deleted 12y ago
[deleted]
boomlinde 12y ago
It's not Paris with an echo as much as it is the raw data of a specific digital encoding of a picture of Paris fed through a digital filter suited for linear streams of samples. The effect is a direct result of the encoding more than anything.
mxfh 12y ago
Whats next? Somebody discovers what a High-pass filter can do to audio or even images? https://en.wikipedia.org/wiki/High-pass_filter#Applications https://en.wikipedia.org/wiki/High-pass_filter#Applications
JonnieCache 12y ago
For all the naysayers, not understanding DSP is the whole point. The naive abuse of tools designed for other tasks is a powerful thing, its a totally different way of working than deliberate application of reasoned technique.

95 points by SanderMak 12y ago

For glitch art amateurs, I'd advise /r/glitch_art: https://www.reddit.com/r/glitch_art https://www.reddit.com/r/glitch_art.

graeham 12y ago

The effects are pretty cool, but I can't help but wonder how much better it would be if done with a bit of intention and knowledge of what these algorithms are actually doing. "We found that while the raw data makes no music, the waveforms share patterns" Its not that surprising, since music is frequency based while pictures are spatially based. I expect you could get something that sounded like music from a picture if you did some Fourier-transform filtering before with different interval domains (different sizes of 'chunks' being converted to frequencies).

parksy 12y ago
These days there are a number of audio synthesizers which do basically what you said, such as the Beepmap plugin that comes stock with FL Studio. Among the first to use the technique artistically, Aphex Twin famously drew his own face in the spectrum of his track "Equation" (https://www.youtube.com/watch?v=wSYAZnQmffg https://www.youtube.com/watch?v=wSYAZnQmffg - skip to 5:20 for the face). Here's a few more, if you're interested: http://twistedsifter.com/2013/01/hidden-images-embedded-into-songs-spectrographs/ http://twistedsifter.com/2013/01/hidden-images-embedded-into...
- kumarharsh 12y ago
  the Equation song does sound really interesting!

thegeomaster 12y ago

I suspect that better sounds could be made from images if the data was treated as if it were in frequency domain instead of time domain, because then some filtering could make more sense. This could open up some more possibilities, such as even transforming color distribution data into musical notes (I wonder if nice-looking color chords can be transformed into nice-sounding musical chords). This is very interesting, and I love both coding and music, so I guess I'll get on making something like that :)

stephencanon 12y ago
You're looking for MetaSynth (http://www.uisoftware.com/MetaSynth/index.php http://www.uisoftware.com/MetaSynth/index.php) or one of a variety of other similar tools.
nakkiel 12y ago
Sounds like decent as it is to me. I guess it all depends on your musical culture (not intended as a derogatory remark).
yread 12y ago
I'm not sure I understand your comment, but I think that's what is already happening. Consider this: WAV is a sequence of samples - BMP is sequence of RGB values. So you can more or less treat it the same. If you would try to read JPEG (i.e. frequency domain data) as WAV it would be a mess. The "appropriate" format would then be something like MP3 but you would run into issues - MP3 contains more metadata in the streams, JPEG encodes 8x8px 2D images while MP3 encodes ~576 1D samples...

janaagaard 12y ago

Side note: You actually need permission to use pictures of the Eiffel Tower if it is illuminated. In other words "it is no longer legal to publish contemporary photographs of the tower at night without permission in France and some other countries." https://en.wikipedia.org/wiki/Eiffel_Tower#Image_copyright_claims https://en.wikipedia.org/wiki/Eiffel_Tower#Image_copyright_c...

robin_reala 12y ago
This happens quite a bit. I took a nice photo of the Asahi building[0] in Tokyo and tried to add it to Wikipedia, only to be told that the gold embellishment on top was considered sculpture under Japanese law and therefore subject to copyright and not permissible on Wikimedia. [0] https://encrypted.google.com/search?tbm=isch&q=asahi%20building&tbs=imgo:1 https://encrypted.google.com/search?tbm=isch&q=asahi%20build...
- _delirium 12y ago
  In some countries even just regular buildings are considered copyrighted if they have at all nontrivial decoration or design, and photos of them require permission of the person/company that holds the design copyright on the facade. The keyword to find each country's laws is "freedom of panorama", an exception to copyright law that allows photographing public places, even if the photograph includes something copyrighted. Some countries give a blanket exception; others give an exception only for buildings (but not sculptures or other artwork that appears in public, which can complicate photographing a building if it also has sculptures); and those differ on whether it includes only building exteriors or also photographs of the interiors of public lobbies; still others have freedom of panorama but only for non-commercial use; etc. Due to having to sort it out repeatedly, I think Wikimedia has the most up-to-date rundown of each country's laws on the subject: https://commons.wikimedia.org/wiki/Commons:Freedom_of_panorama https://commons.wikimedia.org/wiki/Commons:Freedom_of_panora...
sailfast 12y ago
Thank you for enlightening me on this issue. I had no idea this kind of thing was even desirable for sovereign territories, let alone actually in place legally. The lengths some people will go...

thisjepisje 12y ago

I think it would've been more interesting to apply a fourier transformation to the image, convert that to audio, and apply the wah-wah (which is essentially just a low-pass filter) to that.

mbrock 12y ago
Why would you need a Fourier transformation to apply a filter?
- thisjepisje 12y ago
  Because then you'd actually be low-passing the frequencies present in the image, instead of some random bits.
  - mbrock 12y ago
    Huh? If you can apply an FFT and apply a filter in the frequency domain, what's preventing you from applying the convolution in the spatial domain?
- boomlinde 12y ago
  The filters used in the article are meant to be applied on linear sample streams. A RAW image is not a linear sample stream. Re-encoding the image into a linear sample stream, applying the effect and then decoding the resulting signal to a RAW image again could be truer to the idea of "Paris with an echo".

kimburgess 12y ago

On a similar note, the images to sounds thing has been used by musicians / producers for a while. One of the more high profile users of this is Aphex Twin. [0] http://en.wikipedia.org/wiki/Windowlicker#Hidden_images http://en.wikipedia.org/wiki/Windowlicker#Hidden_images [1] http://www.boiledbeans.net/wp-content/uploads/2011/03/1ac7454c7382aefbd50246fca08c0c6a.png http://www.boiledbeans.net/wp-content/uploads/2011/03/1ac745...

rch 12y ago
I was going to post exactly this. The effect in Windowlicker is definitely memorable.

ris 12y ago

Oh jesus I can't believe this made the washington post. It's basically a bunch of people not-really-understanding DSP. It's a highly known and highly explored field.

skrebbel 12y ago
Yep. It's basically fractal art all over again. "Press random buttons until something looks vaguely interesting"

pdkl95 12y ago

http://www.warmplace.ru/soft/pixivisor/ http://www.warmplace.ru/soft/pixivisor/ If you like this, I suggest checking out PixiVisor. It's an experimental tool by NightRadio (Alexander Zolotov), who also made the amazing modular synth + tracker "SunVox". PixiVisor takes a short video loop (~animated-GIF or sequence of images), modulates the frames (progressive-scan) into audio. A 2nd instance of the app (usually on another device) demodulates the audio back into similar video frames automagically. "Similar"? Well... after picking up the multi-path room echoes and the "colorful" transfer functions of the DAC, amp, speaker, mic, ADC, etc, there is substantial distortion. This is intentional, as you can vary the signal considerably and the receiver app still reliably demodulates it back into video. This hit the "wait, isn't that impossible?!" stage for me in this demo video: https://www.youtube.com/watch?v=HkdLAWPsONQ https://www.youtube.com/watch?v=HkdLAWPsONQ Using two mono channels of an audio mixer to mix video live is already insane. Then the effect insert is turned up, and various types of reverb get added to the video frames in a way that perfectly matches what you hear. Also, this video, of live room echoes and hand waves that clearly result in changing reflections/blocking across many frequencies. Multi-path is also very visible. https://www.youtube.com/watch?v=mF5C93cDocA https://www.youtube.com/watch?v=mF5C93cDocA It's even multi-platform! (Windows, Linux, OSX, Android, iOS) /* sigh, now I'm going to be goofing off with SunVox instead of writing RSpec */

fuzzix 12y ago
> by NightRadio (Alexander Zolotov), who also made the amazing modular synth + tracker "SunVox". Heh, first thing I thought when reading the article was SunVox. PixiVisor is closer in effect to what the article describes, but I was reminded of the "SunVox: Sampler eats JPEG images" video: https://www.youtube.com/watch?v=ColyxxBzeiE https://www.youtube.com/watch?v=ColyxxBzeiE

deleted 12y ago

[deleted]

boomlinde 12y ago

It's not Paris with an echo as much as it is the raw data of a specific digital encoding of a picture of Paris fed through a digital filter suited for linear streams of samples. The effect is a direct result of the encoding more than anything.

mxfh 12y ago

Whats next? Somebody discovers what a High-pass filter can do to audio or even images? https://en.wikipedia.org/wiki/High-pass_filter#Applications https://en.wikipedia.org/wiki/High-pass_filter#Applications

JonnieCache 12y ago

For all the naysayers, not understanding DSP is the whole point. The naive abuse of tools designed for other tasks is a powerful thing, its a totally different way of working than deliberate application of reasoned technique.