4 ms·

The obvious question here is: what does JPEG encoded music sound like?

by kardos 10y ago

agumonkey 10y ago
Are you trying to fund a gif music player ?
ygra 10y ago
Horrible. I actually tried applying image compression to sounds over a decade ago (was still in school, did that for Jugend forscht). I used a Fourier transform to generate image sections of a specific window of samples and used that. In hindsight, perhaps just treating the waveform as image data may have been better. The idea behind what I did was to see whether a spectrogram can be compressed well, considering that there's lots of repetition and blank space usually. In any case, JPEG murders pretty much everything in thag representation.
- retroencabulato 10y ago
  It kind of reads like you have discredited frequency domain compression algorithms. FYI JPEG used a block-based DCT which is very similar to a Fourier transform.
  - ygra 10y ago
    That was not my intent. I just noted that my approach came from visually inspecting spectrograms and noting that they might be compressed quite well by image compression algorithms. Only to notice in the end that lossy image compression completely butchers the signal with noise if doing that. Had I just used the original waveform as raw image data that wouldn't have been as much of a problem. Compression would still have changed the samples, but overall probably not as badly. Heck, I've been 17 when I did that. I certainly didn't really know much about the theory behind or why it almost certainly would fail. But I guess by now I wouldn't even attempt such craziness, so in a way it's probably a good thing to not know things too well, sometimes.
    - ska 10y ago
      The problem is that we are much more sensitive to audio distortion than visual, particularly dynamic (e.g.video). Pretty much all lossy audio compression does a bad job without a psycho-acoustic model, whereas image compression works pretty well on simple metrics. It doesn't help that psycho-visual modelling is less well understood (than audio).
garaetjjte 10y ago
Lots of noise. https://milek7.pl/.stuff/jpegcompress.flac https://milek7.pl/.stuff/jpegcompress.flac (24bit PCM as 10000px wide RGB image compressed using JPEG quality 85, ~50% of uncompressed size)
- kardos 10y ago
  Sounds like a weak radio signal! I wonder if we could pass it through a deringing filter to remove some of that noise, eg, like these guys have done: http://www.cs.rpi.edu/~oztanb/Papers/spie07paper.pdf http://www.cs.rpi.edu/~oztanb/Papers/spie07paper.pdf
- lcrs 10y ago
  Similar results - converting to a square image at 8bit then lowering JPEG quality just gets noisy in an uninteresting way: sox mo.wav -e unsigned -b 8 -c 1 -r 48k mo.raw bytes=`stat -f %z mo.raw` width=`echo sqrt$$bytes$ | bc` square_bytes=`echo $width \* $width | bc` dd if=mo.raw of=mo_square.raw bs=$square_bytes count=1 gm convert -depth 8 -size ${width}x${width} gray:mo_square.raw -quality 50 mo_square.jpg gm convert mo_square.jpg gray:mo_square_jpg.raw sox -e unsigned -b 8 -c 1 -r 48k -t raw mo_square_jpg.raw mo_jpg.wav
Artlav 10y ago
Not that bad, actually. Just tried converting the sound to 8 bit, compressing it as a monochrome image, then back. You can't really hear much difference at higher quality settings, and as you turn it down it sound like there is more and more digital-ish noise. I had to turn the compression way up before the voice became almost illegible, but you can still hear that it's someone talking.
- plussed_reader 10y ago
  I'd be curious what you consider garbage audio, as compared to 'not that bad.'
  - Artlav 10y ago
    It's garbage when you need to switch to CW to continue the conversation. :) Seriously, when it's unpleasant to listen to. As i said in the other reply, with music the quality is pretty bad on almost any setting, voice just happen to be an exception.
Artlav 10y ago
Tried again with music... Even of high quality, there is a background sheesh kind of noise, similar to what you get if you play a 3D printed record. On higher compression it's turning into something rather crumbly, with a recognizable tune but horrible sound quality. So, yeah. Not quite usable.
- BurningFrog 10y ago
  > similar to what you get if you play a 3D printed record I guess I'm behind the times, not relating to that reference...
JoshCole 10y ago
I'm more curious about a different idea. If you hook up electrodes to a tongue and then start activating its sensors according to the amount of light being received, people quickly develop the ability to see via their tongue. So what happens when you encode visual information into sound in such a way as to have a sound-to-environment signal and you just keep that active and running real time. We already know that this mechanism exists in nature and can even be developed in humans, since echo-location is a thing. Edit: It occurs to me that actually doing sensory experiments like this might really mess with someones head... maybe not a good idea to play around with it casually. Edit 2: If we were going to swap out one of our senses for another, which should we get rid of and which should we gain?
- dr_zoidberg 10y ago
  Actually there are around (and have been for a few years aready) some glasses that scan the image you'd see and translate that into a sound that you need a bit training to understand as an image[0]. Some people have gotten used to them, while others have gone the way of the bat and learned to echolocate[1]. [0] http://gizmodo.com/these-synthesia-glasses-help-blind-people-see-via-son-745027691 http://gizmodo.com/these-synthesia-glasses-help-blind-people... [1] https://en.wikipedia.org/wiki/Human_echolocation https://en.wikipedia.org/wiki/Human_echolocation
- Houshalter 10y ago
  It's been done and it works. I believe they just map the image to a spectrogram and convert it to audio: https://www.seeingwithsound.com/ https://www.seeingwithsound.com/
- Srikanth 10y ago
  Your comment reminded me the TED talk "How a blind astronomer found a way to hear the stars"[1]. [1] https://www.ted.com/talks/wanda_diaz_merced_how_a_blind_astronomer_found_a_way_to_hear_the_stars?language=en https://www.ted.com/talks/wanda_diaz_merced_how_a_blind_astr...
- bshanks 10y ago
  Here's an Android app that converts images from the phone's camera to sounds as a tool for the blind: https://play.google.com/store/apps/details?id=vOICe.vOICe&hl=en https://play.google.com/store/apps/details?id=vOICe.vOICe&hl...

The obvious question here is: what does JPEG encoded music sound like?

by kardos 10y ago

The obvious question here is: what does JPEG encoded music sound like?

agumonkey 10y ago

Are you trying to fund a gif music player ?

ygra 10y ago

Horrible. I actually tried applying image compression to sounds over a decade ago (was still in school, did that for Jugend forscht). I used a Fourier transform to generate image sections of a specific window of samples and used that. In hindsight, perhaps just treating the waveform as image data may have been better. The idea behind what I did was to see whether a spectrogram can be compressed well, considering that there's lots of repetition and blank space usually. In any case, JPEG murders pretty much everything in thag representation.

retroencabulato 10y ago
It kind of reads like you have discredited frequency domain compression algorithms. FYI JPEG used a block-based DCT which is very similar to a Fourier transform.
- ygra 10y ago
  That was not my intent. I just noted that my approach came from visually inspecting spectrograms and noting that they might be compressed quite well by image compression algorithms. Only to notice in the end that lossy image compression completely butchers the signal with noise if doing that. Had I just used the original waveform as raw image data that wouldn't have been as much of a problem. Compression would still have changed the samples, but overall probably not as badly. Heck, I've been 17 when I did that. I certainly didn't really know much about the theory behind or why it almost certainly would fail. But I guess by now I wouldn't even attempt such craziness, so in a way it's probably a good thing to not know things too well, sometimes.
  - ska 10y ago
    The problem is that we are much more sensitive to audio distortion than visual, particularly dynamic (e.g.video). Pretty much all lossy audio compression does a bad job without a psycho-acoustic model, whereas image compression works pretty well on simple metrics. It doesn't help that psycho-visual modelling is less well understood (than audio).

garaetjjte 10y ago

Lots of noise. https://milek7.pl/.stuff/jpegcompress.flac https://milek7.pl/.stuff/jpegcompress.flac (24bit PCM as 10000px wide RGB image compressed using JPEG quality 85, ~50% of uncompressed size)

kardos 10y ago
Sounds like a weak radio signal! I wonder if we could pass it through a deringing filter to remove some of that noise, eg, like these guys have done: http://www.cs.rpi.edu/~oztanb/Papers/spie07paper.pdf http://www.cs.rpi.edu/~oztanb/Papers/spie07paper.pdf
lcrs 10y ago
Similar results - converting to a square image at 8bit then lowering JPEG quality just gets noisy in an uninteresting way: sox mo.wav -e unsigned -b 8 -c 1 -r 48k mo.raw bytes=`stat -f %z mo.raw` width=`echo sqrt$$bytes$ | bc` square_bytes=`echo $width \* $width | bc` dd if=mo.raw of=mo_square.raw bs=$square_bytes count=1 gm convert -depth 8 -size ${width}x${width} gray:mo_square.raw -quality 50 mo_square.jpg gm convert mo_square.jpg gray:mo_square_jpg.raw sox -e unsigned -b 8 -c 1 -r 48k -t raw mo_square_jpg.raw mo_jpg.wav

Artlav 10y ago

Not that bad, actually. Just tried converting the sound to 8 bit, compressing it as a monochrome image, then back. You can't really hear much difference at higher quality settings, and as you turn it down it sound like there is more and more digital-ish noise. I had to turn the compression way up before the voice became almost illegible, but you can still hear that it's someone talking.

plussed_reader 10y ago
I'd be curious what you consider garbage audio, as compared to 'not that bad.'
- Artlav 10y ago
  It's garbage when you need to switch to CW to continue the conversation. :) Seriously, when it's unpleasant to listen to. As i said in the other reply, with music the quality is pretty bad on almost any setting, voice just happen to be an exception.

Artlav 10y ago

Tried again with music... Even of high quality, there is a background sheesh kind of noise, similar to what you get if you play a 3D printed record. On higher compression it's turning into something rather crumbly, with a recognizable tune but horrible sound quality. So, yeah. Not quite usable.

BurningFrog 10y ago
> similar to what you get if you play a 3D printed record I guess I'm behind the times, not relating to that reference...

JoshCole 10y ago

I'm more curious about a different idea. If you hook up electrodes to a tongue and then start activating its sensors according to the amount of light being received, people quickly develop the ability to see via their tongue. So what happens when you encode visual information into sound in such a way as to have a sound-to-environment signal and you just keep that active and running real time. We already know that this mechanism exists in nature and can even be developed in humans, since echo-location is a thing. Edit: It occurs to me that actually doing sensory experiments like this might really mess with someones head... maybe not a good idea to play around with it casually. Edit 2: If we were going to swap out one of our senses for another, which should we get rid of and which should we gain?

dr_zoidberg 10y ago
Actually there are around (and have been for a few years aready) some glasses that scan the image you'd see and translate that into a sound that you need a bit training to understand as an image[0]. Some people have gotten used to them, while others have gone the way of the bat and learned to echolocate[1]. [0] http://gizmodo.com/these-synthesia-glasses-help-blind-people-see-via-son-745027691 http://gizmodo.com/these-synthesia-glasses-help-blind-people... [1] https://en.wikipedia.org/wiki/Human_echolocation https://en.wikipedia.org/wiki/Human_echolocation
Houshalter 10y ago
It's been done and it works. I believe they just map the image to a spectrogram and convert it to audio: https://www.seeingwithsound.com/ https://www.seeingwithsound.com/
Srikanth 10y ago
Your comment reminded me the TED talk "How a blind astronomer found a way to hear the stars"[1]. [1] https://www.ted.com/talks/wanda_diaz_merced_how_a_blind_astronomer_found_a_way_to_hear_the_stars?language=en https://www.ted.com/talks/wanda_diaz_merced_how_a_blind_astr...
bshanks 10y ago
Here's an Android app that converts images from the phone's camera to sounds as a tool for the blind: https://play.google.com/store/apps/details?id=vOICe.vOICe&hl=en https://play.google.com/store/apps/details?id=vOICe.vOICe&hl...