4 ms·

For what it's worth, the presence of seemingly significant signal in the difference between the original and compressed tracks does not necessarily mean that si

For what it's worth, the presence of seemingly significant signal in the difference between the original and compressed tracks does not necessarily mean that significant sonic/perceptual loss has occurred. Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context (e.g. the quietest signals in a loud section). So if you find something beautiful about the ghost tracks (and I think there is something beautiful to find), don't immediately jump to concluding that mp3 is awful for cutting these sounds—they might be hardly perceptible when added to the mix. Of course, at high-compression rates mp3 does begin to significantly degrade fidelity. Edit: all of this is not to put down the project—I still think it's pretty cool as art and as a demonstration of the encoder, I just didn't want people to think that this was some sort of massive failing of mp3.

derf_ 12y ago
> Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context This. The idea is called "masking", and a well-designed encoder explicitly removes the most details from areas where they are masked by other, louder parts of the signal. See the presentation on CELT I gave at linux.conf.au in 2009, and in particular listen to the audio samples that go with slide 47: http://www.celt-codec.org/presentations/ http://www.celt-codec.org/presentations/
- logfromblammo 12y ago
  Look into the sky at night during a new moon, and count the stars. Look again during the day. The stars are still there, but the atmospheric diffusion of blue light from the sun prevents you from discerning the fainter light of a more distant star. Lower yourself to the bottom of a deep well during the day, and look up again. You can see the stars again, because the diffuse blue sunlight is much weaker there. If you were to compress an image of the daytime sky in the same manner that an MP3 compresses audio, you could simply remove all that variation from faint stars and only keep the information from bright sunlight. Follow the ghost discovery technique by taking the difference between the original and the compressed image, and you will see an image resembling the night sky. But most people would not be able to discern much of a difference with the naked eye. For audio, if you have a pianissimo piccolo solo, and someone plays a crash cymbal in the middle, you could remove some of the harmonics from the melody because humans are incapable of hearing them, and you can also remove that portion drowned out by the crash, because humans can't hear quiet sounds through louder noises. Finding the "ghost" track is actually revealing all the sound in a piece of music that no one ever hears anyway, whether the track is compressed or not, simply due to the hardware limitations of the human auditory system.
panzi 12y ago
And then they proceed and stream the recovered signals as mp3 again. Kinda ironic.
pgeorgi 12y ago
From a technical POV I also question that contemporary encoders still optimize for those three tracks (which means any encoder but the ISO reference code and very close derivatives like 8Hz). IIRC LAME which was used for this project, replaced the entire psychoacoustic modelling code (which is possible to do since MP3-the-format only cares about frequency bands and buckets, not about what the encoder does to them), and they likely tuned their new model to larger, more diverse sets of samples and ears.
- keeperofdakeys 12y ago
  http://lame.sourceforge.net/gpsycho.php http://lame.sourceforge.net/gpsycho.php This seems to have some more information.
bdonlan 12y ago
Is it possible that some of the differences shown in this page come from phase changes rather than attenuation? If so, is such a phase change perceptible, even in principle?
- dspig 12y ago
  Yes, the difference signal is present in both the residue and the mp3, not lost just shifted in phase so it doesn't cancel exactly any more. In general the phase shifts are not audible but you can construct some signals (like two impulses very close together) that fool MP3 so the encoded signal has the correct spectrum but the wrong phase to reconstruct the impulses so sounds different.
moultano 12y ago
Even more importantly, there are many signals that sound identical to the human ear, yet are entirely different in waveform. (For instance, two sines shifted 90 degrees.) You can save space by representing all similar sounding waveforms as the same waveform. This might make for a giant delta when you subtract it from the original recording, even though nothing audible was lost.
- schiffern 12y ago
  Good point! I believe the author in this particular case was subtracting in the frequency domain, not in the time domain — subtracting spectrographs instead of waveforms. In that case, a 90° phase-shifted sine wave would indeed cancel out.
quattrofan 12y ago
Ok but isn't it true that sounds we cannot hear interfere with sounds we can i.e. interference?
Zigurd 12y ago
> sounds which may not be perceptible in context There is good reason to think that's different. Audio quality that's measured by subjects' ability to perceive, and report perception of a sound masks the difference between their ear being unable to detect a signal, and hence no signal comes over the auditory nerve and a situation where a soft sound might not be noticed in a loud passage but the ear responds to the signal. Is this significant? One would probably have to do an FMRI study to answer that. In some applications, like understanding the words of someone speaking, by definition it isn't. But music is emotional and there might be aspects of music listening that are outside the ability to consciously report. An obvious case of audition working when we can't consciously report what we hear is when we are sleeping, but experiments can detect the influence of those sounds.

derf_ 12y ago

> Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context This. The idea is called "masking", and a well-designed encoder explicitly removes the most details from areas where they are masked by other, louder parts of the signal. See the presentation on CELT I gave at linux.conf.au in 2009, and in particular listen to the audio samples that go with slide 47: http://www.celt-codec.org/presentations/ http://www.celt-codec.org/presentations/

logfromblammo 12y ago
Look into the sky at night during a new moon, and count the stars. Look again during the day. The stars are still there, but the atmospheric diffusion of blue light from the sun prevents you from discerning the fainter light of a more distant star. Lower yourself to the bottom of a deep well during the day, and look up again. You can see the stars again, because the diffuse blue sunlight is much weaker there. If you were to compress an image of the daytime sky in the same manner that an MP3 compresses audio, you could simply remove all that variation from faint stars and only keep the information from bright sunlight. Follow the ghost discovery technique by taking the difference between the original and the compressed image, and you will see an image resembling the night sky. But most people would not be able to discern much of a difference with the naked eye. For audio, if you have a pianissimo piccolo solo, and someone plays a crash cymbal in the middle, you could remove some of the harmonics from the melody because humans are incapable of hearing them, and you can also remove that portion drowned out by the crash, because humans can't hear quiet sounds through louder noises. Finding the "ghost" track is actually revealing all the sound in a piece of music that no one ever hears anyway, whether the track is compressed or not, simply due to the hardware limitations of the human auditory system.

panzi 12y ago

And then they proceed and stream the recovered signals as mp3 again. Kinda ironic.

pgeorgi 12y ago

From a technical POV I also question that contemporary encoders still optimize for those three tracks (which means any encoder but the ISO reference code and very close derivatives like 8Hz). IIRC LAME which was used for this project, replaced the entire psychoacoustic modelling code (which is possible to do since MP3-the-format only cares about frequency bands and buckets, not about what the encoder does to them), and they likely tuned their new model to larger, more diverse sets of samples and ears.

keeperofdakeys 12y ago
http://lame.sourceforge.net/gpsycho.php http://lame.sourceforge.net/gpsycho.php This seems to have some more information.

bdonlan 12y ago

Is it possible that some of the differences shown in this page come from phase changes rather than attenuation? If so, is such a phase change perceptible, even in principle?

dspig 12y ago
Yes, the difference signal is present in both the residue and the mp3, not lost just shifted in phase so it doesn't cancel exactly any more. In general the phase shifts are not audible but you can construct some signals (like two impulses very close together) that fool MP3 so the encoded signal has the correct spectrum but the wrong phase to reconstruct the impulses so sounds different.

moultano 12y ago

Even more importantly, there are many signals that sound identical to the human ear, yet are entirely different in waveform. (For instance, two sines shifted 90 degrees.) You can save space by representing all similar sounding waveforms as the same waveform. This might make for a giant delta when you subtract it from the original recording, even though nothing audible was lost.

schiffern 12y ago
Good point! I believe the author in this particular case was subtracting in the frequency domain, not in the time domain — subtracting spectrographs instead of waveforms. In that case, a 90° phase-shifted sine wave would indeed cancel out.

quattrofan 12y ago

Ok but isn't it true that sounds we cannot hear interfere with sounds we can i.e. interference?

Zigurd 12y ago

> sounds which may not be perceptible in context There is good reason to think that's different. Audio quality that's measured by subjects' ability to perceive, and report perception of a sound masks the difference between their ear being unable to detect a signal, and hence no signal comes over the auditory nerve and a situation where a soft sound might not be noticed in a loud passage but the ear responds to the signal. Is this significant? One would probably have to do an FMRI study to answer that. In some applications, like understanding the words of someone speaking, by definition it isn't. But music is emotional and there might be aspects of music listening that are outside the ability to consciously report. An obvious case of audition working when we can't consciously report what we hear is when we are sleeping, but experiments can detect the influence of those sounds.