11 ms·
MP3 for Image Compression (2006)
- bwang29 10y agoNow I wonder, what happens if you add sound effect, to change pitch, tone, or widen the sound stage. What would do to the decoded image? I think the 2.00 bits/ pixel result looks quite more "analog" with a film grain effect to me.
- Itsdijital 10y agoHackaday just had this post yesterday https://hackaday.com/2017/04/16/mangling-images-with-audio-effects/ https://hackaday.com/2017/04/16/mangling-images-with-audio-e...
- heywire 10y agoAwesome, thanks!
- simcop2387 10y agoI'm kinda wondering if this post was because of that, because I found the link in question and posted it there. It's kinda neat to see a possible circuitous route that something comes somewhere else with.
- deleted 10y ago[deleted]
- joshumax 10y agoSo you were the person who posted that link ;)
- moutansos 10y agoI did this with Audacity to some bmp images. I got some interesting results. I just read in the bmp files as raw data and manipulated them that way.
- agumonkey 10y agoAudacity allows for raw data input ?
- grenoire 10y agoYes, you have to import raw data.
- digi_owl 10y agoReminds me that one could (can?) pipe random data to /dev/dsp and get it interpreted as sound data. Heck, you could pipe a wav file and actually hear it.
- garaetjjte 10y agoOr pipe output of C oneliner: http://canonical.org/~kragen/bytebeat/ http://canonical.org/~kragen/bytebeat/
- gedy 10y agoI did similar a long time ago with zip compressed TIFs and JPGs. Added reverb and the results were really unusual: https://vimeo.com/105317804 https://vimeo.com/105317804
- peterwwillis 10y agoI really wanted salad fingers to pop out of my screen. Thanks for that.
- heywire 10y agoI love this. I am really hoping others come to the comments to share similar "misuse" of technology stories.
- dhbx9 10y agoAs long as we're dealing with signals, everything is within the realm of possibility.
- amelius 10y agoI'm hoping for someone to drive the timing of a spark-ignition internal-combustion-engine with MP3 data, and report what it sounds like.
- heywire 10y agoNot exactly what you were asking for, but made me think of these: https://www.youtube.com/watch?v=Tr4zb-HHZs4 https://www.youtube.com/watch?v=Tr4zb-HHZs4 and https://www.youtube.com/watch?v=Ee5evlN8Bbs https://www.youtube.com/watch?v=Ee5evlN8Bbs and if you go down the rabbit hole, you end up here: https://www.youtube.com/watch?v=b9UO9tn4MpI https://www.youtube.com/watch?v=b9UO9tn4MpI
- E6300 10y agoIt would be interesting to see if the horizontal artifacts could be avoided by feeding the pixels in a different order to the encoder.
- derefr 10y agoOrdering the pixel data by their locations along a space-filling curve, maybe.
- jay-anderson 10y agoDefinitely: https://en.wikipedia.org/wiki/Hilbert_curve https://en.wikipedia.org/wiki/Hilbert_curve. I'd still expect the final result to look worse than jpeg, but it'd be a much more interesting comparison.
- E6300 10y agoWouldn't the Hilbert curve require a POT bitmap?
- lunixbochs 10y agoYou could try something akin to tessellation, with either more than one hilbert curve, or a truncated curve.
- mxfh 10y agoor split the image in its frequency domains first, and then run the mp3 compression on a spatial curve, but at this point you're already doing a https://en.wikipedia.org/wiki/Discrete_cosine_transform https://en.wikipedia.org/wiki/Discrete_cosine_transform a core part of JPEG compression, which was actually adapted for mp3: https://en.wikipedia.org/wiki/Modified_discrete_cosine_transform https://en.wikipedia.org/wiki/Modified_discrete_cosine_trans...
- leetbulb 10y agoBefore I clicked the OP link, I originally thought it was going to be this type of implementation...cool nonetheless, but would really like to see someone try this and share!
- amelius 10y agoMP3 is inherently a one dimensional codec, whereas JPEG is two dimensional. No wonder it performs much better.
- kardos 10y agoYou could iterate a bit by leveraging 'joint stereo' mode [1], eg, by considering pairs of rows as left/right channels. And perhaps even further with more 'channels'. [1] https://en.wikipedia.org/wiki/Joint_(audio_engineering) https://en.wikipedia.org/wiki/Joint_(audio_engineering)
- deleted 10y ago[deleted]
- spacemanmatt 10y agoI wonder how mp3 would perform against a Hilbert-type path taken through the image.
- lumpypua 10y agoJPEG uses this path for its blocks, rather than anything fancy and Hilbert-ish: http://www.johnloomis.org/ece563/notes/compression/jpeg/tutorial/jpeg_zig.gif http://www.johnloomis.org/ece563/notes/compression/jpeg/tuto... I think this alone would be an improvement for the mp3 encoder.
- seiferteric 10y agoNot only that, but doesn't MP3 remove "sounds changes you can't hear" or whatever, whereas jpg removes "color changes you don't see"? There is a parallel there, but I would imagine they would be quite different since one is based on human auditory response vs visual response.
- rzzzt 10y agoA psychovisual model.
- kardos 10y agoThe obvious question here is: what does JPEG encoded music sound like?
- agumonkey 10y agoAre you trying to fund a gif music player ?
- ygra 10y agoHorrible. I actually tried applying image compression to sounds over a decade ago (was still in school, did that for Jugend forscht). I used a Fourier transform to generate image sections of a specific window of samples and used that. In hindsight, perhaps just treating the waveform as image data may have been better. The idea behind what I did was to see whether a spectrogram can be compressed well, considering that there's lots of repetition and blank space usually. In any case, JPEG murders pretty much everything in thag representation.
- retroencabulato 10y agoIt kind of reads like you have discredited frequency domain compression algorithms. FYI JPEG used a block-based DCT which is very similar to a Fourier transform.
- ygra 10y agoThat was not my intent. I just noted that my approach came from visually inspecting spectrograms and noting that they might be compressed quite well by image compression algorithms. Only to notice in the end that lossy image compression completely butchers the signal with noise if doing that. Had I just used the original waveform as raw image data that wouldn't have been as much of a problem. Compression would still have changed the samples, but overall probably not as badly. Heck, I've been 17 when I did that. I certainly didn't really know much about the theory behind or why it almost certainly would fail. But I guess by now I wouldn't even attempt such craziness, so in a way it's probably a good thing to not know things too well, sometimes.
- deleted 10y ago[deleted]
- xamuel 10y agoThere's a whole subreddit for things like this. https://www.reddit.com/r/glitch_art/ https://www.reddit.com/r/glitch_art/
- robert_foss 10y agoI just posted the article[1] that I think prompted this script to be brought up. What I did instead was to run images through an audio editing tool, which lets you apply echoes or do mindboggling things like change the volume of the image. The script can be found on github[2]. [1] http://memcpy.io/audio-editing-images.html http://memcpy.io/audio-editing-images.html [2] https://github.com/robertfoss/audio_shop/ https://github.com/robertfoss/audio_shop/
- lcrs 10y agoNice script :) I did a similar thing by hand with graphicsmagick and sox a couple years ago - the most fun result was from wahwah, pretty much a wide bandpass filter with centre frequency varying cyclically over time: http://lewisinthelandofmachines.tumblr.com/post/59405537096/the-cover-of-ripley-through-a-wah-wah-audio http://lewisinthelandofmachines.tumblr.com/post/59405537096/...
- robert_foss 10y agoThank you! I haven't even tried the wahwah effect, you've definitely piqued my interest. I had a look through the SoX effects list[1] for the wahwah, but can't seem to find it. The closest thing I found was the 'flanger' effect[2]. [1] http://sox.sourceforge.net/sox.html#EFFECTS http://sox.sourceforge.net/sox.html#EFFECTS [2] https://allg.one/zKMP https://allg.one/zKMP
- lcrs 10y agoShamefully I can't remember which wahwah I used - I was treating the audio with both Audacity and Pro Tools as well as sox though...
- hammock 10y agoNo description of each image. Which one did you change the volume?
- robert_foss 10y ago
- avian 10y agoI did something similar with Ogg Vorbis back in 2006. There are also some results of experiments with the Opus codec posted in the comments. https://www.tablix.org/~avian/blog/archives/2006/01/lossy_compression/ https://www.tablix.org/~avian/blog/archives/2006/01/lossy_co...
- jacquesm 10y agoThe search for an image that encodes the Close Encounters theme is on.
- marvy 10y agoI'm amazed this works at all!
- acjohnson55 10y agoIt kind of has to -- both compression algorithms work by doing a bunch of lossless steps to get to a format that can easily be quantized for lossy compression. The prioritization of how much quantization to apply is picked according to perceptual models of vision/hearing, to devote more bits to quantizing the things we can actually notice. Needless to say, a perceptual model designed for audio is a pretty bad choice for a long string of grayscale pixels, interpreted as sampled audio. It looks like a lot of high-frequency content was discarded, resulting in horizontal blurring.
- marvy 10y agoHas to? Are you saying that the whole thing would not fall apart if, say, someone used color images instead of greyscale?
- acjohnson55 10y agoIf you ordered each subpixel linearly, that would work too. You'd see pretty much the same effect of horizontal distortion. Because the input data is PCM (just magnitude values), same as audio, it "works" just fine. If the input data were some other representation, like text, you'd get gibberish out.
- marvy 10y agoOkay, so it does depend on the representation, but most reasonable representations will work. I guess the big exception would be trying to do this on something that is already compressed, like a gzipped bmp, or something.
- hgears 10y agoI think the question really is, does it do anything for file size? If there were a radical difference in total size, the quality degradation might be an interesting compromise.
- vernie 10y agoThe side-by-side assessment compares JPEG and MP3 at the same bitrate.
- deleted 10y ago[deleted]
- randcraw 10y agoNow for big time shits-and-grins, add a deep learning GAN to generate a more refined signal during the decompression / upsampling stage. That's something your Turbo Pascal code never attempted.
- bluedino 10y agoI remember reading an article where file data (a ZIP file I believe) was converted into a bitmap image, and then it was compressed with PNG for another few % of compression.
- alkoumpa 10y agoWhile it might be possible for special cases [1], I have my concerns, you really can't compress (lossless) beyond the (information theoretic) entropy -- in which case one would try to compress deflate (zip) with deflate (png). [1]: I remember when I was taking an information theory lesson at the university and told the (video/jpeg compression guru) professor that I compressed an AVI file with my dump arithmetic coding implementation, he was shocked, turns out the file had some large crappy header from the editor
- charonn0 10y ago> another few % of compression ZIP consists of sections of both compressed and uncompressed data, so some compressibility is to be expected.
- gwbas1c 10y agoWhere are the images???
- tonymillion 10y agoThey're both based on the DCT. Mp3 (and AAC and Vorbis and more) use a modified DCT which uses block overlapping to mitigate aberrations on the block boundary. Its no surprise it works, however you wouldn't necessarily get "as good" compression as you would from an optimized DCT coder (JPEG etc) based on the data duplication (2x for the overlapping blocks). See https://en.wikipedia.org/wiki/Modified_discrete_cosine_transform https://en.wikipedia.org/wiki/Modified_discrete_cosine_trans... https://en.wikipedia.org/wiki/Discrete_cosine_transform https://en.wikipedia.org/wiki/Discrete_cosine_transform
- sakawa 10y agoCurious enough, years ago I've seen a blog post where the author used PNG lossless compression for FLAC audios. Guess there should be a lot of room for improving both image and audio compression, even because we're still ending up using jpeg and mp3.
- QuercusMax 10y agoIsn't PNG just DEFLATE? You should get basically the same result by just gzipping it.
- MrBra 10y agoWhat about turning the image into a sound, then compressing it with MP3, then turning it back to an image again?
- deleted 10y ago[deleted]
- dheera 10y agoOur ears are more sensitive to amplitude errors than phase errors (as a function of frequency, in frequency space). Our eyes are the opposite.
- peterburkimsher 10y agoIs there a way to use image compression for MP3s? I'd like to store music files in my phone's camera roll, and easily upload them to a website where some Javascript could decode and play them.
- BinaryBullet 10y agoNot really related, but a while back I wrote a script to visualize/hear audio generation loss with different file formats: https://github.com/skratchdot/audio-generation-loss/tree/master/files/loop01 https://github.com/skratchdot/audio-generation-loss/tree/mas... So, mp3s add a bunch of silence to the beginning of the file, and ogg files start to "chirp". I never got around to putting this info in a consumable, easy to understand format though. The videos in these folders just continuously re-encode a source file w/ a given lossy format. See also: https://en.wikipedia.org/wiki/Generation_loss https://en.wikipedia.org/wiki/Generation_loss
- jedimastert 10y agoKinda sounds like "I'm sitting in a room" https://en.m.wikipedia.org/wiki/I_Am_Sitting_in_a_Room https://en.m.wikipedia.org/wiki/I_Am_Sitting_in_a_Room
- BinaryBullet 10y agoThanks for the link! I hadn't heard of this before.
- lifterbro 10y agoInteresting!
- throwaway19373 10y agoWhile amusing, this is bound to quite bad since the function basis is restricted to a single axis.
- ManlyBread 10y agoI wonder if it's possible to create a set of data that produces an actual image when compressed with JPEG and an actual music when compressed with MP3 (for example, a JPEG picture of pianist that also gives a MP3 piano piece).
- pizza 10y agowell.. does this count? https://en.wikipedia.org/wiki/Windowlicker#Hidden_images https://en.wikipedia.org/wiki/Windowlicker#Hidden_images