21 ms·
Show HN: I've built a spectrogram analyzer web app
- ssgh 3y agoAuthor here. This is a basic spectrogram visualizer that's mobile friendly. It allows to select regions on the spectrogram and play them separately. There is no grand plan behind this web app: it's just a handy basic tool to capture sounds on your phone and see what they look like.
- montag 3y agoLooks like it says “mime type is not supported” on mobile Safari.
- grugagag 3y agoI get the same error on Iphone/Safari
- ssgh 3y agoIt uses "audio/webm;codecs=opus" to record mic. Now it's possible to change it in the config menu in the top right. Safari probably needs audio/mp3. Edit: also consider "audio/foo;codecs=pcm" where "foo" is something compatible with Safari.
- geraldhh 3y agough could you maybe add some code to detect apple-platforms and set this accordingly (like batteries included)? edit: tyvm, nice idea! would very much like to try it
- _emacsomancer_ 3y agoThis is a problem of iOS not supporting modern efficient codecs.
- echelon 3y agoThis is incredible! One of the best spectral tools I've seen. Can we hire you to help us improve the (broken) spectral visualizations on our app? Example: https://fakeyou.com/tts/result/TR:9jy3vew9w0s3ew4keay9m330rdycs https://fakeyou.com/tts/result/TR:9jy3vew9w0s3ew4keay9m330rd... I would so love to hire you to help us. This is freaking cool. Even if you're not interested, mad props. I really love this.
- KennyBlanken 3y agoNot OP, but...why on earth does having your site open in firefox nearly set my computer on fire?
- timlod 3y agoIn case OP doesn't respond, I could probably help you with this - feel free to send an email!
- ssgh 3y agoYour spectrogram looks elongated horizontally because the FFT window size is too large. I use window size 1024 with sample rate 48000 Hz, so one window covers 1024/48000=0.02 sec. This window size looks optimal in most cases: if you change it in my web app, you'll see that all other window sizes get the spectrogram blurry in different ways, but at 1024 it gets into focus. Of course, don't forget the window function (Hann, or raised cosine), but it looks like you've got that covered because your spectrogram looks smooth. The color palette looks good in your case. FWIW, my color function is like this: pow(fft_amp, 1.5) * rgb(9, 3, 1). The pow() part brightens the low/quiet amplitudes, and the (9,3,1) multiplier displays 10x wider amp range by mapping it to a visually long black->orange->yellow->white range of colors. Note, that I don't do log10 mapping of the amplitudes.
- wpietri 3y agoVery neat! May I suggest adding a button to switch to log scale for frequency? I love the ability to select and play back just a particular set of frequencies. But voice uses only about ~15% of the screen height [1], so it's hard to play with. [1] https://en.wikipedia.org/wiki/Voice_frequency https://en.wikipedia.org/wiki/Voice_frequency
- ssgh 3y agoYou can select an area and zoom into it. Another option is to change sample rate in the config in the top right.
- wpietri 3y agoZooming is not really a way to get what I'm after, because I was trying to hear particular bands one after the other. E.g., trying to listen to one octave after the next. And since the octave relationship isn't linear, I'm thinking a non-linear scale would better match what I was trying to do.
- rickcarlino 3y agoI get the mime error on Firefox mobile. Very interesting idea though, hope I can try it in the future.
- Groxx 3y agoQuite neat, thanks for sharing! I've never been able to play portions like this before, it's interesting. Is there any way to make this display in real time, or is that not (currently?) possible with audio APIs?
- Bewelge 3y agoThe WebAudio API has an anlayser node that can create spectrogramms in real-time. The ones I've created in the past were nowhere near as detailed as this one though. https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
- a-dub 3y agoi really like the filter definition as selection rectangle on time frequency plot ux,
- ssgh 3y agoYeah, that's a bandpass filter, essentially. But I did it the lazy way: audio signal -> FFT -> zero out unwanted frequencies -> inverse FFT.
- lokimedes 3y agoHi, I love it. Perhaps we should chat about making it for radio data as well? We could potentially use it for our radar systems.
- shangers 3y agoCan I ask what kind of use cases would a spectrogram have for radar data? I've been messing around with making my own spectrogram app as well (linux desktop app and not web app though) and would be stoked to know if there's any potentially easy to reach use cases for it
- galcerte 3y agoRadar signals are modulated in very specific ways, which are visible right away even with plenty of noise on a spectrogram. Classifying the modulation of radar signals is something common in military contexts, since it allows you to listen for emissions and be able to tell if it's an enemy or an ally. I bet it has more uses than that, but it's the first one I could think of.
- lokimedes 3y agoWe basically make doppler radars - here frequency-shift is proportional to the speed of the object. Most other radars (pulse-radars) uses the bandwidth of a pulse to gain range resolution (the wider the bandwidth the better the res). Here's a few (very old) plots of how our radars see the world through a spectrogram: https://weibelradars.com/space/space-industry/ https://weibelradars.com/space/space-industry/ What would be cool, would be a browser-based way to do soft analysis of these plots.
- ssgh 3y agoYou can reach me at: ssgh at mm dot st.
- slhck 3y agoNice tool. Some suggestions: - Allow playback via Space button. Show a play marker to let the user know where in the sample they are, even without having selected a part. - Choose a sample that is easier on the ears than high-pitched bird song. I was really shocked when the first loud part came.
- peepwaah 3y agoCan you recommend any good references to begin understanding the Spectrogram ? I work in DL based Noise cancellation - major part of my work involves analyzing spectrograms - I find it very difficult to do my work without having an ability to critically analyze these images. Any help from anybody ?
- nomel 3y agohttps://news.ycombinator.com/item?id=33668004 https://news.ycombinator.com/item?id=33668004
- picture 3y agoWhat do you mean by "understanding the Spectrogram"? The graph itself is straightforward: x axis is time, y axis is frequency. The intensity of each pixel represents the intensity of a certain frequency component and a certain point in time. If you're referring to generating spectrograms with Fourier transforms, you will need some math background to properly do the calculation by hand. It largely just boils down to "find the amount of each frequency over time" Last question, if this is the premise your work, shouldn't you know about it already?
- djsamseng 3y agoThis is a pretty good introductory primer. https://medium.com/analytics-vidhya/understanding-the-mel-spectrogram-fca2afa2ce53 https://medium.com/analytics-vidhya/understanding-the-mel-sp... 1. STFT (get frequencies from the audio signal) 2. Log scale/ decibel scale (since we hear on the log scale) 3. Optionally convert to the Mel scale (filters to how humans hear) Happy to answer any questions
- peepwaah 3y agoThanks for your effort in sharing the link- am kind of comfortable with most of the theoretical aspects of STFT/FFT/MelScale etc.. but when i look at the spectrogram i still feel am missing something. When i look at the spectrogram i want to know how clear is the quality of the speech in the audio - is there background noise - Is there a reverb - Is there a loss anywhere - I have a feeling that these are possible to be learnt from analyzing spectrograms but not sure how to do it. Hence the question.
- bitsinthesky 3y agoThis is a lot of fun
- firefoxd 3y agoSimple, straight to the point, and super useful. One place I used these was on a toy AI assistant. I recorded myself saying a trigger word thousands of times, cut the audio in pieces and converted each to a spectrogram image. I then feed those to a training model to help recognize the trigger word. Before the spectrogram, i was feeding the wav file directly, it was incredibly intensive on my laptop. But the image files were easier to process in real time. This tool can be used for debugging.
- jxmorris12 3y agoHow would this work with AI? Don’t you need to train the model to discriminate between the trigger word and other words? If all that’s seen during training is the trigger word, the model will just learn to say “yes” to everything, if you get what I mean.
- firefoxd 3y agoYes, i have recorded myself talking on the phone for hours as well. I should have clarified that.
- vrglvrglvrgl 3y ago[dead]
- nick_m 3y agoBrilliant work - I "get" how this works, I've just spent about half-an-hour playing with this (Chrome browser on my kitchen ChromeBook), singing into it and letting it "listen" to the ambient background noise here (old cooker clock ticking, fridge compressor rumbling occasionally). Useful, educational, and fun also - thanks for publishing/hosting this so others can enjoy it!
- orbisvicis 3y agoWhen I read about ultrasonic cross-device trackers in advertising [1], I installed "org.woheller69.audio_analyzer_for_android" and "hans.b.skewy1_0" (automatic ultrasonic detection) and started scanning through TV channels after running some test tones. Suffice to say I didn't find any, but the entire process was quite fun. There's also "org.billthefarmer.scope" which is an oscilloscope with a spectrum (not spectrogram). 1. https://arstechnica.com/tech-policy/2015/11/beware-of-ads-that-use-inaudible-sound-to-link-your-phone-tv-tablet-and-pc/ https://arstechnica.com/tech-policy/2015/11/beware-of-ads-th...
- jboy55 3y agoHere is a spectrogram of the track, "Look" from the album, "Songs about my Cats" by Venetian Snares. https://imgur.com/sRe6Ypv https://imgur.com/sRe6Ypv Aphex twin did something similar, but this is more playful in my opinion.
- syx 3y agoThis is very interesting what was the Aphex Twin’s track with this concept?
- twelvechairs 3y agoIts usually called 'formula' or 'equation' - B side of 'windowlicker'. There's a video at the link below https://www.reddit.com/r/Damnthatsinteresting/comments/kvjill/aphex_twins_song_known_as_formula_contains/ https://www.reddit.com/r/Damnthatsinteresting/comments/kvjil...
- swah 3y agohttps://www.youtube.com/watch?v=wSYAZnQmffg https://www.youtube.com/watch?v=wSYAZnQmffg at around 5m30
- jo-m 3y agoI once (badly) did something similar as a student [1]. Unfortunately it's in Matlab so I can not run it any more. [1] https://jo-m.ch/posts/2015/01/hack-the-spectrum-hide-images-in-audio-files/ https://jo-m.ch/posts/2015/01/hack-the-spectrum-hide-images-...
- scotteric 3y agoYou know a personal license is only $149 USD right? You could then run your old code. Toolkits for home are $49.
- michaelmior 3y ago$149 is still a significant amount for many people. Especially for something non-essential.
- vishnuharidas 3y agoWeb apps like this that accesses user's data should provide samples for users to experiment and explore before they have to give access to their actual data.
- ssgh 3y agoVery reasonable ask, sir. In fact, I had added sample.mp3, but forgot to add a button in the UI. Now that's fixed.
- djmips 3y agoVery nice, I tried on my phone and was really enjoyed being able to intuitively select regions to playback. Very fun.
- HarHarVeryFunny 3y agoNice - very fast (using WebGPU?). I like the interesting ability to play a "rectangular" (time + frequency limited) section of the audio.
- ssgh 3y agoI do have a WebGL-based implementation of FFT, but here I used good old JS. When properly written, it gets translated into really fast machine code, which is even faster than WebAssembly (I tried!). WebGL's problem is the high toll on the CPU--GPU bridge. When you need to transfer a block of audio data from CPU to GPU to perform calculations, you wait. When you need to transfer the FFT data back, you wait. These waits quickly outweight everything else. However on wavelet transforms GPU comes first because you can store some pre-computed FFTs on GPU and reuse them in multiple runs.
- nixpulvis 3y agoWould be awesome if it told me what file types it supported and also helped transcode some things like videos. Looks very interesting though.
- vjerancrnjak 3y agoVery nice app. I usually use Audacity to inspect the spectrogram of FLAC files and see if they really are 44100Hz or if someone packaged a constant rate 320kbps mp3 encode into a FLAC file. Now I can just check it in my browser :D
- brunorsini 3y agoIzotope, associated with MIT researchers, makes arguably the best such tool for the pro audio industry. Their RX suite is truly miraculous, allowing audio engineers to visualize frequencies in a similar manner, but also offering brush-like tools to do things such as "deleting a dog bark from a guitar take" fairly easily.
- an-unknown 3y agoSeems like you never saw or used SpectraLayers (commercial tool from Steinberg) or Sonic Visualiser (OSS project). Both have much more advanced visualization capabilities than RX. However, RX definitely has the more advanced "semi-automated" editing / repair features.
- brunorsini 3y agoI've witnessed a large number of studios across the US and Latam using RX on a regular basis — places recording anything from indie stars to Grammy-winning artists.
- k8si 3y agoWhat we really need is PraaS (Praat as a Service). Praat Cloud Edition. Etc.
- ddingus 3y agoI like this! Easy to use, fun, useful. Nice work.