4 ms·

Hi Quinn! Coincidentally, I've been working on a sound visualizer which accurately translates sound to visuals, while preserving all the information and, what's

Hi Quinn! Coincidentally, I've been working on a sound visualizer which accurately translates sound to visuals, while preserving all the information and, what's way more important, it transfers the "sound symmetry" which we often refer to as "musical harmony". The visuals can be translated back to sound. As someone with nearly perfect hearing and musical education background, I find the visuals oddly accurate when I also listen to the corresponding music. It can trivially distinguish human vowels, as those produce different recognizable shapes. I've put together a few sample images and a live demo: check out "github soundshader" (the project that talks about ACFs). I'm really curious what you think about this.

codetrotter 6y ago
Clickable link https://github.com/soundshader/soundshader.github.io https://github.com/soundshader/soundshader.github.io
yorwba 6y ago
Is there a reason you don't just link directly to https://soundshader.github.io/ https://soundshader.github.io/ or https://github.com/soundshader/soundshader.github.io https://github.com/soundshader/soundshader.github.io ?
- gbh444g 6y ago
  I tried doing just that not so long ago: HN promptly shadowbanned my account and my IP address.
  - yorwba 6y ago
    If you send an email to hn@ycombinator.com the mods fix false positives like that pretty quickly.
    - whatatita 6y ago
      Is this documented somewhere?
      - yorwba 6y ago
        The contact email is in the footer. The rest of my comment is just based on my experience mailing the mods for various reasons.
nefitty 6y ago
It would be awesome if there was a pre-populated example on the GitHub pages site. I’m on an iPhone Chrome right now and have no mp3s to try and the mic button didn’t do anything.
- gbh444g 6y ago
  I've been thinking to do this, actually. However the mobile Chrome and Firefox don't seem to work yet, at least on Android. For example, mobile Chrome renders only a 1-pixel wide stripe moving upwards. It correctly changes color, so all the machinery around GLSL must be working, but I have no idea why the rest of the image is pitch black.
qkeast 6y ago
Fascinating stuff! I’m going to dig into it more soon. Thanks for sharing! I’ll be sure to follow up.
anonymfus 6y ago
Every mp3 and an attempt to use mic results in "AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported" error message to appear in Firefox 82.0.2 on Windows 10.
- gbh444g 6y ago
  Try adding `?sr=44.1` in the URL query to set custom AudioContext sample rate matching the mp3 file. Usually it's 44.1 kHz, so it's the default value. On desktop Ubuntu, both Chrome and Firefox can seamlessly rescale the sample rates, but I haven't tested the app anywhere else.
sergeykish 6y ago
It definitely reflects rhythm. I've tried i-e transitions, can catch just a slight change of the angle, should be different formants [1]. Spectrogram looks like a closer match, online demo with mic and some samples [2]. I can't imagine person reading it, it is like watching Matrix. [1] https://en.wikipedia.org/wiki/Formant https://en.wikipedia.org/wiki/Formant [2] https://musiclab.chromeexperiments.com/Spectrogram/ https://musiclab.chromeexperiments.com/Spectrogram/
fouc 6y ago
Hmm, I was hoping it would help with pronunciation, visualizing the difference between saying "da" and "ba" etc but it doesn't seem to be sufficiently fine-grained for that

codetrotter 6y ago

Clickable link https://github.com/soundshader/soundshader.github.io https://github.com/soundshader/soundshader.github.io

yorwba 6y ago

Is there a reason you don't just link directly to https://soundshader.github.io/ https://soundshader.github.io/ or https://github.com/soundshader/soundshader.github.io https://github.com/soundshader/soundshader.github.io ?

gbh444g 6y ago
I tried doing just that not so long ago: HN promptly shadowbanned my account and my IP address.
- yorwba 6y ago
  If you send an email to hn@ycombinator.com the mods fix false positives like that pretty quickly.
  - whatatita 6y ago
    Is this documented somewhere?
    - yorwba 6y ago
      The contact email is in the footer. The rest of my comment is just based on my experience mailing the mods for various reasons.

nefitty 6y ago

It would be awesome if there was a pre-populated example on the GitHub pages site. I’m on an iPhone Chrome right now and have no mp3s to try and the mic button didn’t do anything.

gbh444g 6y ago
I've been thinking to do this, actually. However the mobile Chrome and Firefox don't seem to work yet, at least on Android. For example, mobile Chrome renders only a 1-pixel wide stripe moving upwards. It correctly changes color, so all the machinery around GLSL must be working, but I have no idea why the rest of the image is pitch black.

qkeast 6y ago

Fascinating stuff! I’m going to dig into it more soon. Thanks for sharing! I’ll be sure to follow up.

anonymfus 6y ago

Every mp3 and an attempt to use mic results in "AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported" error message to appear in Firefox 82.0.2 on Windows 10.

gbh444g 6y ago
Try adding `?sr=44.1` in the URL query to set custom AudioContext sample rate matching the mp3 file. Usually it's 44.1 kHz, so it's the default value. On desktop Ubuntu, both Chrome and Firefox can seamlessly rescale the sample rates, but I haven't tested the app anywhere else.

sergeykish 6y ago

It definitely reflects rhythm. I've tried i-e transitions, can catch just a slight change of the angle, should be different formants [1]. Spectrogram looks like a closer match, online demo with mic and some samples [2]. I can't imagine person reading it, it is like watching Matrix. [1] https://en.wikipedia.org/wiki/Formant https://en.wikipedia.org/wiki/Formant [2] https://musiclab.chromeexperiments.com/Spectrogram/ https://musiclab.chromeexperiments.com/Spectrogram/

fouc 6y ago

Hmm, I was hoping it would help with pronunciation, visualizing the difference between saying "da" and "ba" etc but it doesn't seem to be sufficiently fine-grained for that