2 ms·

That’s such a cool project! I really like the imagery you were able to produce. One of the next few things I want to work on is more interesting visualization

That’s such a cool project! I really like the imagery you were able to produce. One of the next few things I want to work on is more interesting visualization options. Just so you know, I got an error when trying to use the mic on mobile Safari when using your live demo. this.audioCtx was null when calling .createMediaStreamSource.

gbh444g 6y ago
Ha, thanks! And those visuals are still quite simplistic as I haven't implemented yet all the ideas to visualize tiny wavelets present in any realistic sound. A few possible next steps I had in mind: (1) Turning those ACFs into actual shapes, not just colored rings assembled together, so say an accord on piano would create a sophisticated 2d or 3d shape with elaborate structure. Perhaps, just mixing the flat ACFs with an exp decay would achieve the result. (2) Visualizing the ACF heights map as a water surface with a raytracer. This sounds nonsense until you see a few real (physical) examples: "cymaglyphs". Those are reflections and refractions on the rippled water surface that's assuming shapes of the so-called Faraday waves (a parametrically driven wave equation, see "Numerical simulation of Faraday waves" by L. Tuckerman on arxiv). However this requires solving a complex diff equation + an extremely high quality raytracer, as otherwise you'd get blurry pixelated images. None of the existing raytracers would work here. As for compatibility with other browsers, it works on desktop Chrome and Firefox, but not on their mobile versions. I haven't figured out yet why. Must be something to do with GPU differences. Safari is a whole another story. The visualizer uses WebAudio + WebGLv2, and Safari isn't known for supporting the bleeding edge standards. While you could do everything without GPU, it would crawl at 0.03 fps and you wouldn't be hear and see sound in real time.

gbh444g 6y ago

Ha, thanks! And those visuals are still quite simplistic as I haven't implemented yet all the ideas to visualize tiny wavelets present in any realistic sound. A few possible next steps I had in mind: (1) Turning those ACFs into actual shapes, not just colored rings assembled together, so say an accord on piano would create a sophisticated 2d or 3d shape with elaborate structure. Perhaps, just mixing the flat ACFs with an exp decay would achieve the result. (2) Visualizing the ACF heights map as a water surface with a raytracer. This sounds nonsense until you see a few real (physical) examples: "cymaglyphs". Those are reflections and refractions on the rippled water surface that's assuming shapes of the so-called Faraday waves (a parametrically driven wave equation, see "Numerical simulation of Faraday waves" by L. Tuckerman on arxiv). However this requires solving a complex diff equation + an extremely high quality raytracer, as otherwise you'd get blurry pixelated images. None of the existing raytracers would work here. As for compatibility with other browsers, it works on desktop Chrome and Firefox, but not on their mobile versions. I haven't figured out yet why. Must be something to do with GPU differences. Safari is a whole another story. The visualizer uses WebAudio + WebGLv2, and Safari isn't known for supporting the bleeding edge standards. While you could do everything without GPU, it would crawl at 0.03 fps and you wouldn't be hear and see sound in real time.