5 ms·

Show HN: YouTube Musical Spectrum – visualizer for YouTube with musical notes

72 points by mfcc64 5y ago

nonameiguess 5y ago
Cool, man! I've wanted a standalone player to do this ever since WinAmp used to support this 20 years ago, but seemingly the capability was lost to time when everyone decided they'd rather watch recorded video. The composer Stephen Malinowski's YouTube channel is one of my favorites because all of the videos are visualizations of music he's generated with custom midi-reading software he made, i.e.: https://www.youtube.com/watch?v=ffYKCNY6kUk https://www.youtube.com/watch?v=ffYKCNY6kUk. He first wrote this software in 1985 a decade after getting the idea from listening to Bach on LSD. It's just too bad this is a browser extension because I really don't want more of those. I get that the economics probably don't make sense to do any more, given the effort to make a standalone desktop app would be a lot and a website people could upload tracks to in order to get a visualization would force you to pay hosting costs.
- mfcc64 5y ago
  Actually, you can use mpv player and install visualizer.lua on mpv-scripts repository. The engine of YouTube Musical spectrum (showcqt-js) is a javascript port of ffmpeg showcqt filter.
  - mxmilkiib 5y ago
    For any Arch based users, I made an AUR entry for this; https://aur.archlinux.org/packages/mpv-visualizer/ https://aur.archlinux.org/packages/mpv-visualizer/
- pubutil 5y ago
  That Stephen Malinowski link was great. I don’t know that song by name, but I could unmistakably tell what song it was just by looking at the visualization (even though I left the video muted)!
- shannifin 5y ago
  Also a fan of Malinowski's animated music videos. My immediate thought was that it would be cool to see this extension sort of mimic his piano roll animations by having it appear vertically on the left. Less pixel space though, so I don't know... Would also be cool if the dots on the pitch line that represent accidentals were perhaps a slightly darker shade so it'd be easier to see the typical piano pattern in them. (Like ⦾⦿⦾⦿⦾⦾⦿⦾⦿⦾⦿⦾ kinda thing.) Anyway, to original poster: very cool extension, thanks for sharing! :)
- mfcc64 5y ago
  For a website, probably you want to try this example at https://mfcc64.github.io/html5-showcqtbar/ https://mfcc64.github.io/html5-showcqtbar/
ElijahLynn 5y ago
I left this comment with a video screencapture of the extension in action > https://github.com/mfcc64/youtube-musical-spectrum/issues/2 https://github.com/mfcc64/youtube-musical-spectrum/issues/2 > As someone who has been learning how to sing for the past year, I have tried many spectrum tools and have come across one that is really useful for singing with my own voice but isn't that great for multi-frequency visualization and therefore isn't that good for learning other songs but THIS extension really fills a void for learning other songs!! I like how I can tell the voice from the instruments in this song by the vibrato visualization: https://www.youtube.com/watch?v=450p7goxZqg https://www.youtube.com/watch?v=450p7goxZqg
ElijahLynn 5y ago
I knew there was a payoff from browsing HN today!!! This was totally worth it!!!
- mfcc64 5y ago
  Thank's for the support.
tzm 5y ago
When I read notes, I express through interpretation as a spectrum of emotion. So this spectral visualization is pretty much how I visually interpret musical notes, especially when playing by ear.
- mfcc64 5y ago
  Can you detect notes with missing fundamental?
marcodiego 5y ago
I'd like this visualization on Audacity.
- mfcc64 5y ago
  I'll be glad if someone ports it to Audacity.
nyanpasu64 5y ago
Why does the extension on Firefox block AV1 by default? It seems unrelated to the purpose of the extension. Also have you considered coloring the spectrum according to the musical note? So every C is red, every E is green, every G is cyan, etc.
- mfcc64 5y ago
  >Why does the extension on Firefox block AV1 by default? It seems unrelated to the purpose of the extension. Because AV1's decoding is slow and on some low-end hardware it makes the animation stutter. > Also have you considered coloring the spectrum according to the musical note? So every C is red, every E is green, every G is cyan, etc. The color is used for left-right channel separation.
  - xnzakg 5y ago
    cnlohr's colorchord does something similar to what you mentioned. https://github.com/cnlohr/colorchord https://github.com/cnlohr/colorchord
lostgame 5y ago
This is a very unique, fascinating; useful - and - above all - interesting utility. Thanks for doing the work on this. :).
- mfcc64 5y ago
  Thank's :).
iamcreasy 5y ago
This is pretty cool. Does anyone know any stand alone feature rich audio visualizer for Windows? I am surprised that Spotify never included any visualizer for the desktop client. I miss the old Winamp days.
- knaik94 5y ago
  There is a standalone visualizer that is my go to, projectm. It's a spiritual successor to the visualizer from winamp days and is compatible with the old milkdrop presets. I haven't been able to find a proper way to set up mesh size on windows but if you're running it on linux or android, those options are available too. https://github.com/projectM-visualizer/projectm https://github.com/projectM-visualizer/projectm
  - iamcreasy 5y ago
    Just tried it. Like you said, it mimics Winamp Milkdrop but it has not aged well.
- ugotsta 5y ago
  There are apps like Plane9 (very similar to Milkdrop from Winamp) and the commercial Luminant Music. I'd say FL Studio's integrated plugin ZGameEditor Visualizer is probably the top alternative considering the sheer number of presets and templates as well as configurability. They're all listed at AlternativeTo. https://alternativeto.net/software/projectm/ https://alternativeto.net/software/projectm/
h0l0cube 5y ago
Which transform are you using?
- mfcc64 5y ago
  The visualization engine is showcqt-js (https://github.com/mfcc64/showcqt-js https://github.com/mfcc64/showcqt-js). It is a javascript port of ffmpeg showcqt (Show Constant Q Transform). Probably, more correct name is Variable Q Transform.
  - h0l0cube 5y ago
    I've implemented the Constant Q transform before, and it does work well for western music. Are you using an autocorrelation to boost the fundamental? It can work well, particularly when there's just one note being played
    - mfcc64 5y ago
      No, I don't use autocorrelation. Of course, it isn't a pitch detection software.
      - h0l0cube 5y ago
        It's a cool project. I'm just curious, because I haven't bumped into anyone else who's dabbled with the CQT before. So in what way is it variable Q? I thought the innovation of CQT over a regular FFT is that the bins represent each note step with the same 'Q', making notes more distinguishable? (whereas a FFT has more 'Q' in the high frequencies) I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument.
        mfcc64 5y ago
        linear STFT's window length in time domain = k (constant) CQT's window length in time domain = k/f showcqt-js window length in time domain = a * b / (a / c + b * f / (1 - c)) + a * b / (b * f / c + a / (1 - c)) whera a = 384, b = 0.33, c = 0.17 I also apply asymmetric window to reduce latency before doing VQT. > I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument. Research needed. Doing autocorrelation on simple single instrument monophonic audio is probably easy. But doing it on complex multi instrument audio isn't easy. Probably, it needs some sort of machine learning.
in3d 5y ago
Awesome to see projects like this! I have music-to-visual synesthesia when smoking pot and making something that shows how music looks to me is a project that I’d like get to sometime also. My problem is that I don’t really care to get high often enough to get it done. It would look quite different than yours or Stephen Malinowski’s visualizations.

72 points by mfcc64 5y ago

Cool, man! I've wanted a standalone player to do this ever since WinAmp used to support this 20 years ago, but seemingly the capability was lost to time when everyone decided they'd rather watch recorded video. The composer Stephen Malinowski's YouTube channel is one of my favorites because all of the videos are visualizations of music he's generated with custom midi-reading software he made, i.e.: https://www.youtube.com/watch?v=ffYKCNY6kUk https://www.youtube.com/watch?v=ffYKCNY6kUk. He first wrote this software in 1985 a decade after getting the idea from listening to Bach on LSD. It's just too bad this is a browser extension because I really don't want more of those. I get that the economics probably don't make sense to do any more, given the effort to make a standalone desktop app would be a lot and a website people could upload tracks to in order to get a visualization would force you to pay hosting costs.

mfcc64 5y ago
Actually, you can use mpv player and install visualizer.lua on mpv-scripts repository. The engine of YouTube Musical spectrum (showcqt-js) is a javascript port of ffmpeg showcqt filter.
- mxmilkiib 5y ago
  For any Arch based users, I made an AUR entry for this; https://aur.archlinux.org/packages/mpv-visualizer/ https://aur.archlinux.org/packages/mpv-visualizer/
pubutil 5y ago
That Stephen Malinowski link was great. I don’t know that song by name, but I could unmistakably tell what song it was just by looking at the visualization (even though I left the video muted)!
shannifin 5y ago
Also a fan of Malinowski's animated music videos. My immediate thought was that it would be cool to see this extension sort of mimic his piano roll animations by having it appear vertically on the left. Less pixel space though, so I don't know... Would also be cool if the dots on the pitch line that represent accidentals were perhaps a slightly darker shade so it'd be easier to see the typical piano pattern in them. (Like ⦾⦿⦾⦿⦾⦾⦿⦾⦿⦾⦿⦾ kinda thing.) Anyway, to original poster: very cool extension, thanks for sharing! :)
mfcc64 5y ago
For a website, probably you want to try this example at https://mfcc64.github.io/html5-showcqtbar/ https://mfcc64.github.io/html5-showcqtbar/

ElijahLynn 5y ago

I left this comment with a video screencapture of the extension in action > https://github.com/mfcc64/youtube-musical-spectrum/issues/2 https://github.com/mfcc64/youtube-musical-spectrum/issues/2 > As someone who has been learning how to sing for the past year, I have tried many spectrum tools and have come across one that is really useful for singing with my own voice but isn't that great for multi-frequency visualization and therefore isn't that good for learning other songs but THIS extension really fills a void for learning other songs!! I like how I can tell the voice from the instruments in this song by the vibrato visualization: https://www.youtube.com/watch?v=450p7goxZqg https://www.youtube.com/watch?v=450p7goxZqg

ElijahLynn 5y ago

I knew there was a payoff from browsing HN today!!! This was totally worth it!!!

mfcc64 5y ago
Thank's for the support.

tzm 5y ago

When I read notes, I express through interpretation as a spectrum of emotion. So this spectral visualization is pretty much how I visually interpret musical notes, especially when playing by ear.

mfcc64 5y ago
Can you detect notes with missing fundamental?

marcodiego 5y ago

I'd like this visualization on Audacity.

mfcc64 5y ago
I'll be glad if someone ports it to Audacity.

nyanpasu64 5y ago

Why does the extension on Firefox block AV1 by default? It seems unrelated to the purpose of the extension. Also have you considered coloring the spectrum according to the musical note? So every C is red, every E is green, every G is cyan, etc.

mfcc64 5y ago
>Why does the extension on Firefox block AV1 by default? It seems unrelated to the purpose of the extension. Because AV1's decoding is slow and on some low-end hardware it makes the animation stutter. > Also have you considered coloring the spectrum according to the musical note? So every C is red, every E is green, every G is cyan, etc. The color is used for left-right channel separation.
- xnzakg 5y ago
  cnlohr's colorchord does something similar to what you mentioned. https://github.com/cnlohr/colorchord https://github.com/cnlohr/colorchord

lostgame 5y ago

This is a very unique, fascinating; useful - and - above all - interesting utility. Thanks for doing the work on this. :).

mfcc64 5y ago
Thank's :).

iamcreasy 5y ago

This is pretty cool. Does anyone know any stand alone feature rich audio visualizer for Windows? I am surprised that Spotify never included any visualizer for the desktop client. I miss the old Winamp days.

knaik94 5y ago
There is a standalone visualizer that is my go to, projectm. It's a spiritual successor to the visualizer from winamp days and is compatible with the old milkdrop presets. I haven't been able to find a proper way to set up mesh size on windows but if you're running it on linux or android, those options are available too. https://github.com/projectM-visualizer/projectm https://github.com/projectM-visualizer/projectm
- iamcreasy 5y ago
  Just tried it. Like you said, it mimics Winamp Milkdrop but it has not aged well.
ugotsta 5y ago
There are apps like Plane9 (very similar to Milkdrop from Winamp) and the commercial Luminant Music. I'd say FL Studio's integrated plugin ZGameEditor Visualizer is probably the top alternative considering the sheer number of presets and templates as well as configurability. They're all listed at AlternativeTo. https://alternativeto.net/software/projectm/ https://alternativeto.net/software/projectm/

h0l0cube 5y ago

Which transform are you using?

mfcc64 5y ago
The visualization engine is showcqt-js (https://github.com/mfcc64/showcqt-js https://github.com/mfcc64/showcqt-js). It is a javascript port of ffmpeg showcqt (Show Constant Q Transform). Probably, more correct name is Variable Q Transform.
- h0l0cube 5y ago
  I've implemented the Constant Q transform before, and it does work well for western music. Are you using an autocorrelation to boost the fundamental? It can work well, particularly when there's just one note being played
  - mfcc64 5y ago
    No, I don't use autocorrelation. Of course, it isn't a pitch detection software.
    - h0l0cube 5y ago
      It's a cool project. I'm just curious, because I haven't bumped into anyone else who's dabbled with the CQT before. So in what way is it variable Q? I thought the innovation of CQT over a regular FFT is that the bins represent each note step with the same 'Q', making notes more distinguishable? (whereas a FFT has more 'Q' in the high frequencies) I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument.
      - mfcc64 5y ago
        linear STFT's window length in time domain = k (constant) CQT's window length in time domain = k/f showcqt-js window length in time domain = a * b / (a / c + b * f / (1 - c)) + a * b / (b * f / c + a / (1 - c)) whera a = 384, b = 0.33, c = 0.17 I also apply asymmetric window to reduce latency before doing VQT. > I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument. Research needed. Doing autocorrelation on simple single instrument monophonic audio is probably easy. But doing it on complex multi instrument audio isn't easy. Probably, it needs some sort of machine learning.

in3d 5y ago

Awesome to see projects like this! I have music-to-visual synesthesia when smoking pot and making something that shows how music looks to me is a project that I’d like get to sometime also. My problem is that I don’t really care to get high often enough to get it done. It would look quite different than yours or Stephen Malinowski’s visualizations.