7 ms·

Audio Sparklines

240 points by wlll 5y ago

cormullion 5y ago
These visual displays are very useful. Here's how the Julia REPL shows them: https://github.com/JuliaAudio/SampledSignals.jl#repl-display https://github.com/JuliaAudio/SampledSignals.jl#repl-display
- sudara 5y ago
  This is great, thanks! I looked for existing unicode waveform representations before building, but didn't run into this. The unicode blocks the Julia REPL uses limits the output to magnitude (signed) vs. amplitude (unsigned), but it gets the job done!
ipsum2 5y ago
Braille may also work, and has the benefit of being more compact. If two dots represents a point, an example might be ⣤⣀⣀⣀⡤⠴⠚⠛⠒⠒⠲⣀ (usually terminal fonts hide the white dots). You only get 4 horizontal ticks, but also can do a "interpolation" (like ⠑) that gets you an extra 3.
rsync 5y ago
I'm looking at this final picture in your IDE: https://melatonin.dev/wp-content/uploads/2021/10/image.jpeg https://melatonin.dev/wp-content/uploads/2021/10/image.jpeg ... and I think that is a GUI and not a terminal ... but I'm not sure ? I think the folder icon in the lower left is good evidence that I am looking at a GUI window but would like you to confirm/deny ...
- ninkendo 5y ago
  It’s some flavor of Jetbrains IDE like IntelliJ, maybe CLion?
  - smoofles 5y ago
    Yeah, CLion
- BHSPitMonkey 5y ago
  It's displaying text-based data passed from a debugger.
- picture 5y ago
  What font is that?
  - smoofles 5y ago
    Looks like MonoLisa, at a first glance.
anon9001 5y ago
As a companion to this, it would be cool if there was a way to render any audio file as a spectrogram (https://en.wikipedia.org/wiki/Spectrogram https://en.wikipedia.org/wiki/Spectrogram) directly in the terminal. Since this is HN, does anyone know what are the best ways to draw graphics into modern terminals?
- mprovost 5y ago
  Check out notcurses: https://github.com/dankamongmen/notcurses https://github.com/dankamongmen/notcurses
- joveian 5y ago
  I think Kitty's terminal graphics protocol is the most advanced, although I think there are a couple of others in use as well: https://sw.kovidgoyal.net/kitty/graphics-protocol/ https://sw.kovidgoyal.net/kitty/graphics-protocol/
opminion 5y ago
Sonic Pi, a programmable instrument, has sparklines, which are typically used to represent sequences of midi notes.
zestyping 5y ago
Simple and effective! Very nice.
sandermvanvliet 5y ago
I’ve used a similar approach when visualising telemetry reliability for a whole bunch of real world assets as well as in performance/stress tests. It’s so nice to be able to use this sort of thing in tests when the outcomes don’t represent as text naturally. Makes it super easy to eyeball a failing test and intuitively understand what the problem is.
p0nce 5y ago
Really cool stuff.
jonpalmisc 5y ago
The integration with LLDB here seems really neat and makes me think about what other LLDB scripts could be made to improve debugging of common structures and data. It’s be neat to be able to visualize a tree, etc. in LLDB.
polyterative 5y ago
nice
nyanpasu64 5y ago
On my tweaked Firefox profile, _⎽⎼—⎻⎺‾ renders as 3 different fonts (Courier New, IBM Plex Mono (my monospace font), and Segoe UI Symbol), with different widths and heights, some of which aren't even monospace. On a clean profile, it's only Courier New and Segoe UI Symbol. In both Firefox profiles, ⎺ renders higher than ‾ despite coming before it in order. It might work on certain fonts and terminals. It's not a general solution for blogs IMO. Are we better off with terminals with first-class support for sub-cell graphics rather than text, considering people tend to hack in graphics in terminals (the various htop replacements with terminal graphs, and this)?
- sudara 5y ago
  You found the Achilles' heel of the project! I don't mind the fonts being different sizes or not monospaced, but the fact that there's a difference in height in the font rendering between ⎺ and ‾ on different platforms is a bummer. I have a flag in the C++ implementation so they can still be rendered "correctly" in IDEs like Xcode [1]. I felt doomed to Unicode in this case because of the number of places I wanted them to show up (CLion lldb integration, GitHub actions output, terminal). I would have loved to actually render graphics! I actually never thought about how they would render on a blog article, I wouldn't generally wouldn't use them for blogging... 1. https://github.com/sudara/melatonin_audio_sparklines/blob/main/melatonin_audio_sparklines.h#L89-L94 https://github.com/sudara/melatonin_audio_sparklines/blob/ma...
praash 5y ago
I think the additional zeroes, letters and crosses work heavily against visual intuition. In a single line of monospaced text, their disproportionate size breaks continuity and you're forced to reconstruct the actual shape in your head. [0⎻⎺⎻x—x⎻x—⎼⎽_E_⎽⎼—x⎻⎺‾E‾⎺⎻x_⎽⎼—0(234)] Inserting bracketed numbers throws the whole sparkline concept out the window. If terminal output is the use case here, why not use color when available? Zero crossings could be highlighted in blue, and out-of-bound clipping could be shown in red. Long sequences of zeroes could just be represented with a red line, or even empty space: [ ] [---- ] Silence should be easy to spot, not crammed behind numbers! [0(512)] [----0(256)] Which style would you find easier to understand at a glance?
- sudara 5y ago
  Thanks for the constructive feedback! It's nice to have some new ideas. In the audio contexts I'm working in, it is a feature to have the zero crossings and errors jump out. It's less "pretty" but is as data-dense as I could get just with unicode. > why not use color when available? You are right, color would increase data density. In both my test context and in my debugger, color is unfortunately unavailable... > Silence should be easy to spot, not crammed behind numbers! The condensed 0(256) notation is ugly, it's true. However, you can't miss it, and it's information dense. This was the toughest element to design and I tried a few other styles before this, because yes, it's ugly :) I'm often working with buffers with 256+ zeros, so zeros had to be condensed to stay legible. Simultaneously, it's often necessary to know the exact number of empty samples (they could occur at the start, middle or end of the block). A dash - for a 0 such as in [----0(256)] might have some ambiguity with a quiet part of a signal (it's important to differentiate true zeros) but your post has me thinking!...
  - samatman 5y ago
    My instinct would be to use hex for those sorts of runs, the compactness adds up. That might mean more annoyance in converting values then you're willing to tolerate, and perhaps I'm over-indexing on the 256 in your example.
  - howdydoo 5y ago
erwincoumans 5y ago
It looks nice but I would use OpenGL, ImGUI and STK instead. Specifically, I would use STK instead if JUCE (https://github.com/thestk/stk https://github.com/thestk/stk). In particular RtAudio.cpp, RtMidi.cpp, RtWvIn.cpp) and a local or remote visualizer based on ImGUI and OpenGL, since I have that code already I would only need to add a few lines to my audio app to debug visually, using shared memory (which is much faster than printf and works inside a debugger and anywhere else, on Linux, MacOS and Windows)
- stinos 5y ago
  Wanted to say the same, because I've used in in the past. Don't remember the details, but basically a custom visualizer in VS which forwarded data to a Labview VI (because that has most basics, cursors, scaling builtin) over a local connection did wonders for debugging. Still, it's a cool idea this one.
- sudara 5y ago
  STK is cool! I'm building a synth plugin, so that's why I'm in JUCE... I want to hear more about the remote visualizer! I was using Jim Credland's buffer debugger (again, JUCE)[1]. It pops open a window, making it fairly easy to visualize buffers with one caveat: you actually have to tell it about the buffers you care about visualizing and then recompile. This means debugging can't really be on the fly (unless you only always care about the same buffer or two). The other issue I ran into was viewing audio from my tests. I'd love to hear about the shared memory approach — real waveforms would be ideal! 1. https://github.com/jcredland/juce-toys/blob/master/jcf_debug/source/buffer_debugger.h https://github.com/jcredland/juce-toys/blob/master/jcf_debug...
  - erwincoumans 5y ago
    Let me see, I could create a minimal example using your Buffer API with shared memory and share it on github, if there is interest (upvote?)
    - scoopdewoop 5y ago
gavinray 5y ago
Holy smokes, really cool idea and incredibly useful for audio dev folks!

240 points by wlll 5y ago

These visual displays are very useful. Here's how the Julia REPL shows them: https://github.com/JuliaAudio/SampledSignals.jl#repl-display https://github.com/JuliaAudio/SampledSignals.jl#repl-display

sudara 5y ago
This is great, thanks! I looked for existing unicode waveform representations before building, but didn't run into this. The unicode blocks the Julia REPL uses limits the output to magnitude (signed) vs. amplitude (unsigned), but it gets the job done!

ipsum2 5y ago

Braille may also work, and has the benefit of being more compact. If two dots represents a point, an example might be ⣤⣀⣀⣀⡤⠴⠚⠛⠒⠒⠲⣀ (usually terminal fonts hide the white dots). You only get 4 horizontal ticks, but also can do a "interpolation" (like ⠑) that gets you an extra 3.

rsync 5y ago

I'm looking at this final picture in your IDE: https://melatonin.dev/wp-content/uploads/2021/10/image.jpeg https://melatonin.dev/wp-content/uploads/2021/10/image.jpeg ... and I think that is a GUI and not a terminal ... but I'm not sure ? I think the folder icon in the lower left is good evidence that I am looking at a GUI window but would like you to confirm/deny ...

ninkendo 5y ago
It’s some flavor of Jetbrains IDE like IntelliJ, maybe CLion?
- smoofles 5y ago
  Yeah, CLion
BHSPitMonkey 5y ago
It's displaying text-based data passed from a debugger.
picture 5y ago
What font is that?
- smoofles 5y ago
  Looks like MonoLisa, at a first glance.

anon9001 5y ago

As a companion to this, it would be cool if there was a way to render any audio file as a spectrogram (https://en.wikipedia.org/wiki/Spectrogram https://en.wikipedia.org/wiki/Spectrogram) directly in the terminal. Since this is HN, does anyone know what are the best ways to draw graphics into modern terminals?

mprovost 5y ago
Check out notcurses: https://github.com/dankamongmen/notcurses https://github.com/dankamongmen/notcurses
joveian 5y ago
I think Kitty's terminal graphics protocol is the most advanced, although I think there are a couple of others in use as well: https://sw.kovidgoyal.net/kitty/graphics-protocol/ https://sw.kovidgoyal.net/kitty/graphics-protocol/

opminion 5y ago

Sonic Pi, a programmable instrument, has sparklines, which are typically used to represent sequences of midi notes.

zestyping 5y ago

Simple and effective! Very nice.

sandermvanvliet 5y ago

I’ve used a similar approach when visualising telemetry reliability for a whole bunch of real world assets as well as in performance/stress tests. It’s so nice to be able to use this sort of thing in tests when the outcomes don’t represent as text naturally. Makes it super easy to eyeball a failing test and intuitively understand what the problem is.

p0nce 5y ago

Really cool stuff.

jonpalmisc 5y ago

The integration with LLDB here seems really neat and makes me think about what other LLDB scripts could be made to improve debugging of common structures and data. It’s be neat to be able to visualize a tree, etc. in LLDB.

polyterative 5y ago

nice

nyanpasu64 5y ago

On my tweaked Firefox profile, _⎽⎼—⎻⎺‾ renders as 3 different fonts (Courier New, IBM Plex Mono (my monospace font), and Segoe UI Symbol), with different widths and heights, some of which aren't even monospace. On a clean profile, it's only Courier New and Segoe UI Symbol. In both Firefox profiles, ⎺ renders higher than ‾ despite coming before it in order. It might work on certain fonts and terminals. It's not a general solution for blogs IMO. Are we better off with terminals with first-class support for sub-cell graphics rather than text, considering people tend to hack in graphics in terminals (the various htop replacements with terminal graphs, and this)?

sudara 5y ago
You found the Achilles' heel of the project! I don't mind the fonts being different sizes or not monospaced, but the fact that there's a difference in height in the font rendering between ⎺ and ‾ on different platforms is a bummer. I have a flag in the C++ implementation so they can still be rendered "correctly" in IDEs like Xcode [1]. I felt doomed to Unicode in this case because of the number of places I wanted them to show up (CLion lldb integration, GitHub actions output, terminal). I would have loved to actually render graphics! I actually never thought about how they would render on a blog article, I wouldn't generally wouldn't use them for blogging... 1. https://github.com/sudara/melatonin_audio_sparklines/blob/main/melatonin_audio_sparklines.h#L89-L94 https://github.com/sudara/melatonin_audio_sparklines/blob/ma...

praash 5y ago

I think the additional zeroes, letters and crosses work heavily against visual intuition. In a single line of monospaced text, their disproportionate size breaks continuity and you're forced to reconstruct the actual shape in your head. [0⎻⎺⎻x—x⎻x—⎼⎽_E_⎽⎼—x⎻⎺‾E‾⎺⎻x_⎽⎼—0(234)] Inserting bracketed numbers throws the whole sparkline concept out the window. If terminal output is the use case here, why not use color when available? Zero crossings could be highlighted in blue, and out-of-bound clipping could be shown in red. Long sequences of zeroes could just be represented with a red line, or even empty space: [ ] [---- ] Silence should be easy to spot, not crammed behind numbers! [0(512)] [----0(256)] Which style would you find easier to understand at a glance?

sudara 5y ago
Thanks for the constructive feedback! It's nice to have some new ideas. In the audio contexts I'm working in, it is a feature to have the zero crossings and errors jump out. It's less "pretty" but is as data-dense as I could get just with unicode. > why not use color when available? You are right, color would increase data density. In both my test context and in my debugger, color is unfortunately unavailable... > Silence should be easy to spot, not crammed behind numbers! The condensed 0(256) notation is ugly, it's true. However, you can't miss it, and it's information dense. This was the toughest element to design and I tried a few other styles before this, because yes, it's ugly :) I'm often working with buffers with 256+ zeros, so zeros had to be condensed to stay legible. Simultaneously, it's often necessary to know the exact number of empty samples (they could occur at the start, middle or end of the block). A dash - for a 0 such as in [----0(256)] might have some ambiguity with a quiet part of a signal (it's important to differentiate true zeros) but your post has me thinking!...
- samatman 5y ago
  My instinct would be to use hex for those sorts of runs, the compactness adds up. That might mean more annoyance in converting values then you're willing to tolerate, and perhaps I'm over-indexing on the 256 in your example.
- howdydoo 5y ago

erwincoumans 5y ago

It looks nice but I would use OpenGL, ImGUI and STK instead. Specifically, I would use STK instead if JUCE (https://github.com/thestk/stk https://github.com/thestk/stk). In particular RtAudio.cpp, RtMidi.cpp, RtWvIn.cpp) and a local or remote visualizer based on ImGUI and OpenGL, since I have that code already I would only need to add a few lines to my audio app to debug visually, using shared memory (which is much faster than printf and works inside a debugger and anywhere else, on Linux, MacOS and Windows)

stinos 5y ago
Wanted to say the same, because I've used in in the past. Don't remember the details, but basically a custom visualizer in VS which forwarded data to a Labview VI (because that has most basics, cursors, scaling builtin) over a local connection did wonders for debugging. Still, it's a cool idea this one.
sudara 5y ago
STK is cool! I'm building a synth plugin, so that's why I'm in JUCE... I want to hear more about the remote visualizer! I was using Jim Credland's buffer debugger (again, JUCE)[1]. It pops open a window, making it fairly easy to visualize buffers with one caveat: you actually have to tell it about the buffers you care about visualizing and then recompile. This means debugging can't really be on the fly (unless you only always care about the same buffer or two). The other issue I ran into was viewing audio from my tests. I'd love to hear about the shared memory approach — real waveforms would be ideal! 1. https://github.com/jcredland/juce-toys/blob/master/jcf_debug/source/buffer_debugger.h https://github.com/jcredland/juce-toys/blob/master/jcf_debug...
- erwincoumans 5y ago
  Let me see, I could create a minimal example using your Buffer API with shared memory and share it on github, if there is interest (upvote?)
  - scoopdewoop 5y ago

gavinray 5y ago

Holy smokes, really cool idea and incredibly useful for audio dev folks!