9 ms·

Unicode-based scientific plotting for working in the terminal

136 points by delib 6y ago

sprash 6y ago
I recently found out that XTerm supports sixels natively if started with the "-ti 340" option. For example gnuplot can make use of that feature using the command "set terminal sixlegd". It turns out because of the way sixels are encoded (six vertical pixels in a row using bitplanes) it packs better than png when used in conjunction with the bzip2 packer. Every image can simply be encoded to sixel with imagemagick using "convert image.png sixel:- > image.six" Personally I think sixels are genius should be used much more often.
- petepete 6y ago
  GNUPlot and XTerm also support Tektronix, there's lots of info in this older discussion. https://news.ycombinator.com/item?id=7776904 https://news.ycombinator.com/item?id=7776904
- mongol 6y ago
  The Kitty terminal author also published a "terminal graphics protocol". I don't know what other terminal supports it, however. https://sw.kovidgoyal.net/kitty/graphics-protocol.html https://sw.kovidgoyal.net/kitty/graphics-protocol.html
- dTal 6y ago
  >it packs better than png when used in conjunction with the bzip2 packer. Cute! But is it fair? I don't think it's lossless - the color depth is much reduced.
  - sprash 6y ago
    I took the reduced color depth into consideration. However on second glance it seems like Imagemagick does not compress PNG very efficiently with default settings. After recompressing with zopflipng PNG was easily able to beat bzip2 packed sixels. Results for 256 color 512x512 Lenna: Sixel + bzip2: 245K PNG Imagemagick: 467K PNG Zopfli 10 Iterations: 188K
- brutt 6y ago
  Sixels will be added to libvte (gnome-terminal): https://gitlab.gnome.org/GNOME/vte/-/issues/254 https://gitlab.gnome.org/GNOME/vte/-/issues/254
  - iso-8859-1 6y ago
    Great news! I think we may finally see a resurgence. It will be well supported on mainstream Linux, and given that the web is getting so heavy-weight, this will offer a solid alternative for graphics that you can rely on. Libvte is very popular.
- leephillips 6y ago
  That should be "sixelgd", of course. And there is the classic "dumb" terminal, that plots to any text terminal, optionally using ansi escape codes for color.
- iso-8859-1 6y ago
  I used your ImageMagick command, but the resulting image is very bright and almost invisible. I tried disabling Unicode using +u8, but that didn't work. I tried mlterm, that worked.
  - sprash 6y ago
    I forgot to tell that you have to enable 256 colors as well. Put this into your .Xresources: XTerm.*.decTerminalID: vt340 XTerm.*.numColorRegisters: 256 And refresh with xrdb -merge $HOME/.Xresources It should work now.
visviva 6y ago
This is one of my favorite Julia packages! A fast plotting utility that outputs right to the terminal is one of those things that I didn't realize I needed until I found it.
- vok 6y ago
  I also like that I can incorporate plots into my plain-text notes files.
- ViralBShah 6y ago
  Yes. Some people think this is a gimmick. But it is actually quite a bit useful. Especially, when in the middle of a session, I often do not want to move my hands from the keyboard to the mouse to close windows that plotting tools open, or move my gaze. It's not often that I work like this, but I like it that I can for the times I need to. Having inline plots in the terminal - especially things like histograms which just work - is often quite handy. And it just works over an ssh connection. Also, recently, Julia changed its sparse matrix display to show a picture of the sparse matrix (spy plot) using braille characters! julia> a = sprand(20,20,0.2) 20×20 SparseMatrixCSC{Float64,Int64} with 67 stored entries: ⠀⠀⠁⠐⠄⠋⠡⠀⠐⡀ ⢳⠁⠡⠈⠂⠀⠀⠁⢡⠑ ⡠⠋⢠⡀⢄⢀⢀⠄⠈⠈ ⠩⠄⠐⠨⠈⠀⠆⠂⠨⠤ ⠄⠄⢀⢂⠠⢀⠀⠂⠆⠂
michaelhoffman 6y ago
This is great! Anyone know of anything like this for Python or R?
- metaodi 6y ago
  I guess plotille (https://github.com/tammoippen/plotille https://github.com/tammoippen/plotille) is the closest match for python. And maybe termgraph (https://github.com/mkaz/termgraph https://github.com/mkaz/termgraph) is kind of similar.
  - foob 6y ago
    It's not unicode, but there's also sixelplot [1] which allows matplotlib plotting in a terminal using sixel. The underlying pysixel library [2] unfortunately doesn't support Python 3.7+ however. [1] https://github.com/kktk-KO/sixelplot https://github.com/kktk-KO/sixelplot [2] https://github.com/saitoha/PySixel https://github.com/saitoha/PySixel
    - iso-8859-1 6y ago
      There is a PR that fixes it for 3.7, it needs review: https://github.com/saitoha/PySixel/pull/5 https://github.com/saitoha/PySixel/pull/5
- dheera 6y ago
  I also wrote a more generic Python terminal plotting library: https://github.com/dheera/python-termgraphics https://github.com/dheera/python-termgraphics For robotics people -- the above is used in rosshow: http://github.com/dheera/rosshow http://github.com/dheera/rosshow
- ianhorn 6y ago
  I made a dead simple, somehow surprisingly useful one for python back in 2014. I still use it for quick investigations all the time. hipsterplot: https://github.com/imh/hipsterplot https://github.com/imh/hipsterplot
- SubiculumCode 6y ago
dorfsmay 6y ago
As somebody who grew up during the semi-graphic era, it makes me sad that we still use a 30 year old standard for terminals and cannot do direct plot in the main terminal (as opposed to the Tektronix separate window).
- tempodox 6y ago
  I was thinking something similar. Looking at this makes me wish a terminal app could just embed a PNG or whatever in its output.
  - kps 6y ago
    The OSC 1337 protocol pioneered by iTerm2 does exactly that. You'll have to check whether your preferred terminal supports it (I known mlterm does), either from documentation or just trying it¹. Also, some terminals support the DEC SIXEL protocol² (again, I know mlterm does). ¹ https://www.iterm2.com/utilities/imgcat https://www.iterm2.com/utilities/imgcat ² https://en.wikipedia.org/wiki/Sixel https://en.wikipedia.org/wiki/Sixel Edit: I couldn't find a simple SIXEL sample — you'd think one of the popular sixel libraries would have some in unit tests, but no. So for the purpose of this thread (but maybe not forever, hence /tmp/), curl https://datatravelandexperiments.com/kps/tmp/example.sixel curl https://datatravelandexperiments.com/kps/tmp/example.osc1337 Those are such that output directly to the terminal will produce an image if that format is supported (and probably nothing if not, but I'm not making promises).
  - z3t4 6y ago
    I heard they have something similar at Cern and it also has clickable links and universal resource locators.
- KenoFischer 6y ago
  Several terminals (kitty, iTerm2) do support plotting right into the terminal, but this is still sometimes useful if you don't want to load a while graphics stack just to do a simple plot.
  - 6y ago
charlesdaniels 6y ago
Every time I see these things, I wish Sixel graphics[0] were better supported, or else some more modern alternative. Being able to use GNUPlot with Sixel output or tools like lsix[1] is really cool! I think if support / libraries / knowledge about Sixels was more widespread, we would see some really nifty terminal applications cropping up. 0 - https://saitoha.github.io/libsixel/ https://saitoha.github.io/libsixel/ 1 - https://github.com/hackerb9/lsix https://github.com/hackerb9/lsix
- fishmaster 6y ago
  This looks pretty cool. Why isn't this more popular?
  - dheera 6y ago
    Most of the "default" terminal apps (e.g. gnome-terminal) don't support sixel.
- dheera 6y ago
  I wrote a generic terminal graphics library that uses Unicode Braille with the option of ASCII fallback: https://github.com/dheera/python-termgraphics https://github.com/dheera/python-termgraphics I was thinking about adding sixel support to this to have a universal graphics library that intelligently uses the best method available., but never got around to it yet.
- andi999 6y ago
  What terminal do I need to see these images. I am confused.
  - charlesdaniels 6y ago
    The libsixel website lists several that support it. Xterm can support this in VT430 emulation mode, but support needs to be enabled at compile time.
antirez 6y ago
Trivia: Redis uses a similar technique (Braille unicode points) to show the Redis 5 LOLWUT. http://antirez.com/news/123 http://antirez.com/news/123
lsllc 6y ago
This is awesome. When I build my spaceship, it's going to have a command line interface and I'll use this to display pertinent spaceship-data, such as gravitational pull of nearby black holes, tachyon levels, and warp signatures etc. Can't safely use a touchscreen or mouse when all hell breaks loose in space!

136 points by delib 6y ago

I recently found out that XTerm supports sixels natively if started with the "-ti 340" option. For example gnuplot can make use of that feature using the command "set terminal sixlegd". It turns out because of the way sixels are encoded (six vertical pixels in a row using bitplanes) it packs better than png when used in conjunction with the bzip2 packer. Every image can simply be encoded to sixel with imagemagick using "convert image.png sixel:- > image.six" Personally I think sixels are genius should be used much more often.

petepete 6y ago
GNUPlot and XTerm also support Tektronix, there's lots of info in this older discussion. https://news.ycombinator.com/item?id=7776904 https://news.ycombinator.com/item?id=7776904
mongol 6y ago
The Kitty terminal author also published a "terminal graphics protocol". I don't know what other terminal supports it, however. https://sw.kovidgoyal.net/kitty/graphics-protocol.html https://sw.kovidgoyal.net/kitty/graphics-protocol.html
dTal 6y ago
>it packs better than png when used in conjunction with the bzip2 packer. Cute! But is it fair? I don't think it's lossless - the color depth is much reduced.
- sprash 6y ago
  I took the reduced color depth into consideration. However on second glance it seems like Imagemagick does not compress PNG very efficiently with default settings. After recompressing with zopflipng PNG was easily able to beat bzip2 packed sixels. Results for 256 color 512x512 Lenna: Sixel + bzip2: 245K PNG Imagemagick: 467K PNG Zopfli 10 Iterations: 188K
brutt 6y ago
Sixels will be added to libvte (gnome-terminal): https://gitlab.gnome.org/GNOME/vte/-/issues/254 https://gitlab.gnome.org/GNOME/vte/-/issues/254
- iso-8859-1 6y ago
  Great news! I think we may finally see a resurgence. It will be well supported on mainstream Linux, and given that the web is getting so heavy-weight, this will offer a solid alternative for graphics that you can rely on. Libvte is very popular.
leephillips 6y ago
That should be "sixelgd", of course. And there is the classic "dumb" terminal, that plots to any text terminal, optionally using ansi escape codes for color.
iso-8859-1 6y ago
I used your ImageMagick command, but the resulting image is very bright and almost invisible. I tried disabling Unicode using +u8, but that didn't work. I tried mlterm, that worked.
- sprash 6y ago
  I forgot to tell that you have to enable 256 colors as well. Put this into your .Xresources: XTerm.*.decTerminalID: vt340 XTerm.*.numColorRegisters: 256 And refresh with xrdb -merge $HOME/.Xresources It should work now.

visviva 6y ago

This is one of my favorite Julia packages! A fast plotting utility that outputs right to the terminal is one of those things that I didn't realize I needed until I found it.

vok 6y ago
I also like that I can incorporate plots into my plain-text notes files.
ViralBShah 6y ago
Yes. Some people think this is a gimmick. But it is actually quite a bit useful. Especially, when in the middle of a session, I often do not want to move my hands from the keyboard to the mouse to close windows that plotting tools open, or move my gaze. It's not often that I work like this, but I like it that I can for the times I need to. Having inline plots in the terminal - especially things like histograms which just work - is often quite handy. And it just works over an ssh connection. Also, recently, Julia changed its sparse matrix display to show a picture of the sparse matrix (spy plot) using braille characters! julia> a = sprand(20,20,0.2) 20×20 SparseMatrixCSC{Float64,Int64} with 67 stored entries: ⠀⠀⠁⠐⠄⠋⠡⠀⠐⡀ ⢳⠁⠡⠈⠂⠀⠀⠁⢡⠑ ⡠⠋⢠⡀⢄⢀⢀⠄⠈⠈ ⠩⠄⠐⠨⠈⠀⠆⠂⠨⠤ ⠄⠄⢀⢂⠠⢀⠀⠂⠆⠂

michaelhoffman 6y ago

This is great! Anyone know of anything like this for Python or R?

metaodi 6y ago
I guess plotille (https://github.com/tammoippen/plotille https://github.com/tammoippen/plotille) is the closest match for python. And maybe termgraph (https://github.com/mkaz/termgraph https://github.com/mkaz/termgraph) is kind of similar.
- foob 6y ago
  It's not unicode, but there's also sixelplot [1] which allows matplotlib plotting in a terminal using sixel. The underlying pysixel library [2] unfortunately doesn't support Python 3.7+ however. [1] https://github.com/kktk-KO/sixelplot https://github.com/kktk-KO/sixelplot [2] https://github.com/saitoha/PySixel https://github.com/saitoha/PySixel
  - iso-8859-1 6y ago
    There is a PR that fixes it for 3.7, it needs review: https://github.com/saitoha/PySixel/pull/5 https://github.com/saitoha/PySixel/pull/5
dheera 6y ago
I also wrote a more generic Python terminal plotting library: https://github.com/dheera/python-termgraphics https://github.com/dheera/python-termgraphics For robotics people -- the above is used in rosshow: http://github.com/dheera/rosshow http://github.com/dheera/rosshow
ianhorn 6y ago
I made a dead simple, somehow surprisingly useful one for python back in 2014. I still use it for quick investigations all the time. hipsterplot: https://github.com/imh/hipsterplot https://github.com/imh/hipsterplot
SubiculumCode 6y ago

dorfsmay 6y ago

As somebody who grew up during the semi-graphic era, it makes me sad that we still use a 30 year old standard for terminals and cannot do direct plot in the main terminal (as opposed to the Tektronix separate window).

tempodox 6y ago
I was thinking something similar. Looking at this makes me wish a terminal app could just embed a PNG or whatever in its output.
- kps 6y ago
  The OSC 1337 protocol pioneered by iTerm2 does exactly that. You'll have to check whether your preferred terminal supports it (I known mlterm does), either from documentation or just trying it¹. Also, some terminals support the DEC SIXEL protocol² (again, I know mlterm does). ¹ https://www.iterm2.com/utilities/imgcat https://www.iterm2.com/utilities/imgcat ² https://en.wikipedia.org/wiki/Sixel https://en.wikipedia.org/wiki/Sixel Edit: I couldn't find a simple SIXEL sample — you'd think one of the popular sixel libraries would have some in unit tests, but no. So for the purpose of this thread (but maybe not forever, hence /tmp/), curl https://datatravelandexperiments.com/kps/tmp/example.sixel curl https://datatravelandexperiments.com/kps/tmp/example.osc1337 Those are such that output directly to the terminal will produce an image if that format is supported (and probably nothing if not, but I'm not making promises).
- z3t4 6y ago
  I heard they have something similar at Cern and it also has clickable links and universal resource locators.
KenoFischer 6y ago
Several terminals (kitty, iTerm2) do support plotting right into the terminal, but this is still sometimes useful if you don't want to load a while graphics stack just to do a simple plot.
- 6y ago

charlesdaniels 6y ago

Every time I see these things, I wish Sixel graphics[0] were better supported, or else some more modern alternative. Being able to use GNUPlot with Sixel output or tools like lsix[1] is really cool! I think if support / libraries / knowledge about Sixels was more widespread, we would see some really nifty terminal applications cropping up. 0 - https://saitoha.github.io/libsixel/ https://saitoha.github.io/libsixel/ 1 - https://github.com/hackerb9/lsix https://github.com/hackerb9/lsix

fishmaster 6y ago
This looks pretty cool. Why isn't this more popular?
- dheera 6y ago
  Most of the "default" terminal apps (e.g. gnome-terminal) don't support sixel.
dheera 6y ago
I wrote a generic terminal graphics library that uses Unicode Braille with the option of ASCII fallback: https://github.com/dheera/python-termgraphics https://github.com/dheera/python-termgraphics I was thinking about adding sixel support to this to have a universal graphics library that intelligently uses the best method available., but never got around to it yet.
andi999 6y ago
What terminal do I need to see these images. I am confused.
- charlesdaniels 6y ago
  The libsixel website lists several that support it. Xterm can support this in VT430 emulation mode, but support needs to be enabled at compile time.

antirez 6y ago

Trivia: Redis uses a similar technique (Braille unicode points) to show the Redis 5 LOLWUT. http://antirez.com/news/123 http://antirez.com/news/123

lsllc 6y ago

This is awesome. When I build my spaceship, it's going to have a command line interface and I'll use this to display pertinent spaceship-data, such as gravitational pull of nearby black holes, tachyon levels, and warp signatures etc. Can't safely use a touchscreen or mouse when all hell breaks loose in space!