3 ms·

Following the discussion on ET, here is my current work on a tmux fork to focus on accurate rendering of graphics (ANSI and sixels) My main goal is to be able

by csdvrx 7y ago

Following the discussion on ET, here is my current work on a tmux fork to focus on accurate rendering of graphics (ANSI and sixels) My main goal is to be able to use sixel-gnuplot (a simple recompilation of gnuplot with the sixel backend enabled: https://github.com/csdvrx/sixel-gnuplot https://github.com/csdvrx/sixel-gnuplot ) on both remote and local hosts, in split windows. A few issues remain to do that, but it already works in horizontal split windows after 2 days: https://github.com/csdvrx/sixel-tmux/blob/master/sixel-tmux.jpg https://github.com/csdvrx/sixel-tmux/blob/master/sixel-tmux.... You can recompile the source within msys2 on Windows 10; I also provide a binary on https://github.com/csdvrx/sixel-tmux/raw/master/sixel-tmux.exe https://github.com/csdvrx/sixel-tmux/raw/master/sixel-tmux.e...

abcdef123xyz 7y ago
I took a look at this but at the moment you are just commenting out the bits that stop tmux sending SIXEL to the outside terminal, so there is still a lot of work to do for proper SIXEL support: - You don't check if the terminal outside even supports SIXEL. A major tmux design feature is that it avoids sending output to a terminal if it doesn't know it will understand it. - What if there are two clients attached, one in a terminal with SIXEL, one without? Does it need to draw an ASCII placeholder? - The SIXEL image needs to be saved so it can be redrawn if needed (current window is changed, detach/attach, scrolling into history etc). - If the SIXEL image is scrolled partly off the terminal or the window pans so it is partly off the terminal, the image needs to be cropped to the right size and drawn at the right place. - The problems with the timer and large DCS, just turning this off is bringing back the problems with fast output it is intended to solve. A better solution would be to only disable these while a DCS sequence is in progress. If you get it working with these addressed I would be open to merging it into tmux itself.
- abcdef123xyz 7y ago
  An achievable option in the short term would be to do the following: 1) Fix the DCS issue as I suggested. 2) Check if the terminal supports SIXEL (there is a code in DA or DA2, or perhaps we need a flag). 3) If the terminal supports SIXEL, pass it through, if it doesn't then ignore it. This would allow SIXEL to display on output although it would be corrupted as soon as tmux redraws. It would be a start however and could be added to tmux immediately.
  - abcdef123xyz123 7y ago
    You can try this: https://gist.githubusercontent.com/nicm/9acc00b67035b5f2b8a81ec11c197b15/raw/e8f0c259f8b8146670a331f4aac544d0787e9152/gistfile1.txt https://gist.githubusercontent.com/nicm/9acc00b67035b5f2b8a8... Which does the basics of this: - Bump the sequence timeout to 5 seconds instead of 100 ms for more time for SIXEL images but still not let the terminal hang forever. This could be made more sophisticated perhaps. - Check the code in DA for SIXEL support and also check a terminfo(5) extension flag Sxl. - Recognise SIXEL sequences and pass them through (like the 'tmux' passthrough escape sequence) and copy them to the attached outside terminals if they support SIXEL (DA or Sxl set). I didn't try to disable the terminal-can't-keep-up code during DCS, that is more complicated because some panes may be inside DCS and some may not. I only have a small test SIXEL image so I would need a larger one that demonstrates the problem to look at that. I probably won't move this any further forward unless someone tests this, but it may be a good start for further work on SIXEL.
    - csdvrx 7y ago
      Interesting, I will have a look and test it, thanks! For images to test, on debian I recommend you install libsixel. You can then just use img2sixel to convert a jpg to sixel. Alternatively, install gnuplot-nox and: export GNUTERM="sixelgd size 1280,720 truecolor font arial 16" then you can test with various screen resolution and sizes, like: echo plot sin(x) |gnuplot Yes, my approach does not solves all problems. It is just a quick and dirty fix to show sixels in the current window because I use gnuplot very intensively on remote hosts. As soon as I switch to another pane, the output is lost. But I use plots mostly to get a quick idea of how things are going, every few minutes or hours. About https://news.ycombinator.com/item?id=21646398 https://news.ycombinator.com/item?id=21646398 : 1. yes I do not check for sixel capacity. My approach is that any sequence I do not know about should be left as is, because it may have been made for a good reason, and there will always be newer sequences and terminals I don't know about. For graphics such as sixels, your approach to check for sixel capability might be better as they will need to be redrawn 2. what to do if there are 2 terminals attached is linked to #1. I would by default output the same sequence, because I do not touch what I do not know! If you are checking capabilities, it may be best to show an empty block of the same size at the place where the sixel output would be (a placeholder). Another option would be to use a converter like libcaca to turn the sixel into a rough ascii rendition. It would add a dependency to tmux, but also add a feature for terminals that can't keep up, here because they don't know how to interpret sixel 3. and 4. saving the sixel image is what mlterm does. You could save the .six files in ~/tmux/%PID%WIN%PANE If you want to be consistent with text lines, you could keep all images until the text line after the sixel image is thrown out of the history.

Following the discussion on ET, here is my current work on a tmux fork to focus on accurate rendering of graphics (ANSI and sixels) My main goal is to be able

by csdvrx 7y ago

abcdef123xyz 7y ago

I took a look at this but at the moment you are just commenting out the bits that stop tmux sending SIXEL to the outside terminal, so there is still a lot of work to do for proper SIXEL support: - You don't check if the terminal outside even supports SIXEL. A major tmux design feature is that it avoids sending output to a terminal if it doesn't know it will understand it. - What if there are two clients attached, one in a terminal with SIXEL, one without? Does it need to draw an ASCII placeholder? - The SIXEL image needs to be saved so it can be redrawn if needed (current window is changed, detach/attach, scrolling into history etc). - If the SIXEL image is scrolled partly off the terminal or the window pans so it is partly off the terminal, the image needs to be cropped to the right size and drawn at the right place. - The problems with the timer and large DCS, just turning this off is bringing back the problems with fast output it is intended to solve. A better solution would be to only disable these while a DCS sequence is in progress. If you get it working with these addressed I would be open to merging it into tmux itself.

abcdef123xyz 7y ago
An achievable option in the short term would be to do the following: 1) Fix the DCS issue as I suggested. 2) Check if the terminal supports SIXEL (there is a code in DA or DA2, or perhaps we need a flag). 3) If the terminal supports SIXEL, pass it through, if it doesn't then ignore it. This would allow SIXEL to display on output although it would be corrupted as soon as tmux redraws. It would be a start however and could be added to tmux immediately.
- abcdef123xyz123 7y ago
  You can try this: https://gist.githubusercontent.com/nicm/9acc00b67035b5f2b8a81ec11c197b15/raw/e8f0c259f8b8146670a331f4aac544d0787e9152/gistfile1.txt https://gist.githubusercontent.com/nicm/9acc00b67035b5f2b8a8... Which does the basics of this: - Bump the sequence timeout to 5 seconds instead of 100 ms for more time for SIXEL images but still not let the terminal hang forever. This could be made more sophisticated perhaps. - Check the code in DA for SIXEL support and also check a terminfo(5) extension flag Sxl. - Recognise SIXEL sequences and pass them through (like the 'tmux' passthrough escape sequence) and copy them to the attached outside terminals if they support SIXEL (DA or Sxl set). I didn't try to disable the terminal-can't-keep-up code during DCS, that is more complicated because some panes may be inside DCS and some may not. I only have a small test SIXEL image so I would need a larger one that demonstrates the problem to look at that. I probably won't move this any further forward unless someone tests this, but it may be a good start for further work on SIXEL.
  - csdvrx 7y ago
    Interesting, I will have a look and test it, thanks! For images to test, on debian I recommend you install libsixel. You can then just use img2sixel to convert a jpg to sixel. Alternatively, install gnuplot-nox and: export GNUTERM="sixelgd size 1280,720 truecolor font arial 16" then you can test with various screen resolution and sizes, like: echo plot sin(x) |gnuplot Yes, my approach does not solves all problems. It is just a quick and dirty fix to show sixels in the current window because I use gnuplot very intensively on remote hosts. As soon as I switch to another pane, the output is lost. But I use plots mostly to get a quick idea of how things are going, every few minutes or hours. About https://news.ycombinator.com/item?id=21646398 https://news.ycombinator.com/item?id=21646398 : 1. yes I do not check for sixel capacity. My approach is that any sequence I do not know about should be left as is, because it may have been made for a good reason, and there will always be newer sequences and terminals I don't know about. For graphics such as sixels, your approach to check for sixel capability might be better as they will need to be redrawn 2. what to do if there are 2 terminals attached is linked to #1. I would by default output the same sequence, because I do not touch what I do not know! If you are checking capabilities, it may be best to show an empty block of the same size at the place where the sixel output would be (a placeholder). Another option would be to use a converter like libcaca to turn the sixel into a rough ascii rendition. It would add a dependency to tmux, but also add a feature for terminals that can't keep up, here because they don't know how to interpret sixel 3. and 4. saving the sixel image is what mlterm does. You could save the .six files in ~/tmux/%PID%WIN%PANE If you want to be consistent with text lines, you could keep all images until the text line after the sixel image is thrown out of the history.