6 ms·

Nip2: A spreadsheet-like GUI for the libvips image processing library

97 points by valgaze 4y ago

makeworld 4y ago
Reminds me a bit of GRIP. Unfortunately there seems to be no screenshots, but the flow/modular interface is similar. https://wpiroboticsprojects.github.io/GRIP/#/ https://wpiroboticsprojects.github.io/GRIP/#/
- dansman805 4y ago
  There are some screenshots of GRIP here https://docs.wpilib.org/en/stable/docs/software/vision-processing/grip/introduction-to-grip.html https://docs.wpilib.org/en/stable/docs/software/vision-proce...
- jcupitt 4y ago
  Yes, I based it partly on visual programming system that Canon made back in the 90s. I loved the interactive development, but I really disliked that style of visual programming. Lines between boxes seems very hard to reason about, to me. nip2 tries to do something similar, but using expressions, like a spreadsheet. Unlike graphical plugs and wires, expressions allow things like copy-paste, smart refactoring, referential transparency, and progressive rewriting. nip2 is ancient now (I finished it way back in 2002, I think). I've started a rewrite for gtk4 -- the image display window is here: https://github.com/jcupitt/vipsdisp https://github.com/jcupitt/vipsdisp It's quite fancy -- it has a sparse pyramid of image tiles which get composited to your display by the GPU, and computed asynchronously by a libvips pipeline as you zoom and pan around. You should get a smooth 60 fps even with enormous (eg. 300,000 x 300,000 pixel) images.
Cyberdog 4y ago
GitHub says that 32.3% of the codebase is in "Witcher Script." What the heck is that? > Witcher Script (.ws) is the primary scripting language for The Witcher 3: Wild Hunt. A very large chunk of the game's logic is written in Witcher Script. Mods can override scripts and thus can drastically change game behavior. Witcher Script is either based on or identical to ActionScript 3 (and very similar to UnrealScript). https://witcher-games.fandom.com/wiki/Witcher_Script https://witcher-games.fandom.com/wiki/Witcher_Script Interesting. CDPR had a NIH problem. At any rate I'm guessing GitHub is misidentifying something.
- dljsjr 4y ago
  If you hit `T` to open the file searcher and look for `.ws` it brings up a bunch of "Workspace" XML files. So it's misidentifying a config file.
- jcupitt 4y ago
  nip2 has `.ws` files for storing workspaces (just a big chunk of XML), and `.def` files for its own scripting language. The nip2 scripting language is also very odd: it's something like dynamically typed Haskell, but with classes. You can see roughly what it looks like here: https://github.com/libvips/nip2/blob/master/share/nip2/start/_list.def#L339-L363 https://github.com/libvips/nip2/blob/master/share/nip2/start... All of the menus are implemented in this thing.
  - electroly 4y ago
    You can use a linguist directive in your .gitattributes file if you're interested in fixing GitHub's language identification for those files. A line like the following would hide these files from GitHub's language meter bar. *.ws linguist-documentation
    - jcupitt 4y ago
      Hi electroly, thanks for the tip!
- game-of-throws 4y ago
  > CDPR had a NIH problem. Bit of a tangent, but I don't think that's true. It's just ActionScript with a cute file extension - just like .docx is .xml with a cute file extension.
rsstack 4y ago
What makes this a spreadsheet rather than a notebook (like Mathematica or Jupyter)? It's still great, I just expected something else - something more... spreadsheety.
- jcupitt 4y ago
  I suppose it's only a spreadsheet in a very broad sense: a set of formula which refer to each other, and which are updated automatically on any change. You can't have circular references to the top level, but you can do recursive definitions within a cell, which is maybe spreadsheety too. You can use scraps of code to set the GUI behaviour, which is a bit crazy. Try making a region on an image (open an image view window, hold down CTRL, drag down and right). Back in the main window, press the down button to the left of to the label of the cell containing the region and you'll find you can open it up. Keep clicking and a lot of rows will appear -- including left, top, width, height, etc. If you drag the region around and resize it, you'll see all these numbers update. Click on the number for height and you can edit it. Enter "width * 2" and try resizing the region.
  - JodieBenitez 4y ago
    I don't use Nip2 (yet), but since you're here let me express how grateful I am for libvips and its Python bindings. It's a fantastic and essential tool.
    - jcupitt 4y ago
      I'm glad it's useful!
  - rsstack 4y ago
    I installed nip2 and played with it for a while. It is _the best_ notebook/spreadsheet tool I ever experienced. I don't know if the folks from Equals.app have notifications on their name in Hacker News, but they should consider taking some inspiration from nip2 and not just from Excel.
  - orbital-decay 4y ago
ArrayBoundCheck 4y ago
Does it send me notifications? Is it called NipAlert2? Is professor bighetti a collaborator?
snerbles 4y ago
This seems similar to Cognex's spreadsheet programming model in their industrial smart cameras.
erdmann 4y ago
I would also like to sing the praises of nip2. I use it constantly in my work with enormous images at the Rijksmuseum. It's backed by libvips, which means that it offers a delightful combination of insane speed and unlimited image size. The finishing touches of the 717 gigapixel image of the Night Watch were done in nip2. Also, the spreadsheet-like nature of the interface is excellent for designing, documenting, and repeating complex workflows. Cells contain images, and the reactive trickle-down computation lets you build up multi-step transformations and computations cell by cell. Any cell's contents can be replaced by a new image at will, so it's very easy to repeat an analysis or transformation for new source images. John Cupitt (jcupitt here), the main developer of nip2 and libvips, is super helpful, responsive, generous, and patient. They can pry nip2 from my cold, dead hands.
- jcupitt 4y ago
  Thank you! (the money is in the envelope on your desk)
- kawsper 4y ago
  I've reported many strange bugs to jcupitt and I can speak to his really helpful and patient nature. My favorite one was when JPEG2000s that had an even width was being shown in greyscale on any mac app: Preview, Safari, Pixelmator, etc: https://github.com/libvips/ruby-vips/issues/345 https://github.com/libvips/ruby-vips/issues/345 All images with an odd width was working fine. However the real bug was to do with chroma subsampling (and a lot of other technical stuff that I don't really understand): https://github.com/libvips/libvips/issues/2965 https://github.com/libvips/libvips/issues/2965
hawski 4y ago
This looks nice and useful. I was thinking for a long time about a graphics manipulation program that would put every step as a separate line with all parameters visible. Then you can change any parameter from earlier steps as you see fit. It makes the graphics editing process like a little interactive script. You could then easily parametrize it further and even do batch processing this way. I will have to check this out.
- jcupitt 4y ago
  Yes, nip2 has a no-GUI batch mode. You can make a workspace interactively that does some task, then run it with something like: nip2 -bp --set 'Workspaces.process.A1=Image_file "fred.jpg"' --set main=Workspaces.process.A10 -o fred2.jpg process.ws -b meaning batch mode, -p meaning send the value of main to the output. That'll load the process.ws workspace, set cell A1 to be the image "fred.jpg", set main (the point of execution) to be cell A10, and save it to fred2.jpg. I made a workspace for medical image analysis (my old job). It finds lungs in a 4D PET-CT scan, then fits a two compartment model per voxel and generates a volume image from k2. It has just over 10,000 cells (!!), 70GB of images, and executes in about 2 minutes using around 1GB of ram. I then used a small bash script to loop over my dataset of 2,000 scans computing an inflammation index. It worked surprisingly well, IMO.

97 points by valgaze 4y ago

Reminds me a bit of GRIP. Unfortunately there seems to be no screenshots, but the flow/modular interface is similar. https://wpiroboticsprojects.github.io/GRIP/#/ https://wpiroboticsprojects.github.io/GRIP/#/

dansman805 4y ago
There are some screenshots of GRIP here https://docs.wpilib.org/en/stable/docs/software/vision-processing/grip/introduction-to-grip.html https://docs.wpilib.org/en/stable/docs/software/vision-proce...
jcupitt 4y ago
Yes, I based it partly on visual programming system that Canon made back in the 90s. I loved the interactive development, but I really disliked that style of visual programming. Lines between boxes seems very hard to reason about, to me. nip2 tries to do something similar, but using expressions, like a spreadsheet. Unlike graphical plugs and wires, expressions allow things like copy-paste, smart refactoring, referential transparency, and progressive rewriting. nip2 is ancient now (I finished it way back in 2002, I think). I've started a rewrite for gtk4 -- the image display window is here: https://github.com/jcupitt/vipsdisp https://github.com/jcupitt/vipsdisp It's quite fancy -- it has a sparse pyramid of image tiles which get composited to your display by the GPU, and computed asynchronously by a libvips pipeline as you zoom and pan around. You should get a smooth 60 fps even with enormous (eg. 300,000 x 300,000 pixel) images.

Cyberdog 4y ago

GitHub says that 32.3% of the codebase is in "Witcher Script." What the heck is that? > Witcher Script (.ws) is the primary scripting language for The Witcher 3: Wild Hunt. A very large chunk of the game's logic is written in Witcher Script. Mods can override scripts and thus can drastically change game behavior. Witcher Script is either based on or identical to ActionScript 3 (and very similar to UnrealScript). https://witcher-games.fandom.com/wiki/Witcher_Script https://witcher-games.fandom.com/wiki/Witcher_Script Interesting. CDPR had a NIH problem. At any rate I'm guessing GitHub is misidentifying something.

dljsjr 4y ago
If you hit `T` to open the file searcher and look for `.ws` it brings up a bunch of "Workspace" XML files. So it's misidentifying a config file.
jcupitt 4y ago
nip2 has `.ws` files for storing workspaces (just a big chunk of XML), and `.def` files for its own scripting language. The nip2 scripting language is also very odd: it's something like dynamically typed Haskell, but with classes. You can see roughly what it looks like here: https://github.com/libvips/nip2/blob/master/share/nip2/start/_list.def#L339-L363 https://github.com/libvips/nip2/blob/master/share/nip2/start... All of the menus are implemented in this thing.
- electroly 4y ago
  You can use a linguist directive in your .gitattributes file if you're interested in fixing GitHub's language identification for those files. A line like the following would hide these files from GitHub's language meter bar. *.ws linguist-documentation
  - jcupitt 4y ago
    Hi electroly, thanks for the tip!
game-of-throws 4y ago
> CDPR had a NIH problem. Bit of a tangent, but I don't think that's true. It's just ActionScript with a cute file extension - just like .docx is .xml with a cute file extension.

rsstack 4y ago

What makes this a spreadsheet rather than a notebook (like Mathematica or Jupyter)? It's still great, I just expected something else - something more... spreadsheety.

jcupitt 4y ago
I suppose it's only a spreadsheet in a very broad sense: a set of formula which refer to each other, and which are updated automatically on any change. You can't have circular references to the top level, but you can do recursive definitions within a cell, which is maybe spreadsheety too. You can use scraps of code to set the GUI behaviour, which is a bit crazy. Try making a region on an image (open an image view window, hold down CTRL, drag down and right). Back in the main window, press the down button to the left of to the label of the cell containing the region and you'll find you can open it up. Keep clicking and a lot of rows will appear -- including left, top, width, height, etc. If you drag the region around and resize it, you'll see all these numbers update. Click on the number for height and you can edit it. Enter "width * 2" and try resizing the region.
- JodieBenitez 4y ago
  I don't use Nip2 (yet), but since you're here let me express how grateful I am for libvips and its Python bindings. It's a fantastic and essential tool.
  - jcupitt 4y ago
    I'm glad it's useful!
- rsstack 4y ago
  I installed nip2 and played with it for a while. It is _the best_ notebook/spreadsheet tool I ever experienced. I don't know if the folks from Equals.app have notifications on their name in Hacker News, but they should consider taking some inspiration from nip2 and not just from Excel.
- orbital-decay 4y ago

ArrayBoundCheck 4y ago

Does it send me notifications? Is it called NipAlert2? Is professor bighetti a collaborator?

snerbles 4y ago

This seems similar to Cognex's spreadsheet programming model in their industrial smart cameras.

erdmann 4y ago

I would also like to sing the praises of nip2. I use it constantly in my work with enormous images at the Rijksmuseum. It's backed by libvips, which means that it offers a delightful combination of insane speed and unlimited image size. The finishing touches of the 717 gigapixel image of the Night Watch were done in nip2. Also, the spreadsheet-like nature of the interface is excellent for designing, documenting, and repeating complex workflows. Cells contain images, and the reactive trickle-down computation lets you build up multi-step transformations and computations cell by cell. Any cell's contents can be replaced by a new image at will, so it's very easy to repeat an analysis or transformation for new source images. John Cupitt (jcupitt here), the main developer of nip2 and libvips, is super helpful, responsive, generous, and patient. They can pry nip2 from my cold, dead hands.

jcupitt 4y ago
Thank you! (the money is in the envelope on your desk)
kawsper 4y ago
I've reported many strange bugs to jcupitt and I can speak to his really helpful and patient nature. My favorite one was when JPEG2000s that had an even width was being shown in greyscale on any mac app: Preview, Safari, Pixelmator, etc: https://github.com/libvips/ruby-vips/issues/345 https://github.com/libvips/ruby-vips/issues/345 All images with an odd width was working fine. However the real bug was to do with chroma subsampling (and a lot of other technical stuff that I don't really understand): https://github.com/libvips/libvips/issues/2965 https://github.com/libvips/libvips/issues/2965

hawski 4y ago

This looks nice and useful. I was thinking for a long time about a graphics manipulation program that would put every step as a separate line with all parameters visible. Then you can change any parameter from earlier steps as you see fit. It makes the graphics editing process like a little interactive script. You could then easily parametrize it further and even do batch processing this way. I will have to check this out.

jcupitt 4y ago
Yes, nip2 has a no-GUI batch mode. You can make a workspace interactively that does some task, then run it with something like: nip2 -bp --set 'Workspaces.process.A1=Image_file "fred.jpg"' --set main=Workspaces.process.A10 -o fred2.jpg process.ws -b meaning batch mode, -p meaning send the value of main to the output. That'll load the process.ws workspace, set cell A1 to be the image "fred.jpg", set main (the point of execution) to be cell A10, and save it to fred2.jpg. I made a workspace for medical image analysis (my old job). It finds lungs in a 4D PET-CT scan, then fits a two compartment model per voxel and generates a volume image from k2. It has just over 10,000 cells (!!), 70GB of images, and executes in about 2 minutes using around 1GB of ram. I then used a small bash script to loop over my dataset of 2,000 scans computing an inflammation index. It worked surprisingly well, IMO.