6 ms·

GraphicsMagick Image Processing System

183 points by based2 10y ago

patio11 10y ago
GraphicsMagick has been one of the hidden gems I've found over the years in my career. HN would probably mostly be interested in it for server-side applications [+], but it is an absurdly useful thing to have sitting around your laptop, too, for those times when e.g. you've got a screenshot of something but a government/etc site will only take a file upload in pdf format. "gm convert foo.png foo.pdf" and you're done. [+] Really good for common "take an arbitrary user upload; grab info to crop it from your web app; crop and resize to appropriate dimensions" sorts of tasks.
- meira 10y ago
  The level of serendipity in HN is absurd. This is exactly what I was going to search for today. I'm going to use it local with Electron.
  - teleclimber 10y ago
    > I'm going to use it local with Electron Interesting. I also have a use case for this though I wasn't sure how easy it would be to bundle something like GM with my Electron app. Have you any insights on this? I'm targeting both Win and Mac.
    - mikewhy 10y ago
      Windows can use the compiled binary, Mac will need a static build. If you're using asar packages in your final app make sure it extracts those binaries then do a child_process.spawn
      - teleclimber 10y ago
        Thank you for the info. I'm looking for a link with all the details but I haven't found one so far. If you have that I would appreciate you sharing. Thanks!
    - fenomas 10y ago
      If the app uses JS, I recently did a bunch of image processing using the node GM wrapper ('gm' on npm), and it was generally a huge pain in the ass. I found that I couldn't do the edits I needed in a single atomic call to gm, and I couldn't get the wrapper to pipe multiple calls together, so I wound up having to write and manage temp files, and it was all pretty hairy. I later found a project called 'jimp', which is more or less similar to gm but written in pure JS, and which made it easy to chain calls together into nontrivial edits. Since it has no native dependencies it would probably be easier to bundle with electron as well. Presumably it's slower than gm of course (in my case that didn't matter, so I never looked back).
      - meira 10y ago
        Now this is real serendipity! I was guessing that I'd have a difficult time with this integration. Jumping straight to your recommendation.
        fenomas 10y ago
        Well, YMMV but for me what took me several days with the gm wrapper, I had working in about 15 minutes with jimp. Now that I think about it, I recall that the gm wrapper had a very strange API, that greatly obscured what parameters got passed to gm underneath, and figuring out how that worked was a big part of the headache. With jimp, everything basically did what it said on the tin. I remember wondering why such a useful library wasn't more famous.
        teleclimber 10y ago
        Jimp is something I had come across as well and played with a bit. FYI I wrote in my notes "dependency on jpeg-js is dog slow." I recommend some perf tests for your use case before jumping too far in.
        seandougall 10y ago
        I (and a number of other folks) have also run into an enormous memory leak, apparently originating in jpeg-js, when using jimp. On the order of "loading a 3MB file leaks 200MB of memory", which makes it unusable if you're at all RAM-constrained. Shame, because it has a lot of promise otherwise.
  - olalonde 10y ago
    And I was just debugging a "pixel cache allocation fail" bug with GM: https://github.com/aheckmann/gm/issues/598 https://github.com/aheckmann/gm/issues/598.
- pavlov 10y ago
  Personally I find that Apple Preview has always been good enough for these tasks. It can also do a surprising amount of PDF editing: for example, you can draw annotations on pages (for highlights and signatures), and selectively combine pages from multiple PDF documents into one by simple copy & paste.
  - bootload 10y ago
    "Personally I find that Apple Preview has always been good enough for these tasks." OSS is a good choice because not every developer works on a Mac and with binary apps. You can hack quite a few quick/dirty tools to deal with repetitive tasks. Having said that Apple Preview is a pretty useful tool.
  - pschastain 10y ago
    Except Preview 8 (post-Mavericks) sucks. Annotations of any kind always appear in the center of the page and have to be manually moved into position, it doesn't easily recognize selections, combined documents don't always retain changes, I could go on. I have a Mavericks VM just for Preview 7. [edit] grammar
- revelation 10y ago
  The issue with taking arbitrary user uploads and piping them directly into a program that happily accepts and interprets arbitrary data is security. They claim support for over 200 formats, some of which surely have exciting scripting functionality that no one ever uses: http://www.imagemagick.org/script/formats.php http://www.imagemagick.org/script/formats.php And there have been recent code execution vulnerability reports: https://imagetragick.com/ https://imagetragick.com/ Now libpng (e.g.) has issues as well but if at all possible I would try to restrict the input format and process only with software for the specific format.
- amelius 10y ago
  > HN would probably mostly be interested in it for server-side applications This image processing system would be very useful to have on the client. (Why can't we still use arbitrary stuff on the client that was written for the server?)
  - Spivak 10y ago
    You can if you're shipping a native app. But if you're targeting the web you're limited by tools written in JS or what you can transcribe to JS. asm.js and WebAssembly could very well change this in a few years.
    - bobajeff 10y ago
      To further clarify SIMD.js (along WebAssembly's SIMD.js hooks) will change this. Right now you can still port GraphicsMagick to emscripten you'd just have to accept a performance hit on the parts that rely on SIMD.
- deleted 10y ago
  [deleted]
bsenftner 10y ago
I was confused for a moment: Imagemagick and graphicsmagick are different, http://blog.trackduck.com/2014/11/12/imagemagick-vs-graphicsmagick-one-better-process-images-website/ http://blog.trackduck.com/2014/11/12/imagemagick-vs-graphics...
- ajross 10y ago
  GM is a fork. It's actually one of the more persistent and notable forks in the open source community. Both projects continue development and have no interest in reconciliation.
  - tgb 10y ago
    The last I heard about Image Magick was that it was out-dated and people were suggesting using other tools (though I don't remember what). Can anyone comment on that? This might have been only for a few specific tasks, not for Image Magick as a whole.
    - faitswulff 10y ago
      I don't know about now, but I've encountered significant issues with it every time I've attempted to use it in the last three years. To be fair, I was attempting to use it through wrappers - I had a particularly bad run trying to get around issues with both rmagick and minimagick in ruby, and later on a different project with wand in python. Though, with wand, the issues I encountered there were mostly with getting a buildpack working for it on Heroku. Regardless, whether it's my incompetence or imagemagick itself, I don't have very good associations with the library.
    - Washuu 10y ago
      GraphicsMagick fixed all the memory and CPU problems that ImageMagick was giving us in a large wiki farm production environment. That alone is a reason to switch to it.
      - pentium10 10y ago
mrcactu5 10y ago
CGAL https://news.ycombinator.com/item?id=12793220 https://news.ycombinator.com/item?id=12793220 (from a few days ago). Options!
- pavlov 10y ago
  But GraphicsMagick and CGAL are nothing alike?
jasode 10y ago
Fyi... I last looked at both GraphicsMagick and ImageMagick in 2013 when I was writing a custom batch image converter. Because GraphicsMagick had a reputation for better performance[1] I initially wrote my wrapper tool around GraphicsMagick. However, I encountered bugs with Photoshop PSD files (e.g. not properly reading multi-layer PSD files.[2]) That's when I found out that ImageMagick tracks the PSD file format more reliably than GraphicsMagick. Yes, GM later fixed that particular bug in 2014 but I had already revised my tool to use IM. Lesson learned: if processing PSD files is critical to your project, you'll want to favor ImageMagick over GraphicsMagick. The author of GM admits the parser of PSD is not the best so it's something to keep in mind. [1]Flickr and Etsy used GM on their backends: http://www.slideshare.net/jallspaw/operational-efficiency-hacks-web20-expo2009/33-Image_Processing_2004_Flickr_was http://www.slideshare.net/jallspaw/operational-efficiency-ha... http://www.web2expo.com/webexsf2009/public/schedule/detail/8580 http://www.web2expo.com/webexsf2009/public/schedule/detail/8... [2]https://sourceforge.net/p/graphicsmagick/bugs/242/ https://sourceforge.net/p/graphicsmagick/bugs/242/ [3]also some bonus links on why PSD is very difficult to parse correctly: https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m?utm_content=bufferae738#L108 https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m?... http://blogs.adobe.com/jnack/2009/05/some_thoughts_about_the_psd_format.html http://blogs.adobe.com/jnack/2009/05/some_thoughts_about_the...
- olalonde 10y ago
  PSD support is now disabled by default in GM apparently: https://github.com/Homebrew/homebrew-core/issues/6072#issuecomment-255293094 https://github.com/Homebrew/homebrew-core/issues/6072#issuec...
kjell 10y ago
vips is another great industrial-grade image processing library https://github.com/jcupitt/libvips https://github.com/jcupitt/libvips http://www.vips.ecs.soton.ac.uk/index.php?title=Libvips http://www.vips.ecs.soton.ac.uk/index.php?title=Libvips
- pbowyer 10y ago
  And let's not forget NetPBM either, which (when I was doing this a decade ago) was faster and more memory efficient than (Image|Graphics)Magick.
dahart 10y ago
If you need to process huge images, GraphicsMagick does some things that ImageMagick can't do. A few years ago when I was making poster sized art prints, I would routinely process 60k^2 pixel (4 gigapixel) images. Just a resize with a Gaussian filter would thrash and never finish overnight in ImageMagick, and only took a minute or two in GraphicsMagick. I'm not in love with the CLI for either, but GM's is better and I like that it's all collected under one roof rather than a whole bunch of separate commands. I do wish the projects could merge back into one and have the best of both. I also wish they were better at color handling, resizing needs to be done at a higher bit depth, better support for converting between modern color spaces, etc. I love sips on my mac because it's fast, but I'm not sure if anything's a better cross-platform choice for basic batch operations on images, or for the backend of a web service, than GraphicsMagick.
- rahimnathwani 10y ago
  I'm curious, as I want to make some poster-sized art prints soon: 1) What resolution were the source files? 2) What upscaling options did you use? 3) What multiple of the original file size did you go up to? Were you limited by the quality of the upscaling, or did you just decide on the size of print and dpi, and multiply?
  - dahart 10y ago
    Oh, I was downscaling, mainly for filtering/anti-aliasing. My source files were the 60k^2 images, and the output was more like 20k^2... I shoot for at least 300 dpi and I was printing 60-inch prints. I had enough trouble downscaling such large images that I ended up writing my own downscaling in my renderer so that I wouldn't have to use GraphicsMagick or anything else, but of everything I ever tried, GraphicsMagick was the best at handling enormous images. FWIW, I've never been happy with any upscaling. If you do have the option to render at your target res and not use upscaling, that's ideal. If you must upscale, it's hard to find anything that can do more that 2-3x without noticeable artifacts or visible blurriness.
    - rahimnathwani 10y ago
teilo 10y ago
GraphicsMagick is great, but it's not a replacement for ImageMagick. While it offers a few features not found in IM, it lags far behind on the more advanced stuff, such as image recognition/machine vision, and in general does not track the main project any longer.

183 points by based2 10y ago

GraphicsMagick has been one of the hidden gems I've found over the years in my career. HN would probably mostly be interested in it for server-side applications [+], but it is an absurdly useful thing to have sitting around your laptop, too, for those times when e.g. you've got a screenshot of something but a government/etc site will only take a file upload in pdf format. "gm convert foo.png foo.pdf" and you're done. [+] Really good for common "take an arbitrary user upload; grab info to crop it from your web app; crop and resize to appropriate dimensions" sorts of tasks.

meira 10y ago
The level of serendipity in HN is absurd. This is exactly what I was going to search for today. I'm going to use it local with Electron.
- teleclimber 10y ago
  > I'm going to use it local with Electron Interesting. I also have a use case for this though I wasn't sure how easy it would be to bundle something like GM with my Electron app. Have you any insights on this? I'm targeting both Win and Mac.
  - mikewhy 10y ago
    Windows can use the compiled binary, Mac will need a static build. If you're using asar packages in your final app make sure it extracts those binaries then do a child_process.spawn
    - teleclimber 10y ago
      Thank you for the info. I'm looking for a link with all the details but I haven't found one so far. If you have that I would appreciate you sharing. Thanks!
  - fenomas 10y ago
    If the app uses JS, I recently did a bunch of image processing using the node GM wrapper ('gm' on npm), and it was generally a huge pain in the ass. I found that I couldn't do the edits I needed in a single atomic call to gm, and I couldn't get the wrapper to pipe multiple calls together, so I wound up having to write and manage temp files, and it was all pretty hairy. I later found a project called 'jimp', which is more or less similar to gm but written in pure JS, and which made it easy to chain calls together into nontrivial edits. Since it has no native dependencies it would probably be easier to bundle with electron as well. Presumably it's slower than gm of course (in my case that didn't matter, so I never looked back).
    - meira 10y ago
      Now this is real serendipity! I was guessing that I'd have a difficult time with this integration. Jumping straight to your recommendation.
      - fenomas 10y ago
        Well, YMMV but for me what took me several days with the gm wrapper, I had working in about 15 minutes with jimp. Now that I think about it, I recall that the gm wrapper had a very strange API, that greatly obscured what parameters got passed to gm underneath, and figuring out how that worked was a big part of the headache. With jimp, everything basically did what it said on the tin. I remember wondering why such a useful library wasn't more famous.
      - teleclimber 10y ago
        Jimp is something I had come across as well and played with a bit. FYI I wrote in my notes "dependency on jpeg-js is dog slow." I recommend some perf tests for your use case before jumping too far in.
        seandougall 10y ago
        I (and a number of other folks) have also run into an enormous memory leak, apparently originating in jpeg-js, when using jimp. On the order of "loading a 3MB file leaks 200MB of memory", which makes it unusable if you're at all RAM-constrained. Shame, because it has a lot of promise otherwise.
- olalonde 10y ago
  And I was just debugging a "pixel cache allocation fail" bug with GM: https://github.com/aheckmann/gm/issues/598 https://github.com/aheckmann/gm/issues/598.
pavlov 10y ago
Personally I find that Apple Preview has always been good enough for these tasks. It can also do a surprising amount of PDF editing: for example, you can draw annotations on pages (for highlights and signatures), and selectively combine pages from multiple PDF documents into one by simple copy & paste.
- bootload 10y ago
  "Personally I find that Apple Preview has always been good enough for these tasks." OSS is a good choice because not every developer works on a Mac and with binary apps. You can hack quite a few quick/dirty tools to deal with repetitive tasks. Having said that Apple Preview is a pretty useful tool.
- pschastain 10y ago
  Except Preview 8 (post-Mavericks) sucks. Annotations of any kind always appear in the center of the page and have to be manually moved into position, it doesn't easily recognize selections, combined documents don't always retain changes, I could go on. I have a Mavericks VM just for Preview 7. [edit] grammar
revelation 10y ago
The issue with taking arbitrary user uploads and piping them directly into a program that happily accepts and interprets arbitrary data is security. They claim support for over 200 formats, some of which surely have exciting scripting functionality that no one ever uses: http://www.imagemagick.org/script/formats.php http://www.imagemagick.org/script/formats.php And there have been recent code execution vulnerability reports: https://imagetragick.com/ https://imagetragick.com/ Now libpng (e.g.) has issues as well but if at all possible I would try to restrict the input format and process only with software for the specific format.
amelius 10y ago
> HN would probably mostly be interested in it for server-side applications This image processing system would be very useful to have on the client. (Why can't we still use arbitrary stuff on the client that was written for the server?)
- Spivak 10y ago
  You can if you're shipping a native app. But if you're targeting the web you're limited by tools written in JS or what you can transcribe to JS. asm.js and WebAssembly could very well change this in a few years.
  - bobajeff 10y ago
    To further clarify SIMD.js (along WebAssembly's SIMD.js hooks) will change this. Right now you can still port GraphicsMagick to emscripten you'd just have to accept a performance hit on the parts that rely on SIMD.
deleted 10y ago
[deleted]

bsenftner 10y ago

I was confused for a moment: Imagemagick and graphicsmagick are different, http://blog.trackduck.com/2014/11/12/imagemagick-vs-graphicsmagick-one-better-process-images-website/ http://blog.trackduck.com/2014/11/12/imagemagick-vs-graphics...

ajross 10y ago
GM is a fork. It's actually one of the more persistent and notable forks in the open source community. Both projects continue development and have no interest in reconciliation.
- tgb 10y ago
  The last I heard about Image Magick was that it was out-dated and people were suggesting using other tools (though I don't remember what). Can anyone comment on that? This might have been only for a few specific tasks, not for Image Magick as a whole.
  - faitswulff 10y ago
    I don't know about now, but I've encountered significant issues with it every time I've attempted to use it in the last three years. To be fair, I was attempting to use it through wrappers - I had a particularly bad run trying to get around issues with both rmagick and minimagick in ruby, and later on a different project with wand in python. Though, with wand, the issues I encountered there were mostly with getting a buildpack working for it on Heroku. Regardless, whether it's my incompetence or imagemagick itself, I don't have very good associations with the library.
  - Washuu 10y ago
    GraphicsMagick fixed all the memory and CPU problems that ImageMagick was giving us in a large wiki farm production environment. That alone is a reason to switch to it.
    - pentium10 10y ago

mrcactu5 10y ago

CGAL https://news.ycombinator.com/item?id=12793220 https://news.ycombinator.com/item?id=12793220 (from a few days ago). Options!

pavlov 10y ago
But GraphicsMagick and CGAL are nothing alike?

jasode 10y ago

Fyi... I last looked at both GraphicsMagick and ImageMagick in 2013 when I was writing a custom batch image converter. Because GraphicsMagick had a reputation for better performance[1] I initially wrote my wrapper tool around GraphicsMagick. However, I encountered bugs with Photoshop PSD files (e.g. not properly reading multi-layer PSD files.[2]) That's when I found out that ImageMagick tracks the PSD file format more reliably than GraphicsMagick. Yes, GM later fixed that particular bug in 2014 but I had already revised my tool to use IM. Lesson learned: if processing PSD files is critical to your project, you'll want to favor ImageMagick over GraphicsMagick. The author of GM admits the parser of PSD is not the best so it's something to keep in mind. [1]Flickr and Etsy used GM on their backends: http://www.slideshare.net/jallspaw/operational-efficiency-hacks-web20-expo2009/33-Image_Processing_2004_Flickr_was http://www.slideshare.net/jallspaw/operational-efficiency-ha... http://www.web2expo.com/webexsf2009/public/schedule/detail/8580 http://www.web2expo.com/webexsf2009/public/schedule/detail/8... [2]https://sourceforge.net/p/graphicsmagick/bugs/242/ https://sourceforge.net/p/graphicsmagick/bugs/242/ [3]also some bonus links on why PSD is very difficult to parse correctly: https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m?utm_content=bufferae738#L108 https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m?... http://blogs.adobe.com/jnack/2009/05/some_thoughts_about_the_psd_format.html http://blogs.adobe.com/jnack/2009/05/some_thoughts_about_the...

olalonde 10y ago
PSD support is now disabled by default in GM apparently: https://github.com/Homebrew/homebrew-core/issues/6072#issuecomment-255293094 https://github.com/Homebrew/homebrew-core/issues/6072#issuec...

kjell 10y ago

vips is another great industrial-grade image processing library https://github.com/jcupitt/libvips https://github.com/jcupitt/libvips http://www.vips.ecs.soton.ac.uk/index.php?title=Libvips http://www.vips.ecs.soton.ac.uk/index.php?title=Libvips

pbowyer 10y ago
And let's not forget NetPBM either, which (when I was doing this a decade ago) was faster and more memory efficient than (Image|Graphics)Magick.

dahart 10y ago

If you need to process huge images, GraphicsMagick does some things that ImageMagick can't do. A few years ago when I was making poster sized art prints, I would routinely process 60k^2 pixel (4 gigapixel) images. Just a resize with a Gaussian filter would thrash and never finish overnight in ImageMagick, and only took a minute or two in GraphicsMagick. I'm not in love with the CLI for either, but GM's is better and I like that it's all collected under one roof rather than a whole bunch of separate commands. I do wish the projects could merge back into one and have the best of both. I also wish they were better at color handling, resizing needs to be done at a higher bit depth, better support for converting between modern color spaces, etc. I love sips on my mac because it's fast, but I'm not sure if anything's a better cross-platform choice for basic batch operations on images, or for the backend of a web service, than GraphicsMagick.

rahimnathwani 10y ago
I'm curious, as I want to make some poster-sized art prints soon: 1) What resolution were the source files? 2) What upscaling options did you use? 3) What multiple of the original file size did you go up to? Were you limited by the quality of the upscaling, or did you just decide on the size of print and dpi, and multiply?
- dahart 10y ago
  Oh, I was downscaling, mainly for filtering/anti-aliasing. My source files were the 60k^2 images, and the output was more like 20k^2... I shoot for at least 300 dpi and I was printing 60-inch prints. I had enough trouble downscaling such large images that I ended up writing my own downscaling in my renderer so that I wouldn't have to use GraphicsMagick or anything else, but of everything I ever tried, GraphicsMagick was the best at handling enormous images. FWIW, I've never been happy with any upscaling. If you do have the option to render at your target res and not use upscaling, that's ideal. If you must upscale, it's hard to find anything that can do more that 2-3x without noticeable artifacts or visible blurriness.
  - rahimnathwani 10y ago

teilo 10y ago

GraphicsMagick is great, but it's not a replacement for ImageMagick. While it offers a few features not found in IM, it lags far behind on the more advanced stuff, such as image recognition/machine vision, and in general does not track the main project any longer.