6 ms·

Frankly, I'm not very impressed. This problem is a very very well known problem in Image Processing. Usually people call it background subtraction or the more g

by susi22 13y ago

Frankly, I'm not very impressed. This problem is a very very well known problem in Image Processing. Usually people call it background subtraction or the more general form is image segmentation. There is many really well working algorithms in this field. A google search with the major research conferences (ICCV, ICIP, SIGGRAPH etc) will give you the latest and the greates of these algorithms. You'll also find good (image segmentation) work if you limit the search for csail.mit.edu. If you want your uploader to help you, you can also go for one of the supervised image segmentation algorithms. Otherwise you'll need an unsupervised algorithm. Given that the authors also want to detect what is in the image, this might be helpful for them: http://people.csail.mit.edu/mrub/papers/ObjectDiscovery-cvpr13.pdf http://people.csail.mit.edu/mrub/papers/ObjectDiscovery-cvpr... This guy is also doing some great work in that field: http://www.engr.uconn.edu/~cmli/ http://www.engr.uconn.edu/~cmli/

sjtgraham 13y ago
The corpus of product images at Lyst seems to be quite diverse. How many of those algorithms are good at very generalised segmentation, i.e. equally good at segmenting boat shoes on decking as they are say a tank in a field? What are your favourite papers on the subject?
- amelim 13y ago
  One of the coolest approaches I've seen does some cool inference on fully connected Conditional Random Fields via high-dimensional filtering. Amazing results. http://vladlen.info/publications/efficient-inference-in-fully-connected-crfs-with-gaussian-edge-potentials/ http://vladlen.info/publications/efficient-inference-in-full...
nobodysfool 13y ago
The negation step seems superfluous. The edge detection filter can't detect edges in a regular photo but can in a negative?
- sp332 13y ago
  The reason given is that the algorithm (Sobel) looks for light-to-dark transitions. There are two such transitions in the positive image: one from the background to the edge, and another from the edge into the object. Negating the object leaves only one light-to-dark gradient on the edge of the object.
  - mistercow 13y ago
    Yeah, but that reason is nonsense. The Sobel operator uses convolution to approximate the magnitude of the gradient. It doesn't "look for" anything. And since it's the magnitude of the gradient that Sobel finds, Sobel(Invert(img)) is mathematically equivalent to Sobel(img). The invert step is essentially an expensive noop.
mistercow 13y ago
My thinking exactly. The really crazy thing is that they seem to be reinventing the wheel so that they can lean on the optimized code a graphics library provides. I think that's flawed thinking to begin with. In my experience with graphics programming, a reasonably optimized direct implementation tends to beat the hell out of a nine-step filter chain, no matter how good your graphics library. Even if they used the same algorithm, a direct implementation could condense steps 2, 3, and 4 into a single convolution [edit: whoops, no you cant; Sobel has a non-convolution step]. But more importantly, they've tied their hands by limiting themselves to a tiny set of operations. Combing the computer vision literature would have been a better use of time than trying to chain filters together. In fact, some background subtraction algorithms effectively do as an intermediate step what Lyst wanted as an end result.
- ddt83 13y ago
  Background subtraction operates on video streams, not still images, so this is incorrect.
  - mistercow 13y ago
    "Image segmentation" is usually the term for doing this with still images, and OpenCV provides a couple of functions for it. They're not perfect, but they're probably more effective than what this article describes.
    - ddt83 13y ago
      I agree. I was pointing out that background subtraction is not the same task.
  - michael_h 13y ago
    okay, this is image segmentation that attempts to estimate and remove the 'background' segments using some domain specific assumptions. This is generally referred to, in common language, as 'background subtraction'.
    - 13y ago
msantos 13y ago
I work in the same market as Lyst, and can say for a fact that the images variations we get from clients even from renowned fashion houses is huge. By variations I mean compression, quality, subject's clarity and the list just goes on... A good algorithm would surely solve part of the problem, but given the imagery non-uniform patterns a great deal of resources (engineering skills + computing power) would likely be required. Given Lyst's recent VC rounds maybe they can afford that, but I surely cannot. So I went for the poor-man option: https://gist.github.com/mvsantos/5554663 https://gist.github.com/mvsantos/5554663 (I'm definitely bookmarking the link you suggested and also the one suggested by sjtgraham.)
- troels 13y ago
  Funny - I'm in the same domain and I have found that product images are generally high quality and somewhat uniform. You can usually find the product placed front and centre and with few distractions (Most images have either white or light grey background).
  - msantos 13y ago
    Renowned fashion houses overall do a great job, except when they add gradient or shadows to mask stuff. But the bulk of the problem, in my case, comes from large department stores when they blend furniture or accessories in fashion photoshoots making them complex and distracting from the main image subject.
ddt83 13y ago
I don't think background subtraction means what you think it does. Background subtraction refers to the subtraction of neighboring frames of a video sequence in order to find moving objects in a video sequence. So, yes segmentation is a more general and difficult problem than background subtraction, but it is in no way relevant to the task described in the article.
- mistercow 13y ago
  Image segmentation is exactly what they are trying to do. You can quibble about the exact definition of "background subtraction", but it doesn't change the fact that they are reinventing the wheel over a solved problem.
  - ddt83 13y ago
    Right, image segmentation is the correct terminology. Background subtraction is incorrect. I don't think it's quibbling to point that out.
hnriot 13y ago
at least get your facts and terminology straight before saying things like this.
deleted 13y ago
[deleted]

by susi22 13y ago

sjtgraham 13y ago

The corpus of product images at Lyst seems to be quite diverse. How many of those algorithms are good at very generalised segmentation, i.e. equally good at segmenting boat shoes on decking as they are say a tank in a field? What are your favourite papers on the subject?

amelim 13y ago
One of the coolest approaches I've seen does some cool inference on fully connected Conditional Random Fields via high-dimensional filtering. Amazing results. http://vladlen.info/publications/efficient-inference-in-fully-connected-crfs-with-gaussian-edge-potentials/ http://vladlen.info/publications/efficient-inference-in-full...

nobodysfool 13y ago

The negation step seems superfluous. The edge detection filter can't detect edges in a regular photo but can in a negative?

sp332 13y ago
The reason given is that the algorithm (Sobel) looks for light-to-dark transitions. There are two such transitions in the positive image: one from the background to the edge, and another from the edge into the object. Negating the object leaves only one light-to-dark gradient on the edge of the object.
- mistercow 13y ago
  Yeah, but that reason is nonsense. The Sobel operator uses convolution to approximate the magnitude of the gradient. It doesn't "look for" anything. And since it's the magnitude of the gradient that Sobel finds, Sobel(Invert(img)) is mathematically equivalent to Sobel(img). The invert step is essentially an expensive noop.

mistercow 13y ago

My thinking exactly. The really crazy thing is that they seem to be reinventing the wheel so that they can lean on the optimized code a graphics library provides. I think that's flawed thinking to begin with. In my experience with graphics programming, a reasonably optimized direct implementation tends to beat the hell out of a nine-step filter chain, no matter how good your graphics library. Even if they used the same algorithm, a direct implementation could condense steps 2, 3, and 4 into a single convolution [edit: whoops, no you cant; Sobel has a non-convolution step]. But more importantly, they've tied their hands by limiting themselves to a tiny set of operations. Combing the computer vision literature would have been a better use of time than trying to chain filters together. In fact, some background subtraction algorithms effectively do as an intermediate step what Lyst wanted as an end result.

ddt83 13y ago
Background subtraction operates on video streams, not still images, so this is incorrect.
- mistercow 13y ago
  "Image segmentation" is usually the term for doing this with still images, and OpenCV provides a couple of functions for it. They're not perfect, but they're probably more effective than what this article describes.
  - ddt83 13y ago
    I agree. I was pointing out that background subtraction is not the same task.
- michael_h 13y ago
  okay, this is image segmentation that attempts to estimate and remove the 'background' segments using some domain specific assumptions. This is generally referred to, in common language, as 'background subtraction'.
  - 13y ago

msantos 13y ago

I work in the same market as Lyst, and can say for a fact that the images variations we get from clients even from renowned fashion houses is huge. By variations I mean compression, quality, subject's clarity and the list just goes on... A good algorithm would surely solve part of the problem, but given the imagery non-uniform patterns a great deal of resources (engineering skills + computing power) would likely be required. Given Lyst's recent VC rounds maybe they can afford that, but I surely cannot. So I went for the poor-man option: https://gist.github.com/mvsantos/5554663 https://gist.github.com/mvsantos/5554663 (I'm definitely bookmarking the link you suggested and also the one suggested by sjtgraham.)

troels 13y ago
Funny - I'm in the same domain and I have found that product images are generally high quality and somewhat uniform. You can usually find the product placed front and centre and with few distractions (Most images have either white or light grey background).
- msantos 13y ago
  Renowned fashion houses overall do a great job, except when they add gradient or shadows to mask stuff. But the bulk of the problem, in my case, comes from large department stores when they blend furniture or accessories in fashion photoshoots making them complex and distracting from the main image subject.

ddt83 13y ago

I don't think background subtraction means what you think it does. Background subtraction refers to the subtraction of neighboring frames of a video sequence in order to find moving objects in a video sequence. So, yes segmentation is a more general and difficult problem than background subtraction, but it is in no way relevant to the task described in the article.

mistercow 13y ago
Image segmentation is exactly what they are trying to do. You can quibble about the exact definition of "background subtraction", but it doesn't change the fact that they are reinventing the wheel over a solved problem.
- ddt83 13y ago
  Right, image segmentation is the correct terminology. Background subtraction is incorrect. I don't think it's quibbling to point that out.

hnriot 13y ago

at least get your facts and terminology straight before saying things like this.

deleted 13y ago

[deleted]