11 ms·

So if you were to write an OCR software today, you would just feed a neural network with a whole raw phone image shot of a floppy paper? Or would you try to fla

by lixtra 6y ago

So if you were to write an OCR software today, you would just feed a neural network with a whole raw phone image shot of a floppy paper? Or would you try to flatten the image first, then extract words and letters and then synthesize that back to a bigger model (also using NN) that finally produces your text?

PeterisP 6y ago
This seems a quite orthogonal issue to what I was talking about - your comment seems to be about the separation of pipeline in subtasks, while I was talking about methods used for particular subtasks; I would not consider "try to flatten the image first" as part of feature engineering, I would consider it as a preprocessing task that might be done as an engineered feature, but also might be done as a machine-learned subtask, or skipped entirely, or integrated in a more general transformation that's believed to inherently correct for non-flat images. I don't work on OCR much (and when I did, it was for book digitization which doesn't have these specific challenges), so I don't have an opinion on what's the state of art for a task like you describe (I'm imagining analysis of receipts or something like that), however, across many domains of ML (incliding but not limited to computer vision) we are seeing the advantages of end-to-end optimization. So, for example, the image preprocessing we would like to allow for input of raw phone camera images includes correction for lighting, angle, and crumpledness of paper. Obviously, I agree that these things should be done, but I do not necessarily agree that they must be done as separate engineered features. I don't have an opinion on what's the best option "today in production" for OCR - it's plausible that the engineered feature way is still the best at the moment, but if we're looking at where the field is moving to, then I'd argue that there is a strong tendencytowards (a) using numerically optimizable methods for these corrections as opposed to hand-selected heuristics; (b) optimizing these corrections for a final OCR result as opposed to treating it as a separate task with separate metrics to optimize; and eventually (c) integrating them in an end-to-end system instead of a clearly separable stage of "correcting for X". I'm not certain where the state of art for noisy OCR is today on this, but it's a one-way direction; The key point of my comment above is that once (if!) we get these things to work I would not expect to go backwards to specific handcrafted features ever. It's plausible that some tasks are better treated as separate and can't be integrated well (perhaps the selection of text segments in your OCR example is like that), but for the features that we already have managed to successfully integrate (which was the topic discussed in the grandparent comment), IMHO there is no reason to ever go back. Perhaps a relevant example of a field that has undergone this transformation is machine translation - where just a few years ago production pipelines included literally hundreds of separate subsystems for niche subtasks conceptually similar to those you mention regarding OCR, the shift to neural ML was accompanied by making these subsystems redundant as doing the same thing in an integrated end-to-end optimizable way gets better results and is simpler to implement, maintain and debug due to a more unified architecture. Similar trends have also happened for face recognition and for speech recognition - I would presume that OCR is structurally similar enough to see the same fate if it hasn't already.

by lixtra 6y ago

PeterisP 6y ago

This seems a quite orthogonal issue to what I was talking about - your comment seems to be about the separation of pipeline in subtasks, while I was talking about methods used for particular subtasks; I would not consider "try to flatten the image first" as part of feature engineering, I would consider it as a preprocessing task that might be done as an engineered feature, but also might be done as a machine-learned subtask, or skipped entirely, or integrated in a more general transformation that's believed to inherently correct for non-flat images. I don't work on OCR much (and when I did, it was for book digitization which doesn't have these specific challenges), so I don't have an opinion on what's the state of art for a task like you describe (I'm imagining analysis of receipts or something like that), however, across many domains of ML (incliding but not limited to computer vision) we are seeing the advantages of end-to-end optimization. So, for example, the image preprocessing we would like to allow for input of raw phone camera images includes correction for lighting, angle, and crumpledness of paper. Obviously, I agree that these things should be done, but I do not necessarily agree that they must be done as separate engineered features. I don't have an opinion on what's the best option "today in production" for OCR - it's plausible that the engineered feature way is still the best at the moment, but if we're looking at where the field is moving to, then I'd argue that there is a strong tendencytowards (a) using numerically optimizable methods for these corrections as opposed to hand-selected heuristics; (b) optimizing these corrections for a final OCR result as opposed to treating it as a separate task with separate metrics to optimize; and eventually (c) integrating them in an end-to-end system instead of a clearly separable stage of "correcting for X". I'm not certain where the state of art for noisy OCR is today on this, but it's a one-way direction; The key point of my comment above is that once (if!) we get these things to work I would not expect to go backwards to specific handcrafted features ever. It's plausible that some tasks are better treated as separate and can't be integrated well (perhaps the selection of text segments in your OCR example is like that), but for the features that we already have managed to successfully integrate (which was the topic discussed in the grandparent comment), IMHO there is no reason to ever go back. Perhaps a relevant example of a field that has undergone this transformation is machine translation - where just a few years ago production pipelines included literally hundreds of separate subsystems for niche subtasks conceptually similar to those you mention regarding OCR, the shift to neural ML was accompanied by making these subsystems redundant as doing the same thing in an integrated end-to-end optimizable way gets better results and is simpler to implement, maintain and debug due to a more unified architecture. Similar trends have also happened for face recognition and for speech recognition - I would presume that OCR is structurally similar enough to see the same fate if it hasn't already.