4 ms·

MNIST Handwritten digits classification using Keras, part 1

47 points by Aryal007 9y ago

appleflaxen 9y ago
With all the progress in machine learning it seems like there should be an amazing OCR tool that works out of the box for structured documents. Does it exist? I've used tesseract and its relatives, but they seem to have a hard time with any document that's not a single column. The difference between what they can achieve (which to be fair is amazing) with what I expected based on all the ML demos that only does the first 10% (numbers only, no structure), but does it in a 30-minute demo, is big. Things like affine transforms (scaling, rotation) and decorations like bold, underline, and weird fonts create even more problems. Why isn't there a docker container with an AWS lambda function in it that takes any image format I upload (pdf, png, jpg as the most critical) and returns a UTF string of its content? My god, I'm spoiled by technology.
- zzleeper 9y ago
  Agreed. What I do for tables is use the OCR as a step 1, where I just extract the coordinates of each textbox. Then, use one of several methods to recognize columns (Hough transform, or we search for empty areas). Then, do some clustering to recognize lines. The problem is that there's a million of edge cases that you need to handle (split boxes that span multiple columns, in the case of tables recognize cells that span multiple columns or rows, etc.). There should be a solution for all these edge cases, but it gets tricky quickly.
  - GCU-Empiricist 9y ago
    Having worked on some of these questions, then there comes the question of logical flow, how do you order that inset square of text at the bigger font, is it a picture caption or a header text?
- shepardrtc 9y ago
  Digit/letter recognition is near perfect with the latest algorithms and techniques, but organizing the output from scanning a document is still pretty hard.
- inoop 9y ago
  Have a look at Rekognition: https://aws.amazon.com/rekognition https://aws.amazon.com/rekognition
- Isamu 9y ago
  Google Cloud Vision API does this, and it's pretty good. https://cloud.google.com/vision/ https://cloud.google.com/vision/ https://cloud.google.com/vision/docs/ocr https://cloud.google.com/vision/docs/ocr https://cloud.google.com/vision/docs/supported-files https://cloud.google.com/vision/docs/supported-files You have to sign up for a developer account. If you are only doing low volume requests (up to 1000 a month) it is free.
  - abhishekjha 9y ago
    I have been using it for a while to extract text from invoices/pdfs. The troble that I have to go to math the corresponding header to its value in each row is exhausting to say the least. Once Si het the text, I have to do an innumerable amount of hacks to get valid mappings. I wish we had a tool for that. ;)
    - Isamu 9y ago
      Do you use the bounding box information? That seems like it could help you with table formats.
      - abhishekjha 9y ago
        Not quiet though curious. Where can I read more on Bounding Box information?
ssivark 9y ago
(Apologies in advance if this sounds snarky, don't mean to be) There are hundreds of posts on using Keras/TensorFlow/PyTorch/etc to do MNIST classification, and many examples on Github. All these resources are very easy to find by Googling. This post doesn't seem to have anything different to add to the conversation. So, why do such articles continue to be written, and why do they still get upvoted on HN? Is it that a lot of people want to learn these things, but haven't gotten a chance to, so they upvote with the hope of staying in touch with the topic? Is it FOMO? One might be forgiven for considering this spam.
- wenc 9y ago
  I think this is just the nature of any open forum where everyone has an equal vote (democracy). In any large-enough population, there will invariably be a significant non-cognoscenti segment who will upvote this type of content. And who knows, for this segment, this content may be new despite repeated iterations elsewhere. It seems inefficient, but I'd rather have this type of system than systems that weight upvotes according to karma. This way, at least you avoid polarization by high karma users. The occasional low-quality content is not such a bad price to pay for a self-renewing system. After the hype dies down, there will naturally be fewer MNIST articles. :)
- dang 9y ago
  Moderators routinely downweight tutorial articles for that reason. It took a surprisingly long time to realize it, but tutorials aren't a good fit for Hacker News. HN is for articles that gratify intellectual curiosity. Most tutorials don't do that; it doesn't fit with being a recipe for a task. If a recipe were to dive deeply into how and why its steps work, how they got that way, and compared other recipes that have (or don't have) similar steps, it would gratify intellectual curiosity better. But then it would be a worse recipe. It would also be more work to write. The worst kind of tutorial from an HN point of view is the kind filled with arbitrary details that apply only to a particular program or product, like "Now put a file named foo in a directory called baz". But that is almost all of them. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
stared 9y ago
“Many good ideas will not work well on MNIST (e.g. batch norm). Inversely[,] many bad ideas may work on MNIST and no[t] transfer to real [computer vision]” – a tweet by François Chollet (creator of Keras) So - please, anything harder (or rather: more relevant to deep learning). At least images in CIFAR-10: https://blog.deepsense.ai/deep-learning-hands-on-image-classification/ https://blog.deepsense.ai/deep-learning-hands-on-image-class...
- julsimon 9y ago
  Or Fashion-MNIST https://github.com/zalandoresearch/fashion-mnist https://github.com/zalandoresearch/fashion-mnist :)

47 points by Aryal007 9y ago

With all the progress in machine learning it seems like there should be an amazing OCR tool that works out of the box for structured documents. Does it exist? I've used tesseract and its relatives, but they seem to have a hard time with any document that's not a single column. The difference between what they can achieve (which to be fair is amazing) with what I expected based on all the ML demos that only does the first 10% (numbers only, no structure), but does it in a 30-minute demo, is big. Things like affine transforms (scaling, rotation) and decorations like bold, underline, and weird fonts create even more problems. Why isn't there a docker container with an AWS lambda function in it that takes any image format I upload (pdf, png, jpg as the most critical) and returns a UTF string of its content? My god, I'm spoiled by technology.

zzleeper 9y ago
Agreed. What I do for tables is use the OCR as a step 1, where I just extract the coordinates of each textbox. Then, use one of several methods to recognize columns (Hough transform, or we search for empty areas). Then, do some clustering to recognize lines. The problem is that there's a million of edge cases that you need to handle (split boxes that span multiple columns, in the case of tables recognize cells that span multiple columns or rows, etc.). There should be a solution for all these edge cases, but it gets tricky quickly.
- GCU-Empiricist 9y ago
  Having worked on some of these questions, then there comes the question of logical flow, how do you order that inset square of text at the bigger font, is it a picture caption or a header text?
shepardrtc 9y ago
Digit/letter recognition is near perfect with the latest algorithms and techniques, but organizing the output from scanning a document is still pretty hard.
inoop 9y ago
Have a look at Rekognition: https://aws.amazon.com/rekognition https://aws.amazon.com/rekognition
Isamu 9y ago
Google Cloud Vision API does this, and it's pretty good. https://cloud.google.com/vision/ https://cloud.google.com/vision/ https://cloud.google.com/vision/docs/ocr https://cloud.google.com/vision/docs/ocr https://cloud.google.com/vision/docs/supported-files https://cloud.google.com/vision/docs/supported-files You have to sign up for a developer account. If you are only doing low volume requests (up to 1000 a month) it is free.
- abhishekjha 9y ago
  I have been using it for a while to extract text from invoices/pdfs. The troble that I have to go to math the corresponding header to its value in each row is exhausting to say the least. Once Si het the text, I have to do an innumerable amount of hacks to get valid mappings. I wish we had a tool for that. ;)
  - Isamu 9y ago
    Do you use the bounding box information? That seems like it could help you with table formats.
    - abhishekjha 9y ago
      Not quiet though curious. Where can I read more on Bounding Box information?

ssivark 9y ago

(Apologies in advance if this sounds snarky, don't mean to be) There are hundreds of posts on using Keras/TensorFlow/PyTorch/etc to do MNIST classification, and many examples on Github. All these resources are very easy to find by Googling. This post doesn't seem to have anything different to add to the conversation. So, why do such articles continue to be written, and why do they still get upvoted on HN? Is it that a lot of people want to learn these things, but haven't gotten a chance to, so they upvote with the hope of staying in touch with the topic? Is it FOMO? One might be forgiven for considering this spam.

wenc 9y ago
I think this is just the nature of any open forum where everyone has an equal vote (democracy). In any large-enough population, there will invariably be a significant non-cognoscenti segment who will upvote this type of content. And who knows, for this segment, this content may be new despite repeated iterations elsewhere. It seems inefficient, but I'd rather have this type of system than systems that weight upvotes according to karma. This way, at least you avoid polarization by high karma users. The occasional low-quality content is not such a bad price to pay for a self-renewing system. After the hype dies down, there will naturally be fewer MNIST articles. :)
dang 9y ago
Moderators routinely downweight tutorial articles for that reason. It took a surprisingly long time to realize it, but tutorials aren't a good fit for Hacker News. HN is for articles that gratify intellectual curiosity. Most tutorials don't do that; it doesn't fit with being a recipe for a task. If a recipe were to dive deeply into how and why its steps work, how they got that way, and compared other recipes that have (or don't have) similar steps, it would gratify intellectual curiosity better. But then it would be a worse recipe. It would also be more work to write. The worst kind of tutorial from an HN point of view is the kind filled with arbitrary details that apply only to a particular program or product, like "Now put a file named foo in a directory called baz". But that is almost all of them. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

stared 9y ago

“Many good ideas will not work well on MNIST (e.g. batch norm). Inversely[,] many bad ideas may work on MNIST and no[t] transfer to real [computer vision]” – a tweet by François Chollet (creator of Keras) So - please, anything harder (or rather: more relevant to deep learning). At least images in CIFAR-10: https://blog.deepsense.ai/deep-learning-hands-on-image-classification/ https://blog.deepsense.ai/deep-learning-hands-on-image-class...

julsimon 9y ago
Or Fashion-MNIST https://github.com/zalandoresearch/fashion-mnist https://github.com/zalandoresearch/fashion-mnist :)