6 ms·

Based on the video it works amazingly well. I’ve tried finding a usable way for open source text recognition on desktop, and haven’t found anything even close t

Based on the video it works amazingly well. I’ve tried finding a usable way for open source text recognition on desktop, and haven’t found anything even close to this. Like, I've got scans of letters, machine-written, 300dpi, perfect black/white 1-bit depth, no marks or scratches, perfect quality — and OCRmyPDF (using tesseract) absolutely fails at this task, returning only bullshit. Even if I set the language correctly, or even set the wordlist to a manual transcription of the PDF. I also tried using OCR on screenshots, with the same miserable result. How does Apple’s Vision API do so much better at this kind of task? Is there some trick I'm missing? Like, the images I supply are of such high quality that you can literally just split into characters and search for the nearest match in a database of latin characters, and even that would return better results than tesseract.

asadlionpk 7y ago
In my limited comparison. Apple’s API performed way better than Tessaract. You can check your images using this app or their sample xcode project for vision, I think same API works on macOS too. I would also vouch for google’s ocr API but I opted for Apple’s for privacy reasons and as they were easier to implement.
- toomuchtodo 7y ago
  > but I opted for Apple’s for privacy reasons and as they were easier to implement. Thank you for respecting user privacy and making it a first class citizen.
mendeza 7y ago
I think you have to fiddle with the internal settings of the OCR package you are using to get good results. For tesseract and pytesseract, there is a whole article on improving the quality: https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuality https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuali... My guess is Apple trained on their own massive dataset and better architectures that make their systems better than off the shelf ocr. I am working in this area and one way I see good improvement is training an object detector to first detect words in an image, then you can pass that through tesseract/OCR software. Besides that, finetuning tesseract on data you want strong performance would be the next best alternative. Here is a cool article from dropbox how they engineering their OCR system: https://blogs.dropbox.com/tech/2017/04/creating-a-modern-ocr-pipeline-using-computer-vision-and-deep-learning/ https://blogs.dropbox.com/tech/2017/04/creating-a-modern-ocr... This link below is a really cool project that showcases what building your own OCR engine would look like! https://github.com/awslabs/handwritten-text-recognition-for-apache-mxnet https://github.com/awslabs/handwritten-text-recognition-for-...
- mendeza 7y ago
  Reading all these comments about others having bad experiences with Tesseract makes me think there is a market for a robust, better alternative to Tesseract.
avian 7y ago
If it makes you feel any better, I had a similarly terrible experience with Tesseract. I spent days studying the docs and I failed to find a setup where it would produce anything useful even with the most perfect input. I was also left with a feeling that I must have been missing some trick, because people on the web seem to be using it with good results.
- scarejunba 7y ago
  It requires a lot of tuning. When I first tried it out I rapidly abandoned it when I couldn't get useful stuff out of a screenshot of text I typed in.
skadamat 7y ago
You might like Screenotate by Omar Rizwan - https://screenotate.com/ https://screenotate.com/
lharries 7y ago
It might be worth trying this for desktop: https://github.com/the-paperless-project/paperless https://github.com/the-paperless-project/paperless It handles OCR, indexing and search
- Aardwolf 7y ago
  It appears to also use Tesseract
  - xnyan 7y ago
    The paperless project has some tweaks in the config that can get decent results out of the box. For general use with decent quality documents it will work pretty well right now versus spending hours tweaking Tesseract.
arendtio 7y ago
I use Tesseract to extract text from my scanned documents in order to be able to grep through them. What surprised me, was that the lower quality (resolution) scans had actually much better OCR results. So it seems Tesseract works best with some specific font-size-DPI combination. This is the script I wrote for that purpose (maybe it helps): https://nopaste.xyz/?ea1e6242a659ce5c#vAMyPpajo0OIeCGQe9WkdqGphSFIVF88sZasxL0kJVk= https://nopaste.xyz/?ea1e6242a659ce5c#vAMyPpajo0OIeCGQe9Wkdq... The results aren't perfect, but good enough for my purpose.
shiredude95 7y ago
I had a similar experience working with OCR for a similar project as OP ( https://www.askgoose.com https://www.askgoose.com ). What we realized was tesseract's training data was significantly different to what we were trying to transcribe (screenshots of conversations). The sparseness of text leads to bad transcriptions. One workaround is to do some image processing before hand so you can break the image down into data that looks similar to the train data used for tesseract.
poxrud 7y ago
I've had success with first using imagemagick to cleanup the image, do some thresholding and then using the latest version (4.1) of tesseract. With tesseract you can also try to experiment with different --psm values for better results.
slovenlyrobot 7y ago
You get what you pay for with Tesseract, don't judge OCR performance based on it. Would highly recommend checking out FineReader if you're on a supported platform. You can feed it crumpled beer receipts and the results are still decent
bhanhfo 7y ago
Tesseract needs text on a white background to work well out of the box. For "text on images" try for example https://ocr.space/ocrapi https://ocr.space/ocrapi
saurik 7y ago
I don't know if this is how it works or not, but it is worth noting that searching for text in images is a much easier problem than transcribing images, and if you do the latter as "step 1" you destroy your ability to do the former: if you search for some piece of text you want to find all images that maybe if you squint a bit could be that piece of text, which allows you to deal with cases where your recognition is forced to make a bad guess that does the moral equivalent of rebracketing symbols that is difficult to work around in a later text-only search pass. If you can at all avoid it, always avoid transforming your input information until you know your query, lest you destroy information (this is what always made Microsoft Journal so epic for me, though I should be clear that I haven't used it in 15 years so for all I know they forgot this).
bhl 7y ago
Is there a way to port Apple's Vision API to javascript for a webapp, or use their models without an Apple device like Apple Maps on Duckduckgo?
- busymom0 7y ago
  I don't think that's possible. The Vision VNRecognizeTextRequest is only available for: iOS 13.0+ macOS 10.15+ Mac Catalyst 13.0+ tvOS 13.0+ https://developer.apple.com/documentation/vision/vnrecognizetextrequest https://developer.apple.com/documentation/vision/vnrecognize...

asadlionpk 7y ago

In my limited comparison. Apple’s API performed way better than Tessaract. You can check your images using this app or their sample xcode project for vision, I think same API works on macOS too. I would also vouch for google’s ocr API but I opted for Apple’s for privacy reasons and as they were easier to implement.

toomuchtodo 7y ago
> but I opted for Apple’s for privacy reasons and as they were easier to implement. Thank you for respecting user privacy and making it a first class citizen.

mendeza 7y ago

I think you have to fiddle with the internal settings of the OCR package you are using to get good results. For tesseract and pytesseract, there is a whole article on improving the quality: https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuality https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuali... My guess is Apple trained on their own massive dataset and better architectures that make their systems better than off the shelf ocr. I am working in this area and one way I see good improvement is training an object detector to first detect words in an image, then you can pass that through tesseract/OCR software. Besides that, finetuning tesseract on data you want strong performance would be the next best alternative. Here is a cool article from dropbox how they engineering their OCR system: https://blogs.dropbox.com/tech/2017/04/creating-a-modern-ocr-pipeline-using-computer-vision-and-deep-learning/ https://blogs.dropbox.com/tech/2017/04/creating-a-modern-ocr... This link below is a really cool project that showcases what building your own OCR engine would look like! https://github.com/awslabs/handwritten-text-recognition-for-apache-mxnet https://github.com/awslabs/handwritten-text-recognition-for-...

mendeza 7y ago
Reading all these comments about others having bad experiences with Tesseract makes me think there is a market for a robust, better alternative to Tesseract.

avian 7y ago

If it makes you feel any better, I had a similarly terrible experience with Tesseract. I spent days studying the docs and I failed to find a setup where it would produce anything useful even with the most perfect input. I was also left with a feeling that I must have been missing some trick, because people on the web seem to be using it with good results.

scarejunba 7y ago
It requires a lot of tuning. When I first tried it out I rapidly abandoned it when I couldn't get useful stuff out of a screenshot of text I typed in.

skadamat 7y ago

You might like Screenotate by Omar Rizwan - https://screenotate.com/ https://screenotate.com/

lharries 7y ago

It might be worth trying this for desktop: https://github.com/the-paperless-project/paperless https://github.com/the-paperless-project/paperless It handles OCR, indexing and search

Aardwolf 7y ago
It appears to also use Tesseract
- xnyan 7y ago
  The paperless project has some tweaks in the config that can get decent results out of the box. For general use with decent quality documents it will work pretty well right now versus spending hours tweaking Tesseract.

arendtio 7y ago

I use Tesseract to extract text from my scanned documents in order to be able to grep through them. What surprised me, was that the lower quality (resolution) scans had actually much better OCR results. So it seems Tesseract works best with some specific font-size-DPI combination. This is the script I wrote for that purpose (maybe it helps): https://nopaste.xyz/?ea1e6242a659ce5c#vAMyPpajo0OIeCGQe9WkdqGphSFIVF88sZasxL0kJVk= https://nopaste.xyz/?ea1e6242a659ce5c#vAMyPpajo0OIeCGQe9Wkdq... The results aren't perfect, but good enough for my purpose.

shiredude95 7y ago

I had a similar experience working with OCR for a similar project as OP ( https://www.askgoose.com https://www.askgoose.com ). What we realized was tesseract's training data was significantly different to what we were trying to transcribe (screenshots of conversations). The sparseness of text leads to bad transcriptions. One workaround is to do some image processing before hand so you can break the image down into data that looks similar to the train data used for tesseract.

poxrud 7y ago

I've had success with first using imagemagick to cleanup the image, do some thresholding and then using the latest version (4.1) of tesseract. With tesseract you can also try to experiment with different --psm values for better results.

slovenlyrobot 7y ago

You get what you pay for with Tesseract, don't judge OCR performance based on it. Would highly recommend checking out FineReader if you're on a supported platform. You can feed it crumpled beer receipts and the results are still decent

bhanhfo 7y ago

Tesseract needs text on a white background to work well out of the box. For "text on images" try for example https://ocr.space/ocrapi https://ocr.space/ocrapi

saurik 7y ago

I don't know if this is how it works or not, but it is worth noting that searching for text in images is a much easier problem than transcribing images, and if you do the latter as "step 1" you destroy your ability to do the former: if you search for some piece of text you want to find all images that maybe if you squint a bit could be that piece of text, which allows you to deal with cases where your recognition is forced to make a bad guess that does the moral equivalent of rebracketing symbols that is difficult to work around in a later text-only search pass. If you can at all avoid it, always avoid transforming your input information until you know your query, lest you destroy information (this is what always made Microsoft Journal so epic for me, though I should be clear that I haven't used it in 15 years so for all I know they forgot this).

bhl 7y ago

Is there a way to port Apple's Vision API to javascript for a webapp, or use their models without an Apple device like Apple Maps on Duckduckgo?

busymom0 7y ago
I don't think that's possible. The Vision VNRecognizeTextRequest is only available for: iOS 13.0+ macOS 10.15+ Mac Catalyst 13.0+ tvOS 13.0+ https://developer.apple.com/documentation/vision/vnrecognizetextrequest https://developer.apple.com/documentation/vision/vnrecognize...