8 ms·
In case it's not clear, Tesseract is developed by Google since 2006, having been started at HP in 1985 and open-sourced by HP in 2005. [1] As far as I know, it
by crazygringo 7y ago
In case it's not clear, Tesseract is developed by Google since 2006, having been started at HP in 1985 and open-sourced by HP in 2005. [1]
As far as I know, it powers all OCR at Google (e.g. in Keep, Docs, etc.).
This (Tesseract.js) is a WASM port of the project by a separate group of people.
I investigated using this port a couple years ago, but as you can see from the demo, it's fairly slow to initialize and run, so I never found a practical use for OCR client-side rather than server-side, but I still think it's tremendously cool.
In case anyone's interested (shameless plug), because I do a lot of academic research that involves tons of copying from webpages, PDF's and screenshots and pasting into notes documents, I created a tool at https://pastemagic.com https://pastemagic.com that helps selectively remove rich text formatting, remove line breaks and does OCR on screenshots and camera photos. Setting up Tesseract on my server and creating a simple HTTP endpoint for it took less than an hour, and for free I had OCR as powerful as Google's. Pretty cool I thought.
[1] https://github.com/tesseract-ocr/tesseract https://github.com/tesseract-ocr/tesseract
- bhanhfo 7y ago> As far as I know, it powers all OCR at Google (e.g. in Keep, Docs, etc.). Afaik Google no longer uses Tesseract for any of its products. Googles Clould OCR is much better than Tesseract. I think Google devs still work on Tesseract, but only as their side project (not sure about this, obviously)
- crazygringo 7y agoOh very interesting. I'd verified the output was identical a couple of years ago, and that Keep and Docs in production were using the 4.0 beta release at the time. But if Cloud OCR is better, makes sense they would have switched since then. Tesseract 4.0 has a brand-new neural engine that totally supersedes the earlier engine, however -- I wonder if there's any relation between that and Cloud OCR?
- ComputerGuru 7y ago“Cloud OCR” is an interface. Something is still doing the OCR behind the scenes (and that may indeed not be Tesseract).
- visarga 7y agoIt's probably a detection neural net (such as Faster R-CNN) for putting bounding boxes around words, which is complicated by the fact that it can predict polygons in any orientation, followed by a LSTM-CRF layer for text transcription. It's a good generalist OCR but often has sub-par results for specific types of input. It tens to often miss single letters surrounded by whitespace.
- raisedbyninjas 7y agoIME Google's OCR is much more accurate than Tesseract. I doubt they still use it.
- nkrisc 7y agopastemagic is actually a really cool way to read Wikipedia articles. The link density of the usual Wikipedia article is very high and I find it very distracting when some significant portion of the text is in a different color. Combined with the nice typesetting in the output, it makes for a very pleasant reading experience.
- nick_name 7y agoYou might also like Wikiwand - https://www.wikiwand.com https://www.wikiwand.com
- scottlocklin 7y ago> Tesseract is developed by Google since 2006, having been started at HP in 1985 and open-sourced by HP in 2005. Fun fact: it actually started as a US national defence initiative in the last big AI hype bubble in the 1980s. While it isn't in the wiki, probably for intellectual property reasons, I'm almost positive Cuneiform originated as the Soviet version of the same thing (I have evidence in notebooks somewhere): OCR was something needed for "AI" of the 1980s in the Soviet system as well. Had it been further developed by Microsoft ... or Apple, in contrast to Tesseract, it would have continued to be the official OCR of an opposing world-historical system.
- dstroot 7y agoCan you share more information about how you created an HTTP endpoint? Or code? Glancing at the docs I only saw a command line or C bindings.
- crazygringo 7y agoOh sure, on my webserver I just wrote the POSTed image to a temp file, called the command-line utility itself from within my code, and captured the stdout to return back. The command-line utility initializes very quickly so the performance was fine. The only semi-tricky bits were parsing stderr if anything went wrong (distinguish warnings from actual errors), and the fact that Tesseract doesn't respect the JPEG orientation bit (big problem with iPhone camera images), so checking that and manually rotating the JPEG first if necessary (gibberish otherwise).
- davvolun 7y agoDo you have it somewhere on github? I could see hosting this on my own server for personal use, maybe trying to contribute back as well.
- NewDimension 7y agoSomewhat offtopic, do you know of a library that would allow me to select an area of a PDF through a GUI and only read the text in those coordinates?
- jjohansson 7y agoCommercial or open source? PDFTron can do it, but they’re not an open source project.
- NewDimension 7y agoI prefer something I can install locally (doesn't need to be open source). I'm trying to extract text from a PDF at a certain position, the PDF is indeed text not an image so OCR isn't strictly needed. The goal is to draw a box using GUI, then use those coordinates to extract text from several homogeneous pages. I also have a different goal of trying to interpret structure of a PDF that has visual structure (headers, sections and subsections all numbered). But that seems to lend itself to some sort of text parsing.
- severine 7y agoI also have a different goal of trying to interpret structure of a PDF that has visual structure (headers, sections and subsections all numbered). But that seems to lend itself to some sort of text parsing. Some reading here: https://stackoverflow.com/questions/53219016/detecting-sections-of-a-pdf-with-pdfminer https://stackoverflow.com/questions/53219016/detecting-secti...
- jjohansson 7y agoPDFTron provides an SDK and isn't really meant as a plug-and-play end-user application. But it can accomplish what you're looking for. Here's how to extract text from a PDF based on coordinates (this explains how to do it on web, but it's also possible using other platforms): https://groups.google.com/d/msg/pdfnet-webviewer/h2W3VksbQUI/oGFwbyNeDAAJ https://groups.google.com/d/msg/pdfnet-webviewer/h2W3VksbQUI... Here's how to extract a PDF's logical structure: https://www.pdftron.com/documentation/samples/#logicalstructure https://www.pdftron.com/documentation/samples/#logicalstruct...
- beagle3 7y ago> As far as I know, it powers all OCR at Google (e.g. in Keep, Docs, etc.). Tesseract is acceptable only if the text is neatly laid out, in more-or-less straight parallel lines or at the very least consistent orientation that's close enough to being straight horizontal lines. Google Cloud Vision, however, can read any orientation, any font, through perspective distortion and does not need the different text blobs in the image to be consistent in any way. Superior in every way to plain Tesseract (and if it is Tesseract after preprocessing, the magic is in that preprocessing more than it is in Tesseract) I would actually be very surprised to hear GCV uses Tesseract; and if they don't, why would they use something inferior for other products?
- visarga 7y ago> As far as I know, it powers all OCR at Google (e.g. in Keep, Docs, etc.). Tesseract is the shittiest OCR and Google doesn't use it internally. Their cloud OCR offering is much more performant.
- amelius 7y agoCurious, does it use Deep Learning techniques, and for what tasks?
- antimatter15 7y agoDisclaimer: I was the original author of Tesseract.JS— though all the hard work nowadays is done by Jerome Wu. If you're interested in supporting the project, consider backing the OpenCollective (https://opencollective.com/tesseractjs https://opencollective.com/tesseractjs) By all of that, what I mean to say is that I've learned a decent amount of fun OCR trivia over the past few years. Firstly, the engine that powers Google Cloud Vision is almost certainly an entirely independent code base from Tesseract built on neural networks. In fact, the most recent major version of Tesseract (version 4.0) was a sort of rewrite of the core of Tesseract to use bidirectional LSTMs to seem a bit more like the modern OCR pipeline that systems like GCV use. The original Tesseract algorithm dates back a previous AI spring— in the 80s when neural networks were cool (before they were uncool, and then subsequently cool again). The core of the original algorithm involved fitting polygons to character shapes in order generate features which could be matched by a kind of rudimentary neural network. One of the primary authors of Tesseract is Ray Smith (at Google)— who gave a presentation at some point a few years ago about the history of OCR— though I can't quite find a link to it at the moment. OCR actually predates electronic computers. In 1929, someone had invented a machine that would take a piece of paper and shine a bright light on a single letter, and pass the letter through a carousel of letter masks, so that it could hit a (effectively single pixel) photo-sensor. When the carousel and the letter mask were in alignment with the printed letter, then the drop in brightness registered that a particular letter was seen! OCR was used by the US Postal system for sorting mail as early as 1965, but it wasn't until 1976 that any system could reasonably support more than a certain number of hard-coded fonts (fun fact this was invented by Ray Kurzweil, the Singularity is Near guy).
- alexcnwy 7y agoFirst of all thank you for all your hard work. Major question: Why isn’t Tesseract using neural networks. I know it just introduced LSTM based models but they suck. Why is GCP vision text recognition API so much better than open source alternatives?!
- xanderjanz 7y agoThere are open source versions of everything done within a GCP API call, but it requires multiple machines and lots of data to build an NLP model to be as fast and accurate as GCP, and cloud computing is relatively new compared to OCR.
- lucasverra 7y agoadded your tool to my firefox new tab. If other say that Google OCR is "way better" with don't you implement that behind your endpoint ?
- crazygringo 7y agoI'll have to look into it! But right now it's a free tool so I can't afford to set up paid OCR :)
- alexcnwy 7y agoTesseract simply cannot power the GCP OCR API because Tesseract sucks super bad and the GCP API is mint.
- bhl 7y agoI've been working on an editor on top of ProseMirror to support saving web content in the form of rich text and predefined schemas. Given that you have academic research in this area of web OCR, what's the current literature or tools on saving web content using both html and visual cues from the rendered html? For example, both <figure><img><figcaption> and <div><img><p> visually look like captioned images, but are represented differently in html. Is there a way to parse that into a simple [figure, [img], [figcaption]] schema?