6 ms·
Unlimited OCR: One-shot long-horizon parsing
- Oras 4mo agoOCR has been solved long time ago with vision models. Solutions are consistent, reliable, and stable. What is the point of reinventing the wheel? I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself?
- Aboutplants 4mo agolol nope it hasn’t been solved. I deal with this constantly and we still have a longggg ways to go
- vulture916 4mo agoI haven't done much long-run OCR, so unsure of the current state, but it would seem they overcome this (from their paper): "A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation."
- ta988 4mo agoCost, throughput, latency...
- Oras 4mo agoTraditional OCR is faster, cheaper, and much more reliable than LLMs
- ta988 4mo agoI don't think that's a universal statement that aplies to every kind of documents and languages. Mistral OCR is able to do things no "traditional" OCR was ever able to.
- j16sdiz 4mo agoIf you consider non-English script, traditional OCR is not more reliable. CJK have lots of character and high confusion rate. Arabic scripts are complex and have lots of morphs. Vietnamese have easily confused diacritics. Thai have lots of non-standard fonts.
- JodieBenitez 4mo agoI wish it were. Alas...
- cannonpalms 4mo agoI guess, in theory, the prior distribution of language would allow for improved performance in some cases, especially where input quality is low.
- ta988 4mo agoThis is already used in OCR, tesseract uses that.
- chpatrick 4mo agoIt absolutely hasn't been solved, it's just got pretty decent in recent years.
- malfist 4mo agoPretty decent might be quiet the stretch. I'd term it almost acceptable, but only if you're using commercial solutions like amazon's textract, doing it with open source tools is at best, extremely painful and vaguely accurate.
- JohnKemeny 4mo agoOCR has definitely not "been solved long time ago", what are you talking about? In your opinion, what is SOTA here?
- sscaryterry 4mo agoDetecting characters almost, layout no.
- wongarsu 4mo agoExactly my experience. If you try to OCR hand-filled forms with a fixed structure, traditional OCR models are great. Vision-llms can improve a bit on character recognition, but at the cost of harder to detect failure modes. But if you are trying to ingest diverse documents with headings, multi-column layouts, headers and footers, ad space in the middle of your text, etc, vision-llms are a giant step forward. But you need the context of the previous page to make good decisions about the current page, which is where things quickly get janky (or slow, if you choose the naive approach) Vision-llms also seem to deal much better with variance in scripts. Cursive, random Japanese in the middle of the text, weird math symbols, handwriting from three centuries ago, all "just works" without you even having to remember that this can happen
- joss82 4mo agoI've been working on Parseur for the last 10 years, and OCR has not been solved yet, let me tell you. OCR still sucks in 2026. Hopefully this might improve the situation but I haven't tested it yet.
- ljouhet 4mo agoReal question: what tool do you use? (for long/complex documents with tables, code, maths) - marker (with --force-ocr) gives me the best results - Mistral OCR (seems really great, but I never managed to get it work) - Mathpix (tried a long time ago) - docling (gives me garbage, I must use it wrong) - Unlimited OCR (will try it) - ???
- Oras 4mo ago- Azure Document Intelligence (has an option to return markdown too including headers and footers). - AWS Textract
- badlibrarian 4mo agoExactly. They're both very expensive and prone to surprising you. Sometimes in a good way, sometimes in a bad way. I'd rate them 85%, but you have to run a test because they both fail in different ways on the 15%.
- ai_fry_ur_brain 4mo agopoma-ai has really great chunking techniques that chunk the document based on the document structure/heirarchy. We use it on 200 page IEEE standards that are notoriously complex, filled with tables and diagram. Highly reccomend.
- mschuster91 4mo ago> I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself? Well... the idea seems to be (as far as I understand it, at least) that optical errors and artifacts can now be compensated as the OCR engine is now context-aware. Say, for example, some random long ass name chemical. It's not going to be in a word correction database, but a context-aware engine (ideally, one that has been supplemented with chemistry data) can now correct "bad" reads of the chemical's name. Of course, there remains the issue of how to prevent the infamous Xerox bug [1]... [1] https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
- gettingoverit 4mo agoIs it? I've never seen a single OCR that would replace a human just typing it by hand. What if the goal is something actually useful, such as converting scientific paper PDF back to LaTeX that renders into a pixel-perfect copy? What about converting tables from electronics datasheets into computer-readable form? I wouldn't even expect it in the next decade.
- SyneRyder 4mo agoI've had success with vision models & OCR, saved me many hours / days / weeks of typing work. Last year I finally OCR'd many hundreds of pages of my father's old writings. I found that feeding it to Claude Sonnet 4.x via API gave me results that were perfect. No corrections required. So perfect, that Claude was reading along with the story, and actually pointed out a continuity error in the story where an incorrect character was reciting dialog. Claude asked if it should transcribe exactly as is or if I would like Claude to correct the continuity error. Claude also correctly OCR'd some handwriting that was in the margins of the documents. Sonnet came very close to transcribing a Word Sleuth puzzle, but that was where I hit the limits of its capability at the time. Mistral OCR was also good (and actually what I started with), but it wasn't quite as good as Claude. And when it was wrong, Mistral could be frighteningly wrong - one API call must have failed, the model must have been presented with a pure black / null image, and I got back a "transcription" that described neverending darkness. It read like something the Woodsman would have broadcast in Twin Peaks S3E8. That poor model. Tables from electronics datasheets might be okay, I think I've had success with OCR of technical manuals with tables for 80s synthesizer hardware. But I admit my use cases don't crossover into transcriptions of equations or graphs.
- mamcx 4mo agoAside: what is the best to read receipts/bank statements/invoices?
- robotswantdata 4mo agoVery interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, developers are forced to build janky code that chops PDFs into individual pages, processes them one by one, and glues the text back together. Unlimited OCR uses Reference Sliding Window Attention (R-SWA) to split the AI's focus into two paths: Global Reference: The AI keeps full, uncompromised sight of the original document image so it never loses context. Local Generation: The AI restricts its memory of its own typed text to a tight, moving window (like the last 128 words) and safely forgets the rest. Will be very interesting for local AI and can’t wait to see what the community builds and extends with it!
- d675 4mo agoSee, leetcode is useful. As I do this leetcode grind, I’ve been why techniques exist / how they’re used irl. Lots of interesting stuff there
- ai_fry_ur_brain 4mo agoWho said it wasnt useful, dont listen to those people.
- Xevion 4mo agoPeople who are applying to jobs and are tested with LeetCode problems to assess their skill level, despite the two not really being correlated or relevant for the position
- galbar 4mo agoAs someone that gets very annoyed when having to do LeetCode in interviews... Knowing algorithms, data structures and their memory and time complexities is very relevant for SWE. I've had teammates that didn't understand them and everything was fine until when it wasn't (scaling and performance issues). Or, as I put it to a teammate: "Would you rather review the PR of someone that understands the difference between a set and a list or the PR of someone who doesn't?". This was after we interviewed a candidate with ~15 YoE, on paper, that didn't know the difference.
- KitN 4mo ago"We would like to thank Deepseek-OCR, Deepseek-OCR-2, PaddleOCR for their valuable models and ideas." Class Act.
- gcr 4mo agoI don’t understand the shade being thrown ?
- nickspacek 4mo agoIt's the opposite of shade, unless GP is being sarcastic. "Class act" is normally a compliment, and in the context here it sounds to me like they're congratulating Baidu/the researchers in being transparent about where their ideas came from.
- pbhjpbhj 4mo agoTo be fair, I think I see "[real] class act" almost always used sarcastically.
- squidbeak 4mo agoI've never seen it used that way. Any compliment can be repurposed as sarcasm, but it's obscenely cynical to immediately assume a compliment is sarcastic - instead of just a compliment. And by the way, there's no 'real' in the poster's message.
- pbhjpbhj 4mo agoYet you responded to my comment in the most cynical way possible. I was excusing the misunderstanding -- I assumed that the parent might only have seen it used cynically as that is a charitable way to interpret the apparent miscommunication and is quite possible. You shit on my comment and then told me doing such things is "cynical". Imagine a pipe before the closing square-bracket if you wish -- the standard editorial convention as I learnt it was that square-bracketed terms may be present or not (in English Language prose). Cheers.
- manipalite 4mo agoWhatever happened to Reducto, was very promising 12-15 months ago
- madikz 4mo ago[flagged]
- ramon156 4mo agoI love that the entire goal is to push Deepseek OCR further. The west can learn greatly from these companies
- pmarreck 4mo agomy attempts at using AI to do OCR have always resulted in invented artifacts, which is not production feasible. does this suffer from that as well? A simple example is words that are supposed to be in other languages being automatically translated to English, which ruins the effect
- drakmo 4mo agoIf I would want to achieve 100% recognition results I would combine this method with an image model recreating the original document from the transcribed text and matching the layout. One can do that with using all but the page or paragraph from the document you want to recreate (to avoid recreating the exact passage under test from the image artifact directly). After reconstructing you can do an optical comparison that specifically matches misaligned characters and find the errors. Rinse and repeat. Expensive but it would guarantee 100% recognition.
- pbhjpbhj 4mo agoYou almost don't want [super-]word level ML (ie word-pair/phrase/sentence/document/corpus level). In transcription, you want near certainty, or you want marking that the word could not be read with certainty - yes, context lets you guess, but you want - for some OCR - to know when it's a guess based on other than the letters in order forming a word. Example, in a census document on familysearch.com the transcriber "corrected" a name as Joseph. The literal letters in the handwritten document spell Josepth ... and sure enough that's a local variant spelling (Eire). In another document the writer has used "Joh" as an abbreviation, a [human, I assume] transcriber put that as John ... which is most likely, but happens to be wrong. Sometimes you care that it's guessed, sometimes you want just the best guess.
- messe 4mo ago> Eire A nitpick, because it's often a dogwhistle: but almost nobody in Ireland calls it that when speaking English. And that's still incorrect in Irish, the correct spelling is Éire.
- 4mo ago
- overflowy 4mo agoWhat are the requirements for running this locally?
- alansaber 4mo agoWe've invented chunking? We are so back.
- ahknight 4mo agoStreaming.
- peatmoss 4mo agoI recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music is basically a greenfield for AI wherever you look. Optical music recognition is pretty terrible. AI understanding of music theory is terrible (actually looking at music that is; LLMs do okay at text descriptions of theory concepts where you can imagine some online texts making it in). I think the issue is that we still don't have great digital formats that encode the dots on paper that musicians read. Music notation is pretty rich. Midi doesn't capture all of what's needed for symbolic understanding, because it was mostly made for capturing aspects relevant for playback or performance. MusicXML seems to be the closest for a digital format that encodes the information a musician would want, but there aren't great corpora of training data that would connect a MusicXML representation to sheet music images or to audio. I think that's because MusicXML falls short of encoding enough information to engrave music. Tools like MuseScore need to track a bunch of layout information that isn't encodable in MusicXML. Lilypond format is less verbose that MusicXML and contains a bit more information that is useful to the score creators, but most people don't create sheet music in lilypond. (As an aside, Lilypond bums me out with the state of jazz fonts. I hate looking at "legit" scores in jazz context) I realize this is mildly off topic, but every time I see people making incremental gains on OCR, which to my mind is pretty good, I am reminded of how abysmal OMR is.
- singpolyma3 4mo agoWhat about sheet music typesetting formats like https://abcnotation.com/ https://abcnotation.com/ ?
- peatmoss 4mo agoI forgot to mention ABC. I have seen a few LLMs look at that. There was a model / paper published a couple years back called ChatMusician that built around it. With the caveat that I'm not terribly fluent in ABC, it seems to me that simple things are simple, but hard things seem to be nearly pathological. And (again, maybe a lapse in my understanding) it seems like there may be a fair number of concepts that are impossible to convey in ABC? Lastly, if I understand correctly, ABC got its start and is mostly popular as a simplified format for church songbooks. I'd imagine that would, uh, influence the training corpora towards sounding a bit... church songbooky. EDIT: I may have been overly dismissive of ABC on first glance. It does seem like people have extended it quite a bit, and that it's at least, in theory, capable of encoding most of what I'd expect. And it's human readable, which is a benefit. Though, readability does take a stiff penalty the more richness you add (e.g. dynamics, articulations, stacked notes, etc)
- deleted 4mo ago[deleted]
- swordlucky666 4mo ago[flagged]
- shevy-java 4mo agoIs this an academic paper that is published in year xyz, but in +5 years nobody will remember it anymore?
- janpeuker 4mo agoPaper under https://arxiv.org/abs/2606.23050 https://arxiv.org/abs/2606.23050 (As a side note, I do OCR locally as a small RAG for citations I read in books and also chunk input, but merely to save RAM - interesting this natural approach also work in a streaming model)
- piterrro 4mo agocan someone explain how is this different than feeding the VLM model one page at a time?
- functional_dev 4mo agoI did not know this until today... vision models don't hunt for the white space between letters like old OCR does. They just chop the whole image into a fixed grid of equal squares (patches) and treat each square like a word in a sentence. This will help if you want to dig deeper - https://vectree.io/c/how-ocr-works-traditional-pipelines-vs-vision-language-models https://vectree.io/c/how-ocr-works-traditional-pipelines-vs-...
- novoreorx 4mo agoFYI, "Unlimited OCR Works" is a Fate/stay night reference. The original "Unlimited Blade Works" is a magic whose entire premise is copying weapons other people forged
- Ey7NFZ3P0nzAe 4mo agoThanks. Full context : https://typemoon.fandom.com/wiki/Unlimited_Blade_Works https://typemoon.fandom.com/wiki/Unlimited_Blade_Works
- aozelai 4mo ago[flagged]
- gettingoverit 4mo agoHow does it compare against Finereader? Comparisons against transformer-based OCRs don't really tell anything. The last time I checked, neither of them were of "OCR this legal document" quality.
- aliljet 4mo agoHow does this compare with infinty parser 2 which seemed to be running the table on every other OCR tool (https://huggingface.co/datasets/allenai/olmOCR-bench https://huggingface.co/datasets/allenai/olmOCR-bench). To be fair, there's no single winning OCR benchmark and this isn't showing up anywhere yet..
- jingpostmedia 4mo ago[flagged]
- lacoolj 4mo agoThis looks more promising than what Mistral just launched (coincidence?????? i think not.) This approach feels like it could be used for image gen as well (in some combination). Read/view image, start drawing image using illustrator/inkscape/etc (or just SVG), then fill in with what was missed after
- arboles 4mo agoI'm going to sound like I live under a rock, but what is the true reason companies open-source genuinely good software? Shouldn't Baidu (or Google) hoard it for themselves to extract the value in a way the competition isn't be able to imitate?
- SirYandi 4mo agoSome people working in big companies believe in the ideals of open source and convince their employers to allow open sourcing a project. Employers get prestige (useful for the hiring funnel) and sometimes strategically disrupt competitors (e.g. Meta releasing Ollama)
- jerrygenser 4mo agoReleasing open source models can drive revenue away from them US AI LABS. This can help china win by depriving those labs of revenue for further investment in winning the long term race.
- AaronNewcomer 4mo ago[flagged]
- dagni132 4mo ago[flagged]
- jbarrow 4mo agoI'm always glad to see more multi-page work in VLM-based OCR. Especially single-pass. One of the few other multi-page papers from recently, MinerU-Popo, treats fixing up multi-page outputs as a post-processing correction step (https://arxiv.org/abs/2605.24973 https://arxiv.org/abs/2605.24973). Interesting to see the drop-off in quality as you up page count, though. I also think the attention approach (always attend to the image/prefix, with a sliding window for local context) is neat! I do wish they updated their comparison table to include more recent work (that scores marginally better on OmniDocBench), like dots.mocr.
- vrc 4mo agoWhat are your thoughts on the detector --> VLM pipelines, and if there's ever a world where a small LM or LM augmented detector can be efficient enough to play a role as router. I ask because I recognize you from your handle and am very familiar with your work in the doc+detector space.
- madikz 4mo ago[flagged]