5 ms·

> I try a bunch of different OCR programs, but can't find any that can transcribe the document with 100% accuracy. They often confuse certain letters or numbers

> I try a bunch of different OCR programs, but can't find any that can transcribe the document with 100% accuracy. They often confuse certain letters or numbers (like 0 and C, 9 and 4, 0 and D). Sometimes they omit characters, sometimes they introduce new ones. I try different font sizes and different fonts, but it doesn't matter. I decided to OCR a hex dump from an old computer magazine a while back and fixed this problem by writing a tool to help verify the OCR result. Basically you input the OCR'd result and segment the numbers. It'll display the original segmented characters ordered by their class, and the human eye will very quickly find any chars that do not belong, e.g. 3s sorted under "8" etc. https://blog.qiqitori.com/2023/03/ocring-hex-dumps-or-other-monospace-text-and-verifying-the-result/ https://blog.qiqitori.com/2023/03/ocring-hex-dumps-or-other-... https://blog.qiqitori.com/2023/03/ai-day-in-retroland-prolog-on-the-commodore-vic-20-needs-expanded-ram/ https://blog.qiqitori.com/2023/03/ai-day-in-retroland-prolog... I wrote two blog posts about this, and the tools are also linked from the blog posts. Note: the tools are just slightly more user-friendly than sendmail. That said, I don't know if these old Apple laptops came with anything resembling a programming environment (or at least that ancient version of Microsoft Word?), but even if not... There must be a better way (even without hardware hacking)!

OptionOfT 2y ago
This reminds me of an OCR application I used for converting PGS subs to SRT. It would show you words/letters grouped together and you had to approve / correct its translation.
jdthedisciple 2y ago
> I try a bunch of different OCR programs, but can't find any that can transcribe the document with 100% accuracy. They often confuse certain letters or numbers (like 0 and C, 9 and 4, 0 and D). Sometimes they omit characters, sometimes they introduce new ones. I try different font sizes and different fonts, but it doesn't matter. I feel like this could be trivially solved by plugging an LLM to the OCR output with the sole task to correct spelling errors like that. That's pretty much one of the tasks LLMs should excell the most at.
- wizzwizz4 2y ago
  Denoising algorithms are always lossy. An LLM (or, y'know, Markov chain) could do this job by exploiting statistical regularities in the English language, but a hex dump isn't quite the English language, so it'd be completely useless. Even if this text were English, though, the LLM would make opinionated edits (e.g. twiddling the punctuation): you'd be unlikely to get a faithful reproduction out the other end.
  - jdthedisciple 2y ago
    > hex dump ah, missed that, was just skipping through
    - Moru 2y ago
      Still would not solve the problem of copying data without changing it.
  - ratboy666 2y ago
    Of course, use search and replace to change 0 to zero... etc. The OCR will (should) work better.
    - wizzwizz4 2y ago
      You might as well just use an error-correction code: same result, less overhead.
- mminer237 2y ago
  It's hexadecimal. There is no spelling, so there's no way for an LLM to know if something is supposed to be a `D` or a `0` any more than traditional OCR software can.
  - jdthedisciple 2y ago
    yes i noticed that way too late, my bad
zamfi 2y ago
One could also find-and-replace in Word to pick a set of 16 characters that avoids OCR confusion…
- akoboldfrying 2y ago
  Good idea. Maybe making the replacements short, space-separated words would have helped even more, by countering any "helpful corrections" the OCR might have been making based on English words (e.g., it would not surprise me if some OCR software would "correct" "B0B" to "BOB").
tfvlrue 2y ago
This is great! I'm amazed that someone else has solved the same problem in a different context. Good stuff.
lisper 2y ago
Programming environments were available of course, but none were built-in. The fact that the external floppy doesn't work is the main problem here. I think a better solution would have been to use the serial port rather than the modem port to send a fax. Serial-to-USB adapters are easy to find.
- romwell 2y ago
  >I think a better solution would have been to use the serial port rather than the modem port to send a fax. Serial-to-USB adapters are easy to find. With the file sizes involved, one could take a picture of the screen showing the hex dump with a smartphone, and then OCR that. No need to even bother with external hardware.
  - remram 2y ago
    Given that the perfect image from the fax barely worked, a photograph would certainly not do.
    - wruza 2y ago
      Or they’d do with a similar accuracy, since the main factor here is font rendering resolution.
    - romwell 2y ago
      >Given that the perfect image from the fax barely worked, a photograph would certainly not do. Photo resolution is higher than a fax resolution, and one can increase the font size / decrease screen resolution. Plus, they can still apply the same approach they used for fax.
- II2II 2y ago
  > I think a better solution would have been to use the serial port rather than the modem port to send a fax. Serial-to-USB adapters are easy to find. Two potential problems. I have tried serial transfers on a similar vintage of Mac, and it seems to require a USB/Serial adapter that supports proper RS-232 logic levels. Many (perhaps most) don't. I ended up using the built-in serial port on an older PC. The second issue is that you require software on the Mac side to perform the data transfer. Something like the rather common ClarisWorks would do the job, but the presence of Word and PowerPoint suggests they didn't have ClarisWorks. (Even if it did have ClarisWorks, they would need to know that ClarisWorks includes built-in communications software. It isn't exactly a common component in office suites.) I suppose they could have explored AppleTalk, print drivers, or (depending on the version of MacOS) TCP/IP. Each option is limited though. AppleTalk requires another suitably equipped Mac or a PC with rare hardware and ancient software. (I think Sun systems would work too, but then we are really getting into the weeds.) I'm pretty sure the Mac print drivers rasterize the text, so you would still have to undo that. (Though it would probably transfer the data faster.) I think TCP/IP only shipped with MacOS 7.5 and later, would have required setup on another PC, and would still be convoluted (e.g. downloading FTP software to the Mac, so it could upload to a PC). It's going to be a tricky situation.
Terr_ 2y ago
That could become an automated improvement as well... although it was probably published in some paper 40 years ago and I just don't know the name of it. Basically a second-pass where you look for outliers in each character-group, reassign them to another group, and keep the change if it improves similarity-scores the character groups involved. Then iterate a bit until nothing seems to be improving. For example, one might find "the 3 that is least-like all the other 3s", temporarily reassign it to "8", and then keep that change if it means an improvement in the the scores for "how closely all sub-pictures of 3s resemble each other" and "how closely all sub-pictures 8s resemble each other". That might backfire if a document has different typefaces in it though, where it makes the mistake of putting all "3"s from different typefaces together, ruining the group-similarity scores.
- KeepFlying 2y ago
  For typefaces you check for the distribution of similarities in each group. If it has large clusters then group by 3, 3', etc then run your outlier check on each of those groups Still would risk some weirdness, but would help a bit I'd hope. I wonder if it would be worth running some kind of language analysis or spelling/Grammer check to verify the scan too. At least for text, you'd need another solution for number tables.

OptionOfT 2y ago

This reminds me of an OCR application I used for converting PGS subs to SRT. It would show you words/letters grouped together and you had to approve / correct its translation.

jdthedisciple 2y ago

wizzwizz4 2y ago
Denoising algorithms are always lossy. An LLM (or, y'know, Markov chain) could do this job by exploiting statistical regularities in the English language, but a hex dump isn't quite the English language, so it'd be completely useless. Even if this text were English, though, the LLM would make opinionated edits (e.g. twiddling the punctuation): you'd be unlikely to get a faithful reproduction out the other end.
- jdthedisciple 2y ago
  > hex dump ah, missed that, was just skipping through
  - Moru 2y ago
    Still would not solve the problem of copying data without changing it.
- ratboy666 2y ago
  Of course, use search and replace to change 0 to zero... etc. The OCR will (should) work better.
  - wizzwizz4 2y ago
    You might as well just use an error-correction code: same result, less overhead.
mminer237 2y ago
It's hexadecimal. There is no spelling, so there's no way for an LLM to know if something is supposed to be a `D` or a `0` any more than traditional OCR software can.
- jdthedisciple 2y ago
  yes i noticed that way too late, my bad

zamfi 2y ago

One could also find-and-replace in Word to pick a set of 16 characters that avoids OCR confusion…

akoboldfrying 2y ago
Good idea. Maybe making the replacements short, space-separated words would have helped even more, by countering any "helpful corrections" the OCR might have been making based on English words (e.g., it would not surprise me if some OCR software would "correct" "B0B" to "BOB").

tfvlrue 2y ago

This is great! I'm amazed that someone else has solved the same problem in a different context. Good stuff.

lisper 2y ago

Programming environments were available of course, but none were built-in. The fact that the external floppy doesn't work is the main problem here. I think a better solution would have been to use the serial port rather than the modem port to send a fax. Serial-to-USB adapters are easy to find.

romwell 2y ago
>I think a better solution would have been to use the serial port rather than the modem port to send a fax. Serial-to-USB adapters are easy to find. With the file sizes involved, one could take a picture of the screen showing the hex dump with a smartphone, and then OCR that. No need to even bother with external hardware.
- remram 2y ago
  Given that the perfect image from the fax barely worked, a photograph would certainly not do.
  - wruza 2y ago
    Or they’d do with a similar accuracy, since the main factor here is font rendering resolution.
  - romwell 2y ago
    >Given that the perfect image from the fax barely worked, a photograph would certainly not do. Photo resolution is higher than a fax resolution, and one can increase the font size / decrease screen resolution. Plus, they can still apply the same approach they used for fax.
II2II 2y ago
> I think a better solution would have been to use the serial port rather than the modem port to send a fax. Serial-to-USB adapters are easy to find. Two potential problems. I have tried serial transfers on a similar vintage of Mac, and it seems to require a USB/Serial adapter that supports proper RS-232 logic levels. Many (perhaps most) don't. I ended up using the built-in serial port on an older PC. The second issue is that you require software on the Mac side to perform the data transfer. Something like the rather common ClarisWorks would do the job, but the presence of Word and PowerPoint suggests they didn't have ClarisWorks. (Even if it did have ClarisWorks, they would need to know that ClarisWorks includes built-in communications software. It isn't exactly a common component in office suites.) I suppose they could have explored AppleTalk, print drivers, or (depending on the version of MacOS) TCP/IP. Each option is limited though. AppleTalk requires another suitably equipped Mac or a PC with rare hardware and ancient software. (I think Sun systems would work too, but then we are really getting into the weeds.) I'm pretty sure the Mac print drivers rasterize the text, so you would still have to undo that. (Though it would probably transfer the data faster.) I think TCP/IP only shipped with MacOS 7.5 and later, would have required setup on another PC, and would still be convoluted (e.g. downloading FTP software to the Mac, so it could upload to a PC). It's going to be a tricky situation.

Terr_ 2y ago

That could become an automated improvement as well... although it was probably published in some paper 40 years ago and I just don't know the name of it. Basically a second-pass where you look for outliers in each character-group, reassign them to another group, and keep the change if it improves similarity-scores the character groups involved. Then iterate a bit until nothing seems to be improving. For example, one might find "the 3 that is least-like all the other 3s", temporarily reassign it to "8", and then keep that change if it means an improvement in the the scores for "how closely all sub-pictures of 3s resemble each other" and "how closely all sub-pictures 8s resemble each other". That might backfire if a document has different typefaces in it though, where it makes the mistake of putting all "3"s from different typefaces together, ruining the group-similarity scores.

KeepFlying 2y ago
For typefaces you check for the distribution of similarities in each group. If it has large clusters then group by 3, 3', etc then run your outlier check on each of those groups Still would risk some weirdness, but would help a bit I'd hope. I wonder if it would be worth running some kind of language analysis or spelling/Grammer check to verify the scan too. At least for text, you'd need another solution for number tables.