3 ms·
“Take a screenshot every few seconds, run OCR on it, and dump all the text into a SQLite database” honestly sounds like a machine learning intern wrote a keylog
by fwlr 2y ago
“Take a screenshot every few seconds, run OCR on it, and dump all the text into a SQLite database” honestly sounds like a machine learning intern wrote a keylogger in their coffee break. Hilarious that it’s a real Microsoft feature they proudly advertised.
- kirth_gersen 2y agoYes, it is very easy. I wrote something similar back in 2020 iirc. It only took a few hours. The idea was to have an app that you dumped all your work pdfs into to make them searchable within. It would pull the text and then you could search by keywords and open the pdf at the associated page number. For PDFs that were scanned in from paper, I had to use tesseract OCR. I didn't pursue it further at the time, because the quality of output with OCR was often poor. The OCR would produce large chunks of gibberish mixed into the output text. Now one could do much better. Incorporating the tokenizing for RAG with the OCR output would probably improve the quality drastically, as the small tokens produced during tokenizing would help "filter out" much of the OCR produced gibberish while preserving much more of the actual text and contextual/semantic meaning.