Adept.ai is rumoured to have a model that understands screens and web pages, but they are still in pre-release stage. If you use the image as input, you can tr

Adept.ai is rumoured to have a model that understands screens and web pages, but they are still in pre-release stage. If you use the image as input, you can try the DONUT model [1] that directly answers questions without external OCR. It can be fine-tuned to suit your needs. You can get a pretty decent reading of screens and text documents with Amazon Textract. They find key-value pairs pretty reliably. [1] https://arxiv.org/abs/2111.15664 https://arxiv.org/abs/2111.15664