well, the problem is data. NN needs a lot (depending on the problem thousands or millions of samples). It will probably get very difficult to get this much data

well, the problem is data. NN needs a lot (depending on the problem thousands or millions of samples). It will probably get very difficult to get this much data needed to train an ML algorithm the location of the relevant OCR text. Often ML problems way more experimental than "normal" coding. I would first try modelling it as a classification-problem and just do some cross-entropy validation to check the performance of the model. If it's useful, go with it, if not back to the drawing board. You will need some serious computing power, so either buy some GPUs or use the cloud. You could train a random forest based on the inputs of OCR and NN, if you want to get total ML. You would gain some interpretability (i don't know whether thats important, but i would guess it might) I am sorry that I can't give you a more concrete answer, these are just ideas. They are probably wrong. Like i said, i am a beginner and also don't really know the problem. Edit other idea: If you know the location of the relevant OCR-text, you could use the following approach: Use the NN for classification. It will return probability-like values for every category. Take the top 2 (or 3, or every top until they add up to 70 percent...idk). Then do some OCR for every category you have to check. If one is positive you have your result, if not run the others.