4 ms·

You underestimate how hard the problem is. Things that in other fields would be minor typographical differences are often semantically meaningful in traditiona

You underestimate how hard the problem is. Things that in other fields would be minor typographical differences are often semantically meaningful in traditional math notation. Formula OCR would need to distinguish chi from x, lowercase delta from o with an overbar, a dot-product dot from a dot intended as multiplication, parentheses from brackets, v from the or sign. It needs to be able to determine whether your fraction "a / b / c" (by / I mean horizontal lines) was intended as "a / (b / c)" or "(a / b) / c" from the size of your letters, or from the relative widths of the fraction lines. Of course, anything a human can do, ML can learn to do eventually. Which invites the question: is reading arbitrary hand-written math actually something any human can do?

shashasha2 4y ago
Microsoft Math Solver does a good job at scanning hand drawn equations. Graph and solve simple polynoms
hilbert42 4y ago
It seems to me that few would underestimate how hard the problem is as the difficulty would be obvious to those who develop such software. There are at least two issues here, the first is that there isn't any agreed typographical standard that would make OCR more reliable, and second, there's no consistent or uniform way OCR algorithms are applied. To use your example, the OCR software ought to be able to recognize 'a dot-product dot from a dot intended as multiplication' from its useage context, and where ambiguities or doubts exist the item or aspect of the converted text should be flagged and dropped into an editor that would provide easy access to a choice of selectable options to choose from. In the absence of accurate AI/ML, having ready access to a flexible mathematically-aware editor so as humans can easily make corrections seems an absolute necessity (especially so when OCRed text originates from source material that has not been typeset with OCR in mind). It seems odd that mathematicians and programmers haven't yet agreed on standards and protocols around the OCR of mathematical formulae given that the problem has been with us since the outset OCR.
anigbrowl 4y ago
> You underestimate how hard the problem is. It's hrd, but it's not that hard. Character recognition is half the problem, the other half is context recognition - and there's abundant reference material available. I think we'll see 'copilot for math' aimed at AP/college level users within a few years. It's not something that any human can do, bu something that any human can do with help. The first killer app will be 'I found this formula, please tell me how to read it.' Not in the sense of being a math tutor (although that may come, but simply in the sense of helping students to read it out loud, identify symbols like hats or bars and son so on. Most math books are terrible in this respect because they assume the student already knows all the notation or has someone who can lecture or tutor them about it. This massively inhibits solo learners who can't engage in the practice of 'teaching themselves' by verbally walking through formulae or discussing them fluently, unless they're lucky enough to have found a good reference for notation. If you are not in the latter group, Wikipedia has OK summary articles on notation; and these two books offer variously concise and in-depth tools to built mathematical literacy: https://www.amazon.com/Mathematical-Notation-Guide-Engineers-Scientists/dp/1466230525 https://www.amazon.com/Mathematical-Notation-Guide-Engineers... https://www.amazon.com/Programmers-Introduction-Mathematics-Second/dp/B088N68LTJ/ https://www.amazon.com/Programmers-Introduction-Mathematics-...

shashasha2 4y ago

Microsoft Math Solver does a good job at scanning hand drawn equations. Graph and solve simple polynoms

hilbert42 4y ago

It seems to me that few would underestimate how hard the problem is as the difficulty would be obvious to those who develop such software. There are at least two issues here, the first is that there isn't any agreed typographical standard that would make OCR more reliable, and second, there's no consistent or uniform way OCR algorithms are applied. To use your example, the OCR software ought to be able to recognize 'a dot-product dot from a dot intended as multiplication' from its useage context, and where ambiguities or doubts exist the item or aspect of the converted text should be flagged and dropped into an editor that would provide easy access to a choice of selectable options to choose from. In the absence of accurate AI/ML, having ready access to a flexible mathematically-aware editor so as humans can easily make corrections seems an absolute necessity (especially so when OCRed text originates from source material that has not been typeset with OCR in mind). It seems odd that mathematicians and programmers haven't yet agreed on standards and protocols around the OCR of mathematical formulae given that the problem has been with us since the outset OCR.

anigbrowl 4y ago

> You underestimate how hard the problem is. It's hrd, but it's not that hard. Character recognition is half the problem, the other half is context recognition - and there's abundant reference material available. I think we'll see 'copilot for math' aimed at AP/college level users within a few years. It's not something that any human can do, bu something that any human can do with help. The first killer app will be 'I found this formula, please tell me how to read it.' Not in the sense of being a math tutor (although that may come, but simply in the sense of helping students to read it out loud, identify symbols like hats or bars and son so on. Most math books are terrible in this respect because they assume the student already knows all the notation or has someone who can lecture or tutor them about it. This massively inhibits solo learners who can't engage in the practice of 'teaching themselves' by verbally walking through formulae or discussing them fluently, unless they're lucky enough to have found a good reference for notation. If you are not in the latter group, Wikipedia has OK summary articles on notation; and these two books offer variously concise and in-depth tools to built mathematical literacy: https://www.amazon.com/Mathematical-Notation-Guide-Engineers-Scientists/dp/1466230525 https://www.amazon.com/Mathematical-Notation-Guide-Engineers... https://www.amazon.com/Programmers-Introduction-Mathematics-Second/dp/B088N68LTJ/ https://www.amazon.com/Programmers-Introduction-Mathematics-...