3 ms·

Immediately made me think of a question on the state of handwriting to LaTeX from a few years ago, and all the massive challenges involved: https://tex.stackex

Immediately made me think of a question on the state of handwriting to LaTeX from a few years ago, and all the massive challenges involved: https://tex.stackexchange.com/questions/1443/what-is-the-status-of-generating-latex-from-handwriting-i-e-ocr https://tex.stackexchange.com/questions/1443/what-is-the-sta... Under API... you're already doing handwriting? This is uh, nontrivial work to say the least. Really impressive. The endorsements are a nice touch. :) Made me really curious how far the system goes, what cases break it. Oh... nevermind. You have a PDF of examples here: https://docs.mathpix.com https://docs.mathpix.com It's honing in on equations without getting distracted by nearby Hanzi or Cyrillic, or even pictures of dogs. Wow. I keep going back to dig through your resources and getting more impressed. EDIT: I guess my only constructive criticism is that you should brag more. I like a simple landing page, but I think you've earned a short list of examples of corner cases you tackle well, if the whole API is packed into that free app, because they're really impressive.

yorwba 9y ago
The perfection in those examples makes me suspect that they are cherry-picked or part of the training data. Especially the handwritten text is not always clear and could reasonably be interpreted differently. I'd expect a machine-learning model to get at least some things wrong some of the time. If I wanted to use this in an application, I'd definitely want to see some accuracy figures on validation data as well as a few failure cases to see whether the output remains reasonable even when it is wrong.
- nicodjimenez 9y ago
  The examples are actually very simple compared to a lot of crazy stuff Mathpix can recognize so it's an honest representation of its capabilities. Mathpix is built for perfection because 99% isn't good enough.
- Const-me 9y ago
  > the handwritten text is not always clear and could reasonably be interpreted differently Digital pen input contains more info than the resulting bitmap; strokes are lost while rasterizing. That info was the reason how old devices were able to reliably recognize characters written by a stylus. It worked well even on prehistoric hardware, such as 16MHz CPU + 128 kB RAM in the first Palm PDA.
  - ocrcustomserver 9y ago
    In the OCR world this is known as offline OCR (OCR on the bitmap) vs online OCR (strokes information). Offline is way harder than online.

yorwba 9y ago

The perfection in those examples makes me suspect that they are cherry-picked or part of the training data. Especially the handwritten text is not always clear and could reasonably be interpreted differently. I'd expect a machine-learning model to get at least some things wrong some of the time. If I wanted to use this in an application, I'd definitely want to see some accuracy figures on validation data as well as a few failure cases to see whether the output remains reasonable even when it is wrong.

nicodjimenez 9y ago
The examples are actually very simple compared to a lot of crazy stuff Mathpix can recognize so it's an honest representation of its capabilities. Mathpix is built for perfection because 99% isn't good enough.
Const-me 9y ago
> the handwritten text is not always clear and could reasonably be interpreted differently Digital pen input contains more info than the resulting bitmap; strokes are lost while rasterizing. That info was the reason how old devices were able to reliably recognize characters written by a stylus. It worked well even on prehistoric hardware, such as 16MHz CPU + 128 kB RAM in the first Palm PDA.
- ocrcustomserver 9y ago
  In the OCR world this is known as offline OCR (OCR on the bitmap) vs online OCR (strokes information). Offline is way harder than online.