3 ms·
I don't know about this project, but I can easily find thousands of images that gpt-4o can't read, but a human expert can. It can do typed text excellently, ant
by vintermann 2y ago
I don't know about this project, but I can easily find thousands of images that gpt-4o can't read, but a human expert can. It can do typed text excellently, antika-style cursive if it's very neat, and kurrent-style cursive never.
- tptacek 2y agoFor straightforward reasons, I am commenting on this project, not the space of all possible projects. I did try, once, to get 4o to decode the Zodiac Killer's message. It didn't work.
- vintermann 2y agoThe point is that gpt-4o can't read cursive very well. The chain of thought reasoning doesn't seem to help at all. It's a thing the model doesn't generalize to, that's relevant to this discussion.
- tptacek 2y agoI'm sure you can find something it does a bad job on (have you, though?) but it's done swimmingly on some really horrendous 18th century examples I've found. Among other obvious things, it translates the Revolutionary War cursive that leads off this article. I think people may be kidding themselves a little bit about this stuff.
- vintermann 2y agoI wish! I do genealogy, I would love automated transcription of historical documents. Have I? Oh yes! It wasn't rhetorical that I could find you thousands of pages. Here's one book if you want a link: https://media.digitalarkivet.no/view/166325/4 https://media.digitalarkivet.no/view/166325/4 But every time I find something that I think "This one is surely easy enough!" it's usually wrong. US revolutionary war cursive is on the easy end; it's quite similar to modern cursive, and what is of training data in the datasets is probably a lot like it, if not actually in it - historians who test this complain about overfitting. Which I believe, because as I said, I see that it's bad at generalizing.