6 ms·

This a cool exploration of technology, and I don't want to take away from that. > The program was efficient enough to run in real-time on a IBM System/360 comp

This a cool exploration of technology, and I don't want to take away from that. > The program was efficient enough to run in real-time on a IBM System/360 computer, and robust enough to properly identify 90 percent of the symbols drawn by first-time users. I just want to point out that 90% accuracy is, from a user's point of view, awful handwriting recognition performance. It means you will be correcting on average about 10 words per paragraph! Even 99% percent accuracy is not nearly good enough to give people a sense that the computer is good at handwriting recognition. I also want to point out the difficulty and danger in interpreting strokes when doing handwriting recognition. In the last demo box, try writing a capital Y without lifting the pen. You'll have to go "up and down" one or both upper branches. Because of this, the recognizer will call it a K, A, or N even though it is obviously a Y when you're done. This demo is constrained to only using one stroke per letter, but systems that permit multiple strokes still get into trouble when the strokes don't match what they are expecting--for example if you draw an X using 4 individual strokes outward from a central point. This also happens with words. In Microsoft's handwriting recognition in Office in the early 2000s, writing the letters of a word out of order completely borked the recognition. For example writing "xample" and then going back and adding an "e" at the beginning would not produce a recognized word of "example." My point with all of this is that there is a reason you probably don't do all your computing with natural handwriting. It's a surprisingly difficult problem. Users do not expect it to matter how they form letters and words on the page. And they have very low tolerance for correcting computer mistakes.

scotu 8y ago
for many of the examples you gave, I think that could be solved through an autocomplete style correction that sure, it's not perfect, but it seems good enough for smartphone users: xample is not a word, so it's probably a typo, so it's probably example... you could also keep multiple interpretation of a word pending (and a text search for all of them would take you there) and eventually ask the user to disambiguate if the user wants to. I assume this would be an acceptable solution for non dictionary words too...
- deleted 8y ago
  [deleted]
taeric 8y ago
I don't really disagree, but I think you overstate it, to an extent. For most people, simple 99% accuracy of their input on their phone's system of capture is probably overstating it. There is a reason people have the clever footers "written on phone." That is to say, people have a higher tolerance for things that are within expected norms of their environment. Ideally, we want no corrections. But, having to do them constantly for a time will quickly desensitize people to this. (And yes, this is currently just an assertion of mine, I don't have data backing it. Just some anecdotes.)
- SmellyGeekBoy 8y ago
  I always saw the "Sent from my iPhone" footer as nothing more than advertising, and the ensuing "Sent from my x" as a small act of rebellion or tongue-in-cheek reference. I hadn't considered that it was intended to act as a warning that the content might be more error-prone.
  - taeric 8y ago
    I've seen a few that were direct statements of more typos because of the device used. Probably did start and largely remain advertising, though.
defgeneric 8y ago
> This demo is constrained to only using one stroke per letter, but systems that permit multiple strokes still get into trouble when the strokes don't match what they are expecting--for example if you draw an X using 4 individual strokes outward from a central point. Arguably, an X drawn this way should NOT be recognized as an X--that's not how an X is spelled. If the task is communicating with the computer, then recognition of the gesture is a valid approach. Just as there are conventions regarding the spelling of words, there are conventions involved in the formation of letters. Why not use them? It would even seem incorrect to leave these out.
- snowwrestler 8y ago
  The human convention of written language it to interpret the symbols after they have been completed, not during the act of writing them. A computer that interprets the behavior of writing, rather than the final symbols, is going to violate user expectations at some point. Why? Because people do not always write as linearly as you might expect, especially when writing fast. They might drop or mis-write letters or words, then go back and fix it. Or quickly jot down just enough letters to remind themselves of what they heard, then go back and fill the rest in. A routine that interprets actions in order is going to have a hard time with actions that the user completes out of order.
  - 1787 8y ago
    "human convention of written language" is a bit much. Stroke order is almost as important as what the actual strokes are in the definition of a Chinese character, for example. Of course unless you literally watch someone write you observe the characters after they're written, but the most predictive latent mental representation of a character does include an order component. I know this because I made the mistake of memorizing many characters almost like bitmaps and have had to go back and learn how to reliably write/read hand written characters.
    - taeric 8y ago
      Even learning English the stroke order can help considerably. Mostly it helps in just learning to write legibly, but that really is just a fancy way of saying it increases the accuracy of recognition. :) I completely concede that it is possible to get the same results with other stroke orderings. However, there is a reason when teaching children how to write, we often get fairly prescriptive with stroke ordering, as well.
unwind 8y ago
I just want to point out that 90% accuracy is, from a user's point of view, awful handwriting recognition performance. It means you will be correcting on average about 10 words per paragraph! Wait, what? Doesn't that imply that a paragraph needs to have 100 words in it, in order for 10 of them to be recognized wrong at 90% success rate? That seems super-long, anyway. My stats are really rusty, perhaps that's just one of those unintuitive cases that confuse people like me.
- snowwrestler 8y ago
  It's a (somewhat dated, probably) copyediting rule of thumb that a written paragraph has about 100-200 words in it. This would be in a writing style you might see in a novel or an essay. For online writing, perhaps more like 50-100. Even that might be long for the style of writing where each sentence is its own paragraph, supposedly for impact or whatever. Not sure you can really call it "paragraphs" when each one is only a sentence. For reference, the above paragraph is 78 words long.

scotu 8y ago

for many of the examples you gave, I think that could be solved through an autocomplete style correction that sure, it's not perfect, but it seems good enough for smartphone users: xample is not a word, so it's probably a typo, so it's probably example... you could also keep multiple interpretation of a word pending (and a text search for all of them would take you there) and eventually ask the user to disambiguate if the user wants to. I assume this would be an acceptable solution for non dictionary words too...

deleted 8y ago
[deleted]

taeric 8y ago

I don't really disagree, but I think you overstate it, to an extent. For most people, simple 99% accuracy of their input on their phone's system of capture is probably overstating it. There is a reason people have the clever footers "written on phone." That is to say, people have a higher tolerance for things that are within expected norms of their environment. Ideally, we want no corrections. But, having to do them constantly for a time will quickly desensitize people to this. (And yes, this is currently just an assertion of mine, I don't have data backing it. Just some anecdotes.)

SmellyGeekBoy 8y ago
I always saw the "Sent from my iPhone" footer as nothing more than advertising, and the ensuing "Sent from my x" as a small act of rebellion or tongue-in-cheek reference. I hadn't considered that it was intended to act as a warning that the content might be more error-prone.
- taeric 8y ago
  I've seen a few that were direct statements of more typos because of the device used. Probably did start and largely remain advertising, though.

defgeneric 8y ago

> This demo is constrained to only using one stroke per letter, but systems that permit multiple strokes still get into trouble when the strokes don't match what they are expecting--for example if you draw an X using 4 individual strokes outward from a central point. Arguably, an X drawn this way should NOT be recognized as an X--that's not how an X is spelled. If the task is communicating with the computer, then recognition of the gesture is a valid approach. Just as there are conventions regarding the spelling of words, there are conventions involved in the formation of letters. Why not use them? It would even seem incorrect to leave these out.

snowwrestler 8y ago
The human convention of written language it to interpret the symbols after they have been completed, not during the act of writing them. A computer that interprets the behavior of writing, rather than the final symbols, is going to violate user expectations at some point. Why? Because people do not always write as linearly as you might expect, especially when writing fast. They might drop or mis-write letters or words, then go back and fix it. Or quickly jot down just enough letters to remind themselves of what they heard, then go back and fill the rest in. A routine that interprets actions in order is going to have a hard time with actions that the user completes out of order.
- 1787 8y ago
  "human convention of written language" is a bit much. Stroke order is almost as important as what the actual strokes are in the definition of a Chinese character, for example. Of course unless you literally watch someone write you observe the characters after they're written, but the most predictive latent mental representation of a character does include an order component. I know this because I made the mistake of memorizing many characters almost like bitmaps and have had to go back and learn how to reliably write/read hand written characters.
  - taeric 8y ago
    Even learning English the stroke order can help considerably. Mostly it helps in just learning to write legibly, but that really is just a fancy way of saying it increases the accuracy of recognition. :) I completely concede that it is possible to get the same results with other stroke orderings. However, there is a reason when teaching children how to write, we often get fairly prescriptive with stroke ordering, as well.

unwind 8y ago

I just want to point out that 90% accuracy is, from a user's point of view, awful handwriting recognition performance. It means you will be correcting on average about 10 words per paragraph! Wait, what? Doesn't that imply that a paragraph needs to have 100 words in it, in order for 10 of them to be recognized wrong at 90% success rate? That seems super-long, anyway. My stats are really rusty, perhaps that's just one of those unintuitive cases that confuse people like me.

snowwrestler 8y ago
It's a (somewhat dated, probably) copyediting rule of thumb that a written paragraph has about 100-200 words in it. This would be in a writing style you might see in a novel or an essay. For online writing, perhaps more like 50-100. Even that might be long for the style of writing where each sentence is its own paragraph, supposedly for impact or whatever. Not sure you can really call it "paragraphs" when each one is only a sentence. For reference, the above paragraph is 78 words long.