3 ms·

The fact that LLMs can recognise ASCII-art letters is actually the most impressive part of this story to me! Yes, they're trained on text, but they mostly only

The fact that LLMs can recognise ASCII-art letters is actually the most impressive part of this story to me! Yes, they're trained on text, but they mostly only "see" byte sequences. They don't "see" letter shapes in the text they're given. I wonder what parts of their training data helped with that skill?

formerly_proven 3y ago
This actually surprised me quite a bit considering how tokenization works.
JadoJodo 3y ago
I’m not sure that they actually are able to. From the article: > Although LLMs find it difficult to recognize specific words represented as ASCII art, they have the ability to infer what such a word might be based on the text content in the remainder of the input statement.
- Sharlin 3y ago
  Yep, they don't. If the prompt hadn't contain a sentence that essentially gives away the missing word, they'd have almost certainly given some nonsense answer.
Sunhold 3y ago
I tried with GPT-4 and it can't recognize the font shown in the article without context.[1] (You can generate it as as the 'block' font using FIGlet or the many online generators that use it.[2]) [1] https://chat.openai.com/share/72d5e89d-1233-4078-8856-40f2130f438a https://chat.openai.com/share/72d5e89d-1233-4078-8856-40f213... [2] https://patorjk.com/software/taag/ https://patorjk.com/software/taag/
ramses0 3y ago
Reminds me of an old "prank" story where some guy convinced his coworker he could write a program that would recognize ascii art (eg: "figlet"). ...so they both go their separate ways, the innocent coworker couldn't come up with anything, the prankster let you paste in something to the program and "a few minutes to hours later" it would spit out what it recognized! An amazing feat of engineering, etc, etc. Co-worker tries to show it to a friend a few weeks later, prankster had to confess that it just emailed him the text, he would respond (manually) with the answer, and the program would insert a plausible/configurable delay before showing the (human-generated) response of "interpreting figlet". Now? Just feed it to an LLM.
- Y_Y 3y ago
  There is a relatively straightforward technique for this that you can use to invert operations like blurring on well-constrained sets. Just generate every letter (in every font) and compare with the start of the input. If you get a good match cut that part off and start over.

formerly_proven 3y ago

This actually surprised me quite a bit considering how tokenization works.

JadoJodo 3y ago

I’m not sure that they actually are able to. From the article: > Although LLMs find it difficult to recognize specific words represented as ASCII art, they have the ability to infer what such a word might be based on the text content in the remainder of the input statement.

Sharlin 3y ago
Yep, they don't. If the prompt hadn't contain a sentence that essentially gives away the missing word, they'd have almost certainly given some nonsense answer.

Sunhold 3y ago

I tried with GPT-4 and it can't recognize the font shown in the article without context.[1] (You can generate it as as the 'block' font using FIGlet or the many online generators that use it.[2]) [1] https://chat.openai.com/share/72d5e89d-1233-4078-8856-40f2130f438a https://chat.openai.com/share/72d5e89d-1233-4078-8856-40f213... [2] https://patorjk.com/software/taag/ https://patorjk.com/software/taag/

ramses0 3y ago

Reminds me of an old "prank" story where some guy convinced his coworker he could write a program that would recognize ascii art (eg: "figlet"). ...so they both go their separate ways, the innocent coworker couldn't come up with anything, the prankster let you paste in something to the program and "a few minutes to hours later" it would spit out what it recognized! An amazing feat of engineering, etc, etc. Co-worker tries to show it to a friend a few weeks later, prankster had to confess that it just emailed him the text, he would respond (manually) with the answer, and the program would insert a plausible/configurable delay before showing the (human-generated) response of "interpreting figlet". Now? Just feed it to an LLM.

Y_Y 3y ago
There is a relatively straightforward technique for this that you can use to invert operations like blurring on well-constrained sets. Just generate every letter (in every font) and compare with the start of the input. If you get a good match cut that part off and start over.