30 ms·
> Being terse and allowing for human interpretation is kind of the entire point of something being written like this. It's also to a certain extent why LLMs wo
by AnthonyMouse 2mo ago
> Being terse and allowing for human interpretation is kind of the entire point of something being written like this.
It's also to a certain extent why LLMs work.
When IBM Watson was playing Jeopardy, one of the game prompts was:
> It was the anatomical oddity of U.S. gymnast George Eyser, who won a gold medal on the parallel bars in 1904
The man was missing a leg and used a prosthetic. Watson's output was, "What is leg?"
At first it was regarded as correct. If a human said that you could conclude that they knew the answer. But then the judges decided not to give Watson the point because its output didn't provide enough specificity to prove that it understood the context.
If you ask an LLM what kinds of things taste sweet it can give you examples like cotton candy or strawberries, but it has never actually tasted anything. All it knows is that the training data contains the association between those tokens. But the human reading the output knows what strawberries are, which is what allows the output to be meaningful.
- cwmoore 2mo agoHuman language requires a human receiver, like art requires an audience.
- AnthonyMouse 2mo agohttps://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno... > The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.
- cwmoore 2mo agoSweet source. I could have said more in one paragraph and 55 pages of formative theory than the two lines above.