2 ms·
If convincingly arguing about something was the only thing we needed for proving correctness, we wouldn't have the scientific method. Or, if word order built t
by bayindirh 4y ago
If convincingly arguing about something was the only thing we needed for proving correctness, we wouldn't have the scientific method.
Or, if word order built the meaning and it's devoid of the character itself, we wouldn't be praising authors for their character building skills and embedding things not written or spoken into their stories, regardless of the medium it's presented.
- pbhjpbhj 4y agoAnd your proof?
- orbital-decay 4y agoTo apply the scientific method and prove correctness, you need to define the subject first. Meaning is ill-defined, just like consciousness, intelligence, and others. LLMs can clearly grasp higher order abstractions and concepts just by reordering the words. In fact, embeddings (developed much earlier) are specifically intended to represent the semantical meaning extracted from text using just statistical methods; until the introduction of transformers they lacked a good architecture to demonstrate the usefulness of that. This makes many people argue that being a sentence builder is enough to be intelligent, as they also received most of their intelligence from the same source (concentrated experience of someone else - social intelligence). "Honesty" and other forms of self-introspection is just a high level construct which current models aren't trained for, and likely not a fundamental issue with being a stochastic parrot.