3 ms·

> Currently, we simply ignore or don't understand the fact that the meaning of a sentence is not solely built with word order Many people argue convincingly th

• bayindirh 3y ago
  If convincingly arguing about something was the only thing we needed for proving correctness, we wouldn't have the scientific method. Or, if word order built the meaning and it's devoid of the character itself, we wouldn't be praising authors for their character building skills and embedding things not written or spoken into their stories, regardless of the medium it's presented.
  • pbhjpbhj 3y ago
    And your proof?
  • orbital-decay 3y ago
    To apply the scientific method and prove correctness, you need to define the subject first. Meaning is ill-defined, just like consciousness, intelligence, and others. LLMs can clearly grasp higher order abstractions and concepts just by reordering the words. In fact, embeddings (developed much earlier) are specifically intended to represent the semantical meaning extracted from text using just statistical methods; until the introduction of transformers they lacked a good architecture to demonstrate the usefulness of that. This makes many people argue that being a sentence builder is enough to be intelligent, as they also received most of their intelligence from the same source (concentrated experience of someone else - social intelligence). "Honesty" and other forms of self-introspection is just a high level construct which current models aren't trained for, and likely not a fundamental issue with being a stochastic parrot.