4 ms·
Words are lossy compressor, LMs speaks in meanings
- allears 1y agoOnce the author started talking about LLMs having a "deeper understanding" I stopped reading.
- retr0reg 1y agoThis high dimensional vector space does allows them to theoretically embed a interpretable "deeper understanding", reference https://www.youtube.com/watch?v=wjZofJX0v4M&t=624s https://www.youtube.com/watch?v=wjZofJX0v4M&t=624s. It's the pre-train process that determines whether or not they demonstrate this "deeper understanding". This "deeper understanding" refers to the understanding to the word, can be contextual, pragmatic, or semantic.