3 ms·
Forgive me if this is a naive assumption, but wouldn’t large language models be fundamentally different for a language that is largely symbols? Again, my unders
by cyberge99 5mo ago
Forgive me if this is a naive assumption, but wouldn’t large language models be fundamentally different for a language that is largely symbols?
Again, my understanding of Mandarin is limited if it exists at all.
- doph 5mo agoAll tokens are symbols. All of the frontier models speak Mandarin.
- boothby 5mo agoThis is why misspellings and homophones are tells of human righting. LLMs strongly prefer word-level tokens, and word substitutions follow semantic similarity and not the more human auditory similarity.
- mejutoco 5mo ago> righting. > LLMs strongly prefer word-level tokens, and word substitutions follow semantic similarity and not the more human auditory similarity. Is this an elaborate joke or your full-word misspelling of writing is both agreeing with your statement (word substitutions) and contradicting it (not semantic but only pronunciation similarity)
- calfuris 5mo agoI don't see the contradiction, unless you believe that the grandparent comment was written by an LLM.
- deleted 5mo ago[deleted]
- omneity 5mo agoFunny, I’ve been cracking[0] at this exact problem with a purpose-built model[1]: 0: https://huggingface.co/posts/omarkamali/593639295164067 https://huggingface.co/posts/omarkamali/593639295164067 1: https://omneitylabs.com/models/sawtone https://omneitylabs.com/models/sawtone
- jddj 5mo agoClaude the other day wrote code where one of the bytes in the array was 0xO5. That's zero ex oh (the letter) five
- wat10000 5mo ago"飞机" and "airplane" aren't fundamentally different in terms of how they're represented to a computer. Especially for an LLM, where tokenization likely turns each of those into a single token.