3 ms·
You can observe (what I assume is) the same tokenization phenomenon in people who are struggling to speak (for example because they’re distracted by something o
by wolfgang42 3y ago
You can observe (what I assume is) the same tokenization phenomenon in people who are struggling to speak (for example because they’re distracted by something or not native speakers): stock fragments will come out all at once, and less common words will get split, usually on affixes or at the join point of compound words.