13 ms·
When it comes to tokenization, yeah, it does differentiate. But its training set is certain to have many examples of different capitalizations and misspellings,
by int_19h 4y ago
When it comes to tokenization, yeah, it does differentiate. But its training set is certain to have many examples of different capitalizations and misspellings, so it can understand them, too. Which is usually what you want...
GPT-4 really takes this to eleven. For example, it can correctly parse and process this request: "Tll m shrt str bt ncrns; 10 sntncs. Wrt t l vwls prprl."