3 ms·
Author here, you are correct! The issue here is due to the fact that a single user-perceived character might span into multiple tokens. This should be fixed now
by dqbd 3y ago
Author here, you are correct! The issue here is due to the fact that a single user-perceived character might span into multiple tokens. This should be fixed now.
- egorfine 3y agoHey. Thank you! However has the fix not been deployed yet? Still shows broken UTF-8. > a single user-perceived character might span into multiple tokens Is this the way it works as designed or is this a bug?