3 ms·
> The problem, column is counted using UTF-16 code units. Rather than fixing the problem by using code points, they made it worse by allowing any encoding. Now
by hgs3 3y ago
> The problem, column is counted using UTF-16 code units.
Rather than fixing the problem by using code points, they made it worse by allowing any encoding. Now, not only do you need to support UTF-16, but also UTF-8 and UTF-32 [1].
[1] https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#positionEncodingKind https://microsoft.github.io/language-server-protocol/specifi...
- sillysaurusx 3y agoOr use UTF-8 everywhere to fix the problem. Code points have their own issues (as does UTF-8, but on balance these seem like better engineering tradeoffs).
- celeritascelery 3y agoUsing UTF-8 only has better engineering trade offs if you represent text as UTF-8. If you don’t (like JavaScript) then it’s just extra complexity. It’s no better than UTF-16. Only advantage it has is it’s more common among newer languages. Code points is the only thing that makes sense for a multi-language protocol. It’s unambiguous and every Unicode client can talk in code points, even if they use some exotic encoding.
- chrismorgan 3y agoYou don’t need to; UTF-16 is mandatory (ugh), but support for other encodings is negotiated (see ClientCapabilities.positionEncoding and ServerCapabilities.positionEncoding).
- layer8 3y agoThis seems somewhat pointless, because for compatibility you need to support UTF-16 anyway, so why bother implementing any of the other encodings in addition.
- chrismorgan 3y agoBecause if the server and client both use UTF-8 internally, they can get better performance by agreeing to use UTF-8.
- layer8 3y agoThe question is if the performance gains are noticeable enough to warrant the added implementation complexity and the added integration testing needed. It seems doubtful to me, given the general protocol overhead.