3 ms·

More programming languages/platforms need to include built-in routines for various kinds of unicode sanitization.

• cryptonector 5y ago
  Yes.
  • deleted 5y ago
    [deleted]
• kazinator 5y ago
  Or maybe Unicode shouldn't include this crap in the first place? If you "sanitize" Unicode you're just removing that functionality. Remove that character, and the right to left reversal wont' be there, right? If you have to break it in the name of security and sanity, it shouldn't exist in the first place.
  • jrochkind1 5y ago
    The functions/codepoints are there for reasons. If you want to research the reasons and decide that text processing doens't need them, that's one thing. But to simply dismiss it off-hand with the argument "if you have to remove it for security it shouldn't exist in the first place" is just ridiculous. It's like saying "if you have to properly escape parameters in SQL to avoid injection attacks, SQL shouldn't exit in the first place!" I am continually baffled about why the HN comment sections are such a hotbed of anti-unicode sentiment. Perhaps most of y'all don't have to write software that deals with non-English text in any significant way. If you just didn't have buffers you wouldn't have to avoid buffer overflow attacks! Come on.
    • kazinator 5y ago
      > if you have to properly escape parameters in SQL to avoid injection attacks, SQL shouldn't exit in the first place!" Yes, this aspect of SQL is poorly conceived because the only way is to manipulate it with text processing. Parameters should be inserted into a query language using AST manipulation, like a Lisp backquote. Now within a particular program, we can wrap something like that around SQL generation, and solve the escaping problems in one place. That approach doesn't apply to character data, which has to be literal and primitive in many situations.