3 ms·

Also, I can't type ™ (that's a (tm)) without  appearing before it. Unicode problem? To reproduce, go to a text box and hold ALT. Press 00153 on the number k

• pg 19y ago
  If anyone knows how to fix this, let me know. Someone told me I could fix it by adding a tag saying meta http-equiv="content-type" content="text/html; charset=UTF-8" within the head on every page, but this had no effect. Would appreciate it if anyone could tell me what I'm doing wrong.
  • deleted 19y ago
    [deleted]
  • abstractbill 19y ago
    Setting the charset on each page is necessary but not sufficient. My guess: you also need to tell the browser to send form data as utf8. Perhaps by specifying "accept-charset" as UTF-8 in each form tag.
    • euccastro 19y ago
      If that works it would be better than guessing. :) Since you are sanitizing the input string, I still recommed translating to native mzScheme/Arc unicode, doing the transformations in that format, then retranslating back to utf-8 for display. The call to get a canonical string from utf-8 is: (bytes->string/utf-8 input-txt)
      • euccastro 19y ago
        And if I may abuse this thread to add one request, please store the raw string as you got it from the browser, and do the processing before displaying it (unless you determine that would be a security risk or a performance killer). If I use ampersand tags or other formatting to write a comment, they are lost when I edit it. It's kinda irritating to lose the indentation of a code sample (lots of  s) when editing out a mistake. :)
  • euccastro 19y ago
    The problem is in your input handling. You are getting latin-1 value for the (tm) sign (\x99, decimal 156), and you are treating that as if it was some unicode code point 156, which AFAIK is nonsense. When you try to encode that bogus thing in utf-8 you get \xc3\x99, which in latin-1 is A-with-circumflex followed by the TM sign, while in utf-8 it's still nonsense. What you need to do is decode your input from the remote system encoding into your native unicode representation, before encoding it in utf-8 to display it. The first problem is detecting the native encoding at the other end. I don't know how to do this, but if you assume latin-1 to begin with, it will still be an improvement over the current situation. Once you know/guess the encoding, you normalize it with something like this (in mzScheme): (bytes->string/latin-1 remote-txt) Then you convert to utf-8 like this (you are already doing this): (string->bytes/utf-8 stored-string) Finally, remember to put back the meta tag. HTH.
• deleted 19y ago
  [deleted]