3 ms·

I cut you off because you basically said "and if we see this thing that is defined as invalid, we treat it as invalid". Which is something that all UTF-8 parser

I cut you off because you basically said "and if we see this thing that is defined as invalid, we treat it as invalid". Which is something that all UTF-8 parsers do, so there's no point in calling it out like this is special behavior.

Dylan16807 9y ago
That's not me. The point is that this variant parser accepts many things that are normally errors and turns them into faux-surrogates, so it's worth restating that it rejects surrogates in the source file.
- eridius 9y ago
  My apologies. In any case, here's what OP said: > Well, that's not the full story: to have this transparency property, we also need to ensure that when some U+DCXX occurs by means of a valid UTF-8 pattern, we nevertheless treat it as invalid. I.e. there is a rule that if the UTF-8 decode works, but a U+DCXX code-point emerges, then we retreat to the start and drop a byte as a replacement character, as if a bad code had been seen. But this whole paragraph is wrongheaded. A U+DCXX codepoint cannot occur as a result of a UTF-8 decode, because it is defined as invalid. Even just that last bit there, "as if a bad code had been seen"... a bad code was seen! This paragraph makes me question whether OP actually understands UTF-8 decoding at all.
  - Dylan16807 9y ago
    I think the meaning is clear. They're talking about part of the UTF-8 decoding process, the part that actually decodes leading/trailing bytes into arbitrary 21 bit numbers. What term would you use for that?
    - eridius 9y ago
      Well, I wouldn't even bring it up, because the UTF-8 decoding process by definition cannot produce code points in the surrogate pair range, as those are illegal to encode in UTF-8. But if I must, I might talk about the "bit pattern" and the integral value that results from interpreting it. I certainly wouldn't talk about code points resulting from a UTF-8 decode.
      - eridius 9y ago
        Somebody is apparently upset about what I said. Why?

Dylan16807 9y ago

That's not me. The point is that this variant parser accepts many things that are normally errors and turns them into faux-surrogates, so it's worth restating that it rejects surrogates in the source file.

eridius 9y ago
My apologies. In any case, here's what OP said: > Well, that's not the full story: to have this transparency property, we also need to ensure that when some U+DCXX occurs by means of a valid UTF-8 pattern, we nevertheless treat it as invalid. I.e. there is a rule that if the UTF-8 decode works, but a U+DCXX code-point emerges, then we retreat to the start and drop a byte as a replacement character, as if a bad code had been seen. But this whole paragraph is wrongheaded. A U+DCXX codepoint cannot occur as a result of a UTF-8 decode, because it is defined as invalid. Even just that last bit there, "as if a bad code had been seen"... a bad code was seen! This paragraph makes me question whether OP actually understands UTF-8 decoding at all.
- Dylan16807 9y ago
  I think the meaning is clear. They're talking about part of the UTF-8 decoding process, the part that actually decodes leading/trailing bytes into arbitrary 21 bit numbers. What term would you use for that?
  - eridius 9y ago
    Well, I wouldn't even bring it up, because the UTF-8 decoding process by definition cannot produce code points in the surrogate pair range, as those are illegal to encode in UTF-8. But if I must, I might talk about the "bit pattern" and the integral value that results from interpreting it. I certainly wouldn't talk about code points resulting from a UTF-8 decode.
    - eridius 9y ago
      Somebody is apparently upset about what I said. Why?