4 ms·
Is there a clever way to remove all the space characters? https://www.cs.tut.fi/~jkorpela/chars/spaces.html https://www.cs.tut.fi/~jkorpela/chars/spaces.html
by criddell 10y ago
Is there a clever way to remove all the space characters?
https://www.cs.tut.fi/~jkorpela/chars/spaces.html https://www.cs.tut.fi/~jkorpela/chars/spaces.html
- gravypod 10y ago"ZERO WIDTH NO-BREAK SPACE foobar No width (the character is invisible)" Time to insert this into some of my friends code.
- squeaky-clean 10y agoI've actually joked with colleagues about this, and often it doesn't work. Try using the U+FEFF in Javascript, in Chrome it's taken as a space (but the console will mark it as a red circle, not an invisible no-width character), and Firefox refuses to even allow it to be entered. You can also use a character and combining character to make a symbol that looks exactly like a semicolon, but technically isn't. Browsers treat it as a semicolon anyways. I wanted to be able to do `var ; = "foobar";` Maybe it will sneak by other languages that allow unicode in the code? Though I don't know of many.
- nitrogen 10y agoI've had problems with code having non-ASCII spaces appended, and the terminal rendering them as zero-width, so I had to use hexdump to find the problem.
- sedachv 10y agoAlso fun are left-to-right and right-to-left marks.
- squeaky-clean 10y agoThis is a level of evil I hadn't considered.
- sedachv 10y agoAs Seen in Production™
- pjscott 10y agoIf you use the naive algorithm at the top of the page but iterate over UTF-8 code points rather than bytes -- which is straightforward, BTW -- you will get some cleverness automatically in the compiler's implementation of the switch statement. I wrote a switch to handle the Unicode "is it space?" function, with the values from the table you linked, and compiled with "clang -Os". You know what it did? It generated a friggin' binary search tree! Some of the leaves were a sequence of straight-line comparisons, because that's more compact, but the higher levels were all a bunch of "if (c > 0x167F) { ... }" sort of code. At one point it subtracts 8192 from something and then compares with 12, and I think this is because the x86 instruction encoding is shorter and the compiler knows that the register won't be needed again along either of the code paths from that point. Compilers are amazing sometimes.
- userbinator 10y agoCompiling switches into binary (and even sometimes trinary) trees is an old technique that's been around since at least the days of 16-bit DOS... they will usually choose between a single-level lookup, double-level lookup, or tree depending on sparsity and options used. Subtraction and addition implicitly sets the flags, so you can generate very small code with inc/dec (1-byte instructions), like this: dec ax jz ax_was_zero dec ax jz ax_was_one ...