4 ms·
What about \n, which decodes to 0x0ah - two bytes to one byte? I can't think of any others like that, but newline in different OS's get represented in various
by ksaj 3y ago
What about \n, which decodes to 0x0ah - two bytes to one byte?
I can't think of any others like that, but newline in different OS's get represented in various different ways (like DOS's famous CRLF, which adds an 0x0dh to the recipe). Operating systems that use 0x0ah alone could very well be the only example.
- dietr1ch 3y agoNote that while in your compiler you don't really care about the representation that the OS expect in real files. It's ok for your intermediate, in-memory representation to be meaningless to the OS. Also, maybe it's not a literal character for character replacement. Since lengths are probably changing, your pre-processed input might be shorter (or longer if you truly need to expand some escaped chars (and you have a program that just wants to break your compiler by abusing that escape sequence :P))
- ksaj 3y agoYes, but some OS's use 0x0ah alone, which is half the bytes of \n representation. People from the 90's will easily remember navigating the ^M (which is the 0x0dh showing up) showing up at the end of every line in a DOS text file when viewing them under a Linux text editor.
- dietr1ch 3y agoThis is a different problem around how to interpret new lines from a Windows file (\r\n) or maybe an old Mac version (\n\r to keep things incompatible I guess). If you are not happy with the OS's choice for new lines, you can pre-process that when reading the input as the very first stage. This is what I meant when I said that your compiler doesn't really care about the OS's semantics once the file is in memory, you only need to respect them if you are writing intermediate state to files (like some IR). The article is talking about how to handle escaped characters in the source, like literal `\n` you can see on `let hi = "Hello\nWorld\nfrom my compiler!"` using your average text editor (0x5c+0x6e in ASCII), and how to build an efficient string type that mostly borrows views from the in-memory version of the source file (which could be a pre-processed version of the raw file if you hate odd line breaks).
- zarzavat 3y agoFWIW the Classic Mac line ending is \r. \n\r was used by Acorn, apparently.
- Joker_vD 3y agoBefore the UNIX had the bright idea to represent newlines as single LFs in disk files and then write a bunch of code in the kernel for the tty driver to convert LF to and from CRLF pairs (which is what ttys actually used), everybody else used CRLF pair to indicate a linebreak. That's why almost all network protocols use CRLF, and not because Microsoft invented the time machine and travelled into the seventies to hinder the UNIX adoption.
- jfk13 3y ago> everybody else used CRLF pair to indicate a linebreak Almost everybody... https://en.wikipedia.org/wiki/Newline#Representation https://en.wikipedia.org/wiki/Newline#Representation mentions a couple that apparently used LFCR.
- chrismorgan 3y agoReread the suggestion, you’ve got it back to front: > no escape sequence is shorter than the text it represents The idea is that you unescape immediately, and if necessary shuffle the remainder back, leaving a blank space at the end, but able to reuse the initial allocation. Also, I don’t even know of anything that actually turns a \n into {CARRIAGE RETURN, LINE FEED}; if you want that, you’ll need \r\n. (A very few situations are a bit fuzzy about it, e.g. \r and \n are a bit wonky in Vim’s :substitute, but that’s all I can think of).
- lelanthran 3y agoIsn't the two byte escape sequence not shorter than the one byte newline? I do inline replacement of escaped values all the time, because the sequence is always longer than the replacement.
- hansvm 3y agoAnd exponentially shorter with nested escapes.