5 ms·

It's a dreadfully common story: some ugly hack which was necessary at the time persists for decades (or centuries, in the case of legal/social systems), because

by TillE 5y ago

It's a dreadfully common story: some ugly hack which was necessary at the time persists for decades (or centuries, in the case of legal/social systems), because nobody can really be bothered to change it. The extra file size is not much of an issue, but it does mean we still require stupid workarounds with Git and anything else that needs to handle cross-platform plaintext.

chrismorgan 5y ago
But it’s still worse when the ugly hack was only useful for a very short time, probably wasn’t even necessary, and ruins other things by existing. UTF-16 is my favourite to complain of: created because a few large companies had put a couple of years of heavy investment into UCS-2—which was a massive breaking change itself—and didn’t want to let that go to waste (with all the bother they’d been putting users through), even though by the time they actually made UTF-16, UTF-8 with its ASCII compatibility had been invented. And so they ruined Unicode for everyone for ever after with that accursed abomination surrogates and the distinction between code points and scalar values.
- colejohnson66 5y ago
  And now Unicode have declared themselves kneecapped to ~20 bits (1,111,998) because of it. I doubt we'll run out of space like we did with 16 bits, but never say never. The whole reason UCS-2 was used was companies saying "16 bits is enough" and then it not actually being enough. Almost every issue with Unicode is because developers made assumptions, despite Unicode themselves providing guidance saying otherwise.
  - layer8 5y ago
    Almost every issue with software in general is because developers made assumptions.
wrs 5y ago
How is this a "hack"? If anything, not using CRLF line endings is the hack, because it means you can't just send the file straight to a terminal: the line endings have to be translated by something on the way to the terminal for it to display properly.
- moring 5y ago
  The "hack" is to wrongly define "move down, move left" to be the same as "line ending". It isn't. The first are commands to a machine, the second are elements of encoding text in binary storage. Or more general, the "hack" is to act as if ASCII was a character encoding, when it's actually the encoding for a set of commands to a machine.
  - hnlmorg 5y ago
    It's both, and more. It's also document encoding (there's characters to encode structured data too). Given the era ASCII was invented, I wouldn't really call it a hack. Character encodings were one of the lowest level abstractions. The size of a byte wasn't always fixed to 8 bits and it was the number of bits in your character set that often defined how big your byte was (yes, I know ASCII is technically 7-bit but one of the original considerations was that the 8th bit would be a parity bit: https://en.wikipedia.org/wiki/Parity_bit https://en.wikipedia.org/wiki/Parity_bit ). It's also the reason the lowest size of addressable data is a byte / char. Bare in mind that we are talking about an era before operating systems as we recognize them. Back then each machine would have it's own operating system and some institutes would even have different operating systems despite having the same machines. Really these operating systems were more like firmware than what we have today. So there wasn't any distinction between character encodings and machine commands but there were differences between machines. So it made sense that ASCII was cross purpose because that's exactly what characters were: raw addressable data containing instructions. Those instructions were sometimes printable data, it was sometimes formatting data, and it was sometimes control data. I doubt any designers of ASCII could have even imagined the way we use computers these days when writing that specification. Even just the fact that character encodings have become an interchangeable abstraction for rendering text would have blown their minds. Never mind everything else.
wruza 5y ago
because nobody can really be bothered to change it Python 2 to 3 all the things, I guess? You’d then need another workaround choosing between CR (now obsolete), CRLF, LF and RS (renamed to LE), the latter being a true newline.
hnlmorg 5y ago
It wasn't a hack though. Characters were low level addressable data. It's no coincidence that on machines that had a 6-bit wide character set the smallest addressable data (a byte) was also 6-bits (I know ASCII is 7-bits but that was to reduce transmission overhead or allow for a parity bit when required, eg on punch tape or cards that often supported 8 columns of bits). The way machines worked back then would be you'd address a byte of data, that byte contained instructions. Those instructions were sometimes printable data, it was sometimes formatting data, and it was sometimes control data. It wasn't until operating systems became more sophisticated, hardware become more sophisticated and the two became decoupled, that character encodings became the abstraction for rendering text that we think of them as today.
- kps 5y ago
  Character codes predate computers. CR and LF codes date to 1901.
  - hnlmorg 5y ago
    Even earlier than that. CR and LF go back to Baudot code (which is where term 'baud' also originates) and that was created in the 1870s. https://en.wikipedia.org/wiki/Baudot_code https://en.wikipedia.org/wiki/Baudot_code
    - kps 5y ago
      Baudot's (ITA1) was linear, with no concept of lines. Murray's (which evolved into ITA2 with minor changes) added CR LF, since he developed a typewriter-like system.
      - hnlmorg 5y ago
        Ahhh I see where you've gotten that 1901 date from now: > In 1901, Baudot's code was modified by Donald Murray > The Murray code also introduced what became known as "format effectors" or "control characters" – the CR (Carriage Return) and LF (Line Feed) codes. (source: https://en.wikipedia.org/wiki/Baudot_code#Murray_code https://en.wikipedia.org/wiki/Baudot_code#Murray_code) Thanks for the correction there. Learned something new today :)

by TillE 5y ago

chrismorgan 5y ago

But it’s still worse when the ugly hack was only useful for a very short time, probably wasn’t even necessary, and ruins other things by existing. UTF-16 is my favourite to complain of: created because a few large companies had put a couple of years of heavy investment into UCS-2—which was a massive breaking change itself—and didn’t want to let that go to waste (with all the bother they’d been putting users through), even though by the time they actually made UTF-16, UTF-8 with its ASCII compatibility had been invented. And so they ruined Unicode for everyone for ever after with that accursed abomination surrogates and the distinction between code points and scalar values.

colejohnson66 5y ago
And now Unicode have declared themselves kneecapped to ~20 bits (1,111,998) because of it. I doubt we'll run out of space like we did with 16 bits, but never say never. The whole reason UCS-2 was used was companies saying "16 bits is enough" and then it not actually being enough. Almost every issue with Unicode is because developers made assumptions, despite Unicode themselves providing guidance saying otherwise.
- layer8 5y ago
  Almost every issue with software in general is because developers made assumptions.

wrs 5y ago

How is this a "hack"? If anything, not using CRLF line endings is the hack, because it means you can't just send the file straight to a terminal: the line endings have to be translated by something on the way to the terminal for it to display properly.

moring 5y ago
The "hack" is to wrongly define "move down, move left" to be the same as "line ending". It isn't. The first are commands to a machine, the second are elements of encoding text in binary storage. Or more general, the "hack" is to act as if ASCII was a character encoding, when it's actually the encoding for a set of commands to a machine.
- hnlmorg 5y ago
  It's both, and more. It's also document encoding (there's characters to encode structured data too). Given the era ASCII was invented, I wouldn't really call it a hack. Character encodings were one of the lowest level abstractions. The size of a byte wasn't always fixed to 8 bits and it was the number of bits in your character set that often defined how big your byte was (yes, I know ASCII is technically 7-bit but one of the original considerations was that the 8th bit would be a parity bit: https://en.wikipedia.org/wiki/Parity_bit https://en.wikipedia.org/wiki/Parity_bit ). It's also the reason the lowest size of addressable data is a byte / char. Bare in mind that we are talking about an era before operating systems as we recognize them. Back then each machine would have it's own operating system and some institutes would even have different operating systems despite having the same machines. Really these operating systems were more like firmware than what we have today. So there wasn't any distinction between character encodings and machine commands but there were differences between machines. So it made sense that ASCII was cross purpose because that's exactly what characters were: raw addressable data containing instructions. Those instructions were sometimes printable data, it was sometimes formatting data, and it was sometimes control data. I doubt any designers of ASCII could have even imagined the way we use computers these days when writing that specification. Even just the fact that character encodings have become an interchangeable abstraction for rendering text would have blown their minds. Never mind everything else.

wruza 5y ago

because nobody can really be bothered to change it Python 2 to 3 all the things, I guess? You’d then need another workaround choosing between CR (now obsolete), CRLF, LF and RS (renamed to LE), the latter being a true newline.

hnlmorg 5y ago

It wasn't a hack though. Characters were low level addressable data. It's no coincidence that on machines that had a 6-bit wide character set the smallest addressable data (a byte) was also 6-bits (I know ASCII is 7-bits but that was to reduce transmission overhead or allow for a parity bit when required, eg on punch tape or cards that often supported 8 columns of bits). The way machines worked back then would be you'd address a byte of data, that byte contained instructions. Those instructions were sometimes printable data, it was sometimes formatting data, and it was sometimes control data. It wasn't until operating systems became more sophisticated, hardware become more sophisticated and the two became decoupled, that character encodings became the abstraction for rendering text that we think of them as today.

kps 5y ago
Character codes predate computers. CR and LF codes date to 1901.
- hnlmorg 5y ago
  Even earlier than that. CR and LF go back to Baudot code (which is where term 'baud' also originates) and that was created in the 1870s. https://en.wikipedia.org/wiki/Baudot_code https://en.wikipedia.org/wiki/Baudot_code
  - kps 5y ago
    Baudot's (ITA1) was linear, with no concept of lines. Murray's (which evolved into ITA2 with minor changes) added CR LF, since he developed a typewriter-like system.
    - hnlmorg 5y ago
      Ahhh I see where you've gotten that 1901 date from now: > In 1901, Baudot's code was modified by Donald Murray > The Murray code also introduced what became known as "format effectors" or "control characters" – the CR (Carriage Return) and LF (Line Feed) codes. (source: https://en.wikipedia.org/wiki/Baudot_code#Murray_code https://en.wikipedia.org/wiki/Baudot_code#Murray_code) Thanks for the correction there. Learned something new today :)