6 ms·

> > The short version of what's going on is that when they put the code into GitHub, GitHub used UTF-8 for encoding, and DOS uses Code Page 437. I don’t think

> > The short version of what's going on is that when they put the code into GitHub, GitHub used UTF-8 for encoding, and DOS uses Code Page 437. I don’t think that’s the actual problem. GitHub does not change the encoding of files in Git repos. Nor (generally speaking) does Git itself [0] It appears most of the encoding issues were introduced by manually editing files, in response to a corporate policy that certain comments be censored (mainly comments that could be viewed as unprofessional). While it is possibly to do this without altering the encoding of the files, it appears the person who did it used an editor that rewrote the file encoding, and didn’t realise it was doing that [0] Git can be configured to do automatic conversion between bare LF and CRLF newline formats. That’s generally a bad idea. It is mainly people on Windows who turn it on, although nowadays many Windows tools can handle Unix style line endings without issue, making it largely unnecessary. Also, you can install Git repository hooks that alter file contents as they are being committed, although the majority of repos don’t have any such hooks installed

rjst01 3y ago
Some tools - even cross-platform ones - will create CRLF files on Windows by default (looking at you JetBrains) so when working on Windows I usually like to have autocrlf set to 'input' to avoid accidentally committing CRLF files into the repository.
- _mlbt 3y ago
  Respecting platform conventions is the sensible default, particularly for a platform that represents over 80% of desktop marketshare. This problem used to be a whole lot worse under pre-OS X versions of macOS that used CR (just CR, no LF) as the line separator. At least now there are only 2 commonly used conventions and you can essentially just ignore any CRs you encounter in most cases.
  - skissane 3y ago
    > This problem used to be a whole lot worse under pre-OS X versions of macOS that used CR (just CR, no LF) as the line separator. That bare CR newline convention used to be very widespread – not just Classic MacOS, also many 8-bit micros (Commodore, Acorn, Apple II, TRS-80, ZX Spectrum, HP-85), Oberon, MIT Lisp Machines and Microware OS-9. But, by the late 1990s, Classic MacOS was the only one of those systems with any mainstream significance. And now you'll only encounter bare CR newlines in retrocomputing or obscure legacy systems – oh, and raw mode terminal input.
  - rjst01 3y ago
    > Respecting platform conventions is the sensible default That's fair, but I'd argue an equally sensible default would be to respect the conventions of the language ecosystems you're working within, and for Java, Python, Rust, that's LF. IntelliJ provides a lot of configuration on a per-language basis but line separators for new files is a global setting for some reason.
    - skissane 3y ago
      I have a few toy programming languages I created as a hobby / learning exercise. (Most of them I’ve never released, maybe some day.) In one of them, I decided to make carriage returns a lexical error. In fact, the only C0 control it allows in source text is LF, it doesn’t even allow tabs (my personal answer to the perennial spaces-vs-tabs debate).
ComputerGuru 3y ago
Ok, but again, why is this codepage workaround only required under FreeDOS?

rjst01 3y ago

Some tools - even cross-platform ones - will create CRLF files on Windows by default (looking at you JetBrains) so when working on Windows I usually like to have autocrlf set to 'input' to avoid accidentally committing CRLF files into the repository.

_mlbt 3y ago
Respecting platform conventions is the sensible default, particularly for a platform that represents over 80% of desktop marketshare. This problem used to be a whole lot worse under pre-OS X versions of macOS that used CR (just CR, no LF) as the line separator. At least now there are only 2 commonly used conventions and you can essentially just ignore any CRs you encounter in most cases.
- skissane 3y ago
  > This problem used to be a whole lot worse under pre-OS X versions of macOS that used CR (just CR, no LF) as the line separator. That bare CR newline convention used to be very widespread – not just Classic MacOS, also many 8-bit micros (Commodore, Acorn, Apple II, TRS-80, ZX Spectrum, HP-85), Oberon, MIT Lisp Machines and Microware OS-9. But, by the late 1990s, Classic MacOS was the only one of those systems with any mainstream significance. And now you'll only encounter bare CR newlines in retrocomputing or obscure legacy systems – oh, and raw mode terminal input.
- rjst01 3y ago
  > Respecting platform conventions is the sensible default That's fair, but I'd argue an equally sensible default would be to respect the conventions of the language ecosystems you're working within, and for Java, Python, Rust, that's LF. IntelliJ provides a lot of configuration on a per-language basis but line separators for new files is a global setting for some reason.
  - skissane 3y ago
    I have a few toy programming languages I created as a hobby / learning exercise. (Most of them I’ve never released, maybe some day.) In one of them, I decided to make carriage returns a lexical error. In fact, the only C0 control it allows in source text is LF, it doesn’t even allow tabs (my personal answer to the perennial spaces-vs-tabs debate).

ComputerGuru 3y ago

Ok, but again, why is this codepage workaround only required under FreeDOS?