5 ms·

But it also feels arbitrary and annoyingly restrictive. On top of that there are at least 25 whitespace codepoints in UTF. Should your language really be opinio

But it also feels arbitrary and annoyingly restrictive. On top of that there are at least 25 whitespace codepoints in UTF. Should your language really be opinionated about when, where, and in what order (for example) the "mongolian vowel separator" appears?

Mogzol 1mo ago
> and annoyingly restrictive How so? In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code? Let alone using esoteric Unicode whitespace characters for indentation. I think it is perfectly reasonable for the language to make the restriction that indentation must be either all tabs, tabs followed by spaces, or all spaces.
- fc417fc802 1mo ago
  > In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code? Writing a lisp in an editor that doesn't do boneheaded things with tabs? That's the only one I've personally run into but lack of imagination is hardly a good excuse to implement arbitrary restrictions. > Let alone using esoteric Unicode whitespace characters for indentation. How do you know what's esoteric in other countries? I certainly don't. I'm not an expert in linguistics but I'm sure that people everywhere in the world write computer programs at this point. > I think it is perfectly reasonable ... Without any concrete justification? Why would entirely artificial restrictions ever be seen as reasonable?
  - rmunn 1mo ago
    What "boneheaded things with tabs" are you referring to? The picture I'm piecing together from your comments suggests that you may use tab characters in a different way than most people seem to, so I'd quite like a further explanation of how you use tab characters and how you expect an editor to handle them.
    - fc417fc802 1mo ago
      > you may use tab characters in a different way than most people seem to As far as I'm aware there are three primary and entirely independent uses for tabs. Indentation, field separation, and typesetting. When it comes to typesetting and also to display of fields with a narrow maximum width the concept of a tabstop is useful. Boneheaded things with tabs was in reference to code editors (where tabs are more or less exclusively used for indentation) most often failing to provide the ability to disable tabstop. In practice this generally hasn't been a point of friction because until fairly recently the most popular languages (ie C & co) didn't support constructs that would lead to adding any indentation outside of the prefix of a line.
      - rmunn 1mo ago
        Agreed... but see my Lisp example in https://news.ycombinator.com/item?id=49593406 https://news.ycombinator.com/item?id=49593406 for a place where tabs end up unavoidably ambiguous. Because in that `cond` example the tabs look like they're used for indentation, but they're actually being used for alignment, something that falls more under typesetting than under indentation. A smart editor that has read the Lisp code (note that I'm using "read" in its Lisp meaning, i.e. "parsed into a syntax tree") can make those tab characters correctly align the forms the way you'd want them to be aligned for maximum understanding... but really, you would want space characters, not tab characters, in that context. So even though in most cases those three uses for tabs are independent, in some languages they get muddled. F# is another language where you may want to align text on line two with the middle of line one, e.g. if you write let person = { FirstName = "Bill" LastName = "Gates" } Now, that's considered bad style according to https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/formatting#avoid-formatting-that-is-sensitive-to-name-length https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/... — and they're entirely write to recommend against that style — but it's legal syntax. And it's probably why https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/formatting#use-spaces-not-tabs https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/... forbids tab characters entirely in F# code. (Those two links point to different anchors in the same document, BTW: the HN link shortening is going to make them look identical until you hover over them).
rmunn 1mo ago
I mean, obviously that one should only appear within Mongolian text and not within indentation. To state explicitly what should be implicitly obvious, there is no valid reason (that I'm aware of, I welcome any non-facetious correction) to use any character except U+0009 and U+0020 within indentation. Horizontal Record Separator? Zero-width joiner? Language-specific whitespace characters like your example? All make sense within human text (well, maybe not HRS), but in programming, they should be eschewed in favor of the characters that can be typed in every single keyboard layout in the world. Even languages that don't put spaces between words, such as Thai, still put spaces between sentences (or comma phrases) and therefore keep the space bar in their keyboard layout. And since mixing tabs and spaces (even between lines, where some lines are tab-indented and some are space-indented) creates problems for whitespace-sensitive language, there's a reason why every whitespace-sensitive language I'm aware of has tended to either outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations, which are usually not available to the validation code running in CI or on other people's machines).
- fc417fc802 1mo ago
  > To state explicitly what should be implicitly obvious, there is no valid reason ... There doesn't need to be an articulable reason. Or rather there's generally no expectation that a central authority will be able to reliably enumerate such. Everything should default to being permitted and only ever be restricted for good reason. But since you asked. U+2003 for example carries formatting information. Maybe an editor could be written (or even already exists) that would find that useful. Who is any third party to dictate that? U+00A0 similarly communicates information about the desired formatting and I can see no reason it would be unreasonable for someone to use it nor why its use should pose a technical challenge to a compiler. > creates problems for whitespace-sensitive langauge Does it? That seems like an invented problem to me. You have a running prefix composed of arbitrary whitespace characters. Any change in that prefix is a change in the level of indentation. You can add or remove arbitrary amounts from the end of the prefix. In the event you remove from it the result must exactly match the previous stack level. What's so complicated about this? > outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations ... Did you mix up your code points there? It's space that's ambiguous, not tab. Regardless I think that a compiler worrying about the specifics of text editors or other tooling would be backward information flow and a massive abstraction violation. Semantic meaning is entirely dictated by the compiler, not the other way around. There's no convincing reason (IMO) to impose restrictions that aren't technically necessary or to otherwise needlessly employ solutions that would reduce generalization.

Mogzol 1mo ago

> and annoyingly restrictive How so? In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code? Let alone using esoteric Unicode whitespace characters for indentation. I think it is perfectly reasonable for the language to make the restriction that indentation must be either all tabs, tabs followed by spaces, or all spaces.

fc417fc802 1mo ago
> In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code? Writing a lisp in an editor that doesn't do boneheaded things with tabs? That's the only one I've personally run into but lack of imagination is hardly a good excuse to implement arbitrary restrictions. > Let alone using esoteric Unicode whitespace characters for indentation. How do you know what's esoteric in other countries? I certainly don't. I'm not an expert in linguistics but I'm sure that people everywhere in the world write computer programs at this point. > I think it is perfectly reasonable ... Without any concrete justification? Why would entirely artificial restrictions ever be seen as reasonable?
- rmunn 1mo ago
  What "boneheaded things with tabs" are you referring to? The picture I'm piecing together from your comments suggests that you may use tab characters in a different way than most people seem to, so I'd quite like a further explanation of how you use tab characters and how you expect an editor to handle them.
  - fc417fc802 1mo ago
    > you may use tab characters in a different way than most people seem to As far as I'm aware there are three primary and entirely independent uses for tabs. Indentation, field separation, and typesetting. When it comes to typesetting and also to display of fields with a narrow maximum width the concept of a tabstop is useful. Boneheaded things with tabs was in reference to code editors (where tabs are more or less exclusively used for indentation) most often failing to provide the ability to disable tabstop. In practice this generally hasn't been a point of friction because until fairly recently the most popular languages (ie C & co) didn't support constructs that would lead to adding any indentation outside of the prefix of a line.
    - rmunn 1mo ago
      Agreed... but see my Lisp example in https://news.ycombinator.com/item?id=49593406 https://news.ycombinator.com/item?id=49593406 for a place where tabs end up unavoidably ambiguous. Because in that `cond` example the tabs look like they're used for indentation, but they're actually being used for alignment, something that falls more under typesetting than under indentation. A smart editor that has read the Lisp code (note that I'm using "read" in its Lisp meaning, i.e. "parsed into a syntax tree") can make those tab characters correctly align the forms the way you'd want them to be aligned for maximum understanding... but really, you would want space characters, not tab characters, in that context. So even though in most cases those three uses for tabs are independent, in some languages they get muddled. F# is another language where you may want to align text on line two with the middle of line one, e.g. if you write let person = { FirstName = "Bill" LastName = "Gates" } Now, that's considered bad style according to https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/formatting#avoid-formatting-that-is-sensitive-to-name-length https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/... — and they're entirely write to recommend against that style — but it's legal syntax. And it's probably why https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/formatting#use-spaces-not-tabs https://learn.microsoft.com/en-us/dotnet/fsharp/style-guide/... forbids tab characters entirely in F# code. (Those two links point to different anchors in the same document, BTW: the HN link shortening is going to make them look identical until you hover over them).

rmunn 1mo ago

I mean, obviously that one should only appear within Mongolian text and not within indentation. To state explicitly what should be implicitly obvious, there is no valid reason (that I'm aware of, I welcome any non-facetious correction) to use any character except U+0009 and U+0020 within indentation. Horizontal Record Separator? Zero-width joiner? Language-specific whitespace characters like your example? All make sense within human text (well, maybe not HRS), but in programming, they should be eschewed in favor of the characters that can be typed in every single keyboard layout in the world. Even languages that don't put spaces between words, such as Thai, still put spaces between sentences (or comma phrases) and therefore keep the space bar in their keyboard layout. And since mixing tabs and spaces (even between lines, where some lines are tab-indented and some are space-indented) creates problems for whitespace-sensitive language, there's a reason why every whitespace-sensitive language I'm aware of has tended to either outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations, which are usually not available to the validation code running in CI or on other people's machines).

fc417fc802 1mo ago
> To state explicitly what should be implicitly obvious, there is no valid reason ... There doesn't need to be an articulable reason. Or rather there's generally no expectation that a central authority will be able to reliably enumerate such. Everything should default to being permitted and only ever be restricted for good reason. But since you asked. U+2003 for example carries formatting information. Maybe an editor could be written (or even already exists) that would find that useful. Who is any third party to dictate that? U+00A0 similarly communicates information about the desired formatting and I can see no reason it would be unreasonable for someone to use it nor why its use should pose a technical challenge to a compiler. > creates problems for whitespace-sensitive langauge Does it? That seems like an invented problem to me. You have a running prefix composed of arbitrary whitespace characters. Any change in that prefix is a change in the level of indentation. You can add or remove arbitrary amounts from the end of the prefix. In the event you remove from it the result must exactly match the previous stack level. What's so complicated about this? > outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations ... Did you mix up your code points there? It's space that's ambiguous, not tab. Regardless I think that a compiler worrying about the specifics of text editors or other tooling would be backward information flow and a massive abstraction violation. Semantic meaning is entirely dictated by the compiler, not the other way around. There's no convincing reason (IMO) to impose restrictions that aren't technically necessary or to otherwise needlessly employ solutions that would reduce generalization.