7 ms·

Why is it that rarely does anyone distinguish output format from source format in these discussions? Two spaces are used to semantically separate sentences in

Why is it that rarely does anyone distinguish output format from source format in these discussions? Two spaces are used to semantically separate sentences in source format. They can be collapsed or modified for typeset output (like HTML rendering), but nobody should be telling an author to use single space in source format if the author wants to add the additional semantic separation between sentences. Particularly at fault are sanitizers like tidy (not just stand-alone, but sometimes used as a component in document editors) which will collapse multiple spaces by default. They're not doing any other html rendering, and they're not minifying the html either, so why do they insist on removing a common form of semantic content information in the source document? Collapse >2 spaces to 2 in some cases, perhaps, but not 2 to 1, particularly not after punctuation. The two spaces vs one space argument for how text should be typeset is a completely different issue and unrelated to the question of how many spaces human typists should use between sentences. Using two spaces between sentences happens to make it easier for typesetting software to automatically apply whatever magical 1+epsilon or 1-epsilon inter-sentence spacing it deems best, assuming the typesetting software can deduce that the author is using the two-space sentence convention (otherwise it has to fall back on heuristics, which are not perfect). Any author is free to choose how many spaces to use between sentences, but collapsing 2 spaces when editing others' documents is wrong, and so is preventing an author from using two spaces. Doing so removes information content from the document, which can only be restored perfectly by humans (or other things with human-level natural language processing ability).

dylangs1030 13y ago
Honestly, I think it's neurotic that human beings will receive criticism for, and have to worry about the choice of how many times they tap a space bar when writing (not even coding). That we have to even include things like output format and source format to distinguish between is just on another level entirely (I'm not criticizing you, your point is valid). I mean, when I read titles like this I think to myself, "Who gives a shit?"...then I read a long thread and think, "Wow, this is actually a real debate." I never knew there was so much that could go into a decision about typesetting spaces. You learn something new everyday.
- harshreality 13y ago
  Find a reason to write a program that needs to separate prose into sentences, then ask yourself if the argument is neurotic. It's common to use tables of abbreviations to know when a period is probably not the end of a sentence, and then apply imperfect rules to handle edge cases like proper nouns. It's tedious and inaccurate.
  - TillE 13y ago
    Tough. You will never ever ever convince the world to conform to a rigid standard of two spaces between sentences, so your program is going to have to handle that kind of input.
    - harshreality 13y ago
      Granted, any program that has to deal with other writers' text has to deal with both possibilities, but two spaces improves accuracy, both for typesetting and for semantic recognition of sentences for other purposes. Even in a document with a mix of two spaces and one space between sentences, a program can settle a case that would otherwise be ambiguous if there are two spaces. The more simplistic (and inaccurate) your single-space sentence determination algorithm is, the more valuable it is to be able to assume (or assign a high weighting that) a two-space separation marks a new sentence. I know the world will never universally adopt using two spaces, but that doesn't mean they should get away with making spurious typesetting-related arguments and undoing the work of those who use 2 spaces. If a few more people decide that semantic richness is enough reason to start using two spaces, so much the better.
      - vizeroth 13y ago
        The problem is that the argument for using 2 spaces was never originally about parsing the text with a computer. Further, even the justification for using 2 spaces in the article is not doing so, either, it's an attempt at justifying using 2 spaces to emulate a typesetting practice (using a single space that was 2 or 3 times as wide as the space between words) which you still would not have expected someone to blindly apply to all text (you might have someone do it to all of the text just so a more experienced typesetter could come along behind them and make the adjustments that required a good eye for the work). Eventually publishers just found it easier to turn authors into apprentice typesetters, and probably found that most of the work they produce isn't worth paying an experienced typesetter to finish with that level of detail (and this attitude also shows in the editing of mass-market material, too). Of course, most mass-market publications use spaces of the same size after the end of a sentence as between words, now, and probably don't do much manual adjustment of the alignment of the text, either. Additionally, if you assume that "period space space" indicates the end of a sentence, eventually you're going to run into some case where that's wrong, too. It's bound to happen in a world where we routinely teach people to blindly hit "period space space." You may be better off incrementally improving your lexer/parser (or using and contributing to an open source project) and throwing out the excess spaces.
peterkelly 13y ago
Definitely in agreement on this one. I have a writing app which uses HTML as it's native format, and I've received a number of complaints from people about how it doesn't allow you to use multiple consecutive spaces. I suspect most of them are coming from a MS Word background (the app lets you edit Word documents but converts them temporarily to HTML for editing). Usually I just explain that this is how HTML works; whitespace is collapsed, and so my app is inherently dependent upon the capabilities of HTML and the layout engine being used (in my case, WebKit). However after the most recent complaint I finally spent some time looking deeper into the CSS spec and found out that you can set white-space: pre-wrap on a style, which will preserve consecutive spaces in the way that some people are accustomed to in Word. Now that I'm aware of this, I'll be turning this on by default for Word documents when I translate them to HTML, and for documents stored directly in HTML format I'll offer this as an option that can be set on a document-wide or style-specific basis. I've always seen the one-space vs. two-space argument as a bit pointless, since this is something that is within the domain of what software can handle, for the most part. LaTeX has facilities for adding extra space after periods, for example. The main difficulty is distinguishing between abbreviations and words like e.g. which don't start a new sentence, so you don't want extra space after them. And I'm not aware if any web browser or HTML layout engines which add extra space like LaTeX (which I think is 1.5 times a normal space), but I cold be wrong. As much as I've been in the camp of "just use one space; what's the big deal?" I think you've raised a very good point that the double-space can be treated as semantic information that instructs the typesetting system to place the surrounding characters further apart, even if it isn't exactly two spaces (and in the case of justified text, giving a greater weight to whitespace specified using two space characters). TL;DR: white-space: pre-wrap is a handy feature of CSS
anigbrowl 13y ago
Two spaces are used to semantically separate sentences in source format. That's the function of the period, which does the job perfectly well already, as you demonstrate yourself. Any author is free to choose how many spaces to use between sentences, but collapsing 2 spaces when editing others' documents is wrong, and so is preventing an author from using two spaces. Doing so removes information content from the document, which can only be restored perfectly by humans (or other things with human-level natural language processing ability). I could just as easily argue that it's wrong to include redundant semantic information. If I were to begin using double commas,, should everyone else alter their reading habits to accommodate my affectation? Fuck that.
- Patrick_Devine 13y ago
  The part which you omitted is that you can also use a period for denoting an abbreviation. It then can become ambiguous as to whether you're parsing that, or the end of a line. The only way to then determine whether it's an abbreviation is to use some other form of heuristic, which becomes a pain in the ass. Regardless, we're seriously into Parkinson's Law of Triviality territory. I'll support either way as long as the bike shed has an aluminum roof.
- deleted 13y ago
  [deleted]
- asdfs 13y ago
  >That's the function of the period, This breaks in the case of "Mr. Jones went to the store. He enjoys buying things." The spacing after 'Mr.' should be less than after 'store.'. If two spaces are prohibited, you have the following options, neither of them particularly satisfactory: -The user must input some form of special Unicode space character after 'Mr.' -The display software must include a list of all words that, when followed by a period, should not include extra space (e.g., "Mr, Mrs, Messrs, etc, e.g, i.e" and so forth).
  - anigbrowl 13y ago
    Says you. I was fine with it being the same when I learned to type (on a typewriter) and having text-presentation software do some micro kerning on certain abbreviations doesn't bother me in the least; automating such drudgery is one of the main reasons to use computers in the first place. A third option is to dispense with the punctuation marks on abbreviations and simply refer to 'Mr Jones.' This took a little getting used to, but since real grammatical consistency would involve abbreviating 'Mister' as 'M'r,' (a practice which you can see in some older books and which persisted with obscure words like f'c's'le) dropping the period seems simpler. It's good enough for the Economist, whose style guide I prefer.

dylangs1030 13y ago

Honestly, I think it's neurotic that human beings will receive criticism for, and have to worry about the choice of how many times they tap a space bar when writing (not even coding). That we have to even include things like output format and source format to distinguish between is just on another level entirely (I'm not criticizing you, your point is valid). I mean, when I read titles like this I think to myself, "Who gives a shit?"...then I read a long thread and think, "Wow, this is actually a real debate." I never knew there was so much that could go into a decision about typesetting spaces. You learn something new everyday.

harshreality 13y ago
Find a reason to write a program that needs to separate prose into sentences, then ask yourself if the argument is neurotic. It's common to use tables of abbreviations to know when a period is probably not the end of a sentence, and then apply imperfect rules to handle edge cases like proper nouns. It's tedious and inaccurate.
- TillE 13y ago
  Tough. You will never ever ever convince the world to conform to a rigid standard of two spaces between sentences, so your program is going to have to handle that kind of input.
  - harshreality 13y ago
    Granted, any program that has to deal with other writers' text has to deal with both possibilities, but two spaces improves accuracy, both for typesetting and for semantic recognition of sentences for other purposes. Even in a document with a mix of two spaces and one space between sentences, a program can settle a case that would otherwise be ambiguous if there are two spaces. The more simplistic (and inaccurate) your single-space sentence determination algorithm is, the more valuable it is to be able to assume (or assign a high weighting that) a two-space separation marks a new sentence. I know the world will never universally adopt using two spaces, but that doesn't mean they should get away with making spurious typesetting-related arguments and undoing the work of those who use 2 spaces. If a few more people decide that semantic richness is enough reason to start using two spaces, so much the better.
    - vizeroth 13y ago
      The problem is that the argument for using 2 spaces was never originally about parsing the text with a computer. Further, even the justification for using 2 spaces in the article is not doing so, either, it's an attempt at justifying using 2 spaces to emulate a typesetting practice (using a single space that was 2 or 3 times as wide as the space between words) which you still would not have expected someone to blindly apply to all text (you might have someone do it to all of the text just so a more experienced typesetter could come along behind them and make the adjustments that required a good eye for the work). Eventually publishers just found it easier to turn authors into apprentice typesetters, and probably found that most of the work they produce isn't worth paying an experienced typesetter to finish with that level of detail (and this attitude also shows in the editing of mass-market material, too). Of course, most mass-market publications use spaces of the same size after the end of a sentence as between words, now, and probably don't do much manual adjustment of the alignment of the text, either. Additionally, if you assume that "period space space" indicates the end of a sentence, eventually you're going to run into some case where that's wrong, too. It's bound to happen in a world where we routinely teach people to blindly hit "period space space." You may be better off incrementally improving your lexer/parser (or using and contributing to an open source project) and throwing out the excess spaces.

peterkelly 13y ago

Definitely in agreement on this one. I have a writing app which uses HTML as it's native format, and I've received a number of complaints from people about how it doesn't allow you to use multiple consecutive spaces. I suspect most of them are coming from a MS Word background (the app lets you edit Word documents but converts them temporarily to HTML for editing). Usually I just explain that this is how HTML works; whitespace is collapsed, and so my app is inherently dependent upon the capabilities of HTML and the layout engine being used (in my case, WebKit). However after the most recent complaint I finally spent some time looking deeper into the CSS spec and found out that you can set white-space: pre-wrap on a style, which will preserve consecutive spaces in the way that some people are accustomed to in Word. Now that I'm aware of this, I'll be turning this on by default for Word documents when I translate them to HTML, and for documents stored directly in HTML format I'll offer this as an option that can be set on a document-wide or style-specific basis. I've always seen the one-space vs. two-space argument as a bit pointless, since this is something that is within the domain of what software can handle, for the most part. LaTeX has facilities for adding extra space after periods, for example. The main difficulty is distinguishing between abbreviations and words like e.g. which don't start a new sentence, so you don't want extra space after them. And I'm not aware if any web browser or HTML layout engines which add extra space like LaTeX (which I think is 1.5 times a normal space), but I cold be wrong. As much as I've been in the camp of "just use one space; what's the big deal?" I think you've raised a very good point that the double-space can be treated as semantic information that instructs the typesetting system to place the surrounding characters further apart, even if it isn't exactly two spaces (and in the case of justified text, giving a greater weight to whitespace specified using two space characters). TL;DR: white-space: pre-wrap is a handy feature of CSS

anigbrowl 13y ago

Two spaces are used to semantically separate sentences in source format. That's the function of the period, which does the job perfectly well already, as you demonstrate yourself. Any author is free to choose how many spaces to use between sentences, but collapsing 2 spaces when editing others' documents is wrong, and so is preventing an author from using two spaces. Doing so removes information content from the document, which can only be restored perfectly by humans (or other things with human-level natural language processing ability). I could just as easily argue that it's wrong to include redundant semantic information. If I were to begin using double commas,, should everyone else alter their reading habits to accommodate my affectation? Fuck that.

Patrick_Devine 13y ago
The part which you omitted is that you can also use a period for denoting an abbreviation. It then can become ambiguous as to whether you're parsing that, or the end of a line. The only way to then determine whether it's an abbreviation is to use some other form of heuristic, which becomes a pain in the ass. Regardless, we're seriously into Parkinson's Law of Triviality territory. I'll support either way as long as the bike shed has an aluminum roof.
deleted 13y ago
[deleted]
asdfs 13y ago
>That's the function of the period, This breaks in the case of "Mr. Jones went to the store. He enjoys buying things." The spacing after 'Mr.' should be less than after 'store.'. If two spaces are prohibited, you have the following options, neither of them particularly satisfactory: -The user must input some form of special Unicode space character after 'Mr.' -The display software must include a list of all words that, when followed by a period, should not include extra space (e.g., "Mr, Mrs, Messrs, etc, e.g, i.e" and so forth).
- anigbrowl 13y ago
  Says you. I was fine with it being the same when I learned to type (on a typewriter) and having text-presentation software do some micro kerning on certain abbreviations doesn't bother me in the least; automating such drudgery is one of the main reasons to use computers in the first place. A third option is to dispense with the punctuation marks on abbreviations and simply refer to 'Mr Jones.' This took a little getting used to, but since real grammatical consistency would involve abbreviating 'Mister' as 'M'r,' (a practice which you can see in some older books and which persisted with obscure words like f'c's'le) dropping the period seems simpler. It's good enough for the Economist, whose style guide I prefer.