6 ms·

It is start and end of line. [1] Usually ^ matches only at the beginning of the string, and $ matches only at the end of the string and immediately before the

It is start and end of line. [1] Usually ^ matches only at the beginning of the string, and $ matches only at the end of the string and immediately before the newline (if any) at the end of the string. When this flag is specified, ^ matches at the beginning of the string and at the beginning of each line within the string, immediately following each newline. Similarly, the $ metacharacter matches either at the end of the string and at the end of each line (immediately preceding each newline). In single-line [2] mode, the line starts at the start of the string and ends at the end of the line where the end of the line is either the end of the string if there is no terminating newline or just before the final newline if there is a terminating newline. In multi-line mode a new line starts at the start of the string and after each newline and ends before each newline or at the end of the string if the last line has no terminating newline. The confusion is that people think that they are in string-mode if they are not in multi-line mode but they are not, they are in single-line mode, ^ and $ still use the semantics of lines and a terminating newline, if present, is still not part of the content of the line. With \n\n\n in single-line mode the non-greedy ^(\n+?)$ will capture only two of the newlines, the third one will be eaten by the $. If you make it greedy ^(\n+)$ will capture all three newlines. So arguably the implementations that do not match cat\n with cat$ are the broken ones. [1] https://docs.python.org/3/howto/regex.html#more-metacharacters https://docs.python.org/3/howto/regex.html#more-metacharacte... [2] I am using single-line to mean not multi-line for convenience even though single-line already has a different meaning.

masklinn 3y ago
> It is start and end of line. You seem to have redefined “line” as “not a line”. > The confusion I’m sure redefining “line” as “nothing like what anyone reasonable would interpret as a line” will help a lot and right clear up the confusion.
- danbruc 3y ago
  The POSIX definition of a line is a sequence of non-newline characters - possibly zero - followed by a newline. Everything that does not end with a newline is not a [complete] line. So strictly speaking it would even be correct that cat$ does not match cat because there is no terminating newline, it should only match cat\n. But as lines missing a terminating newline is a thing, it seems reasonable to be less strict.
  - al_borland 3y ago
    I suppose this is why certain config files are strict about ending with a newline, without it, the last line wouldn’t technically be a line?
  - deleted 3y ago
    [deleted]
  - masklinn 3y ago
    > a line is a sequence of non-newline characters Works for me. How do you square that with your assertion that in your invention of "single-line mode" you implicitly define "line" as matching \n\n?
    - danbruc 3y ago
      If you are not in multi-line mode, then a single line is expected and consequently there is at most one newline at the end of the string. You can of course pick an input that violates this, run it against a multi-line string with several newlines in it. cat\n\n will not match cat$ because there is something between cat and the end of the line, it just happens to be a newline but without any special meaning because it is not the last character and you did not say that the input is multi-line.
  - sltkr 3y ago
    Python violates that definition however, by allowing internal newlines in strings. For example /^c[^a]t$/ matches "c\nt\n", but according to POSIX that's not a line. I suspect the real reason for Python's behavior starts with the early decision to include the terminating newline in the string returned by IOBase.readline(). Python's peculiar choice has some minor advantages: you can distinguish between files that do and don't end with a terminating newline (the latter are invalid according to POSIX, but common in practice, especially on Windows), and you can reconstruct the original file by simply concatenating the line strings, which is occasionally useful. The downside of this choice is that as a caller you have to deal with strings that may-or-may-not contain a terminating newline character, which is annoying (I often end up calling rstrip() or strip() on every line returned by readline(), just to get rid of the newlines; read().splitlines() is an option too if you don't mind reading the entire file into memory upfront). My guess is that Python's behavior is just a hack to make re.match() easier to use with readline(), rather than based on any principled belief about what lines are.
- deleted 3y ago
  [deleted]
- Bjartr 3y ago
  The line delimiter is a newline. If you have a file containing `A\nB\nC` in a file, the file is three lines long. I guess it could be argued that a file containing `A\nB\nC\n` has four lines, with the fourth having zero length. That a regex is applying to an in memory string vs a file doesn't feel to me like it should have different semantics. Digging into the history a little, it looks like regexes were popularized in text editors and other file oriented tooling. In those contexts I imagine it would be far more common to want to discard or ignore the trailing zero length line than to process it like every other line in a file.
  - akdev1l 3y ago
    Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.
    - wtetzner 3y ago
      So if you have "A" in a file with no newline, there are no lines in that file?
      - jepler 3y ago
        Yes, that is a file with zero lines that ends with an "incomplete line". Processing of such files by standard line-oriented utilities is undefined in the opengroup spec. So, for instance, the effect of "grep"ping such a file is not defined. Heck, even "cat"ting such a file gives non-ideal results, such as colliding with the regular shell prompt. For this reason, a lot of software projects I work on check and correct this condition whenever creating a commit. https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_403 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... ("text file")
        rovr138 3y ago
        > Yes, that is a file with zero lines that ends with an "incomplete line". It's a file with zero complete lines. But it has 1 line, that's incomplete, right? The file starts empty. Anything in it starts "a line". So it's 1 incomplete line. I hate weird states.

masklinn 3y ago

> It is start and end of line. You seem to have redefined “line” as “not a line”. > The confusion I’m sure redefining “line” as “nothing like what anyone reasonable would interpret as a line” will help a lot and right clear up the confusion.

danbruc 3y ago
The POSIX definition of a line is a sequence of non-newline characters - possibly zero - followed by a newline. Everything that does not end with a newline is not a [complete] line. So strictly speaking it would even be correct that cat$ does not match cat because there is no terminating newline, it should only match cat\n. But as lines missing a terminating newline is a thing, it seems reasonable to be less strict.
- al_borland 3y ago
  I suppose this is why certain config files are strict about ending with a newline, without it, the last line wouldn’t technically be a line?
- deleted 3y ago
  [deleted]
- masklinn 3y ago
  > a line is a sequence of non-newline characters Works for me. How do you square that with your assertion that in your invention of "single-line mode" you implicitly define "line" as matching \n\n?
  - danbruc 3y ago
    If you are not in multi-line mode, then a single line is expected and consequently there is at most one newline at the end of the string. You can of course pick an input that violates this, run it against a multi-line string with several newlines in it. cat\n\n will not match cat$ because there is something between cat and the end of the line, it just happens to be a newline but without any special meaning because it is not the last character and you did not say that the input is multi-line.
- sltkr 3y ago
  Python violates that definition however, by allowing internal newlines in strings. For example /^c[^a]t$/ matches "c\nt\n", but according to POSIX that's not a line. I suspect the real reason for Python's behavior starts with the early decision to include the terminating newline in the string returned by IOBase.readline(). Python's peculiar choice has some minor advantages: you can distinguish between files that do and don't end with a terminating newline (the latter are invalid according to POSIX, but common in practice, especially on Windows), and you can reconstruct the original file by simply concatenating the line strings, which is occasionally useful. The downside of this choice is that as a caller you have to deal with strings that may-or-may-not contain a terminating newline character, which is annoying (I often end up calling rstrip() or strip() on every line returned by readline(), just to get rid of the newlines; read().splitlines() is an option too if you don't mind reading the entire file into memory upfront). My guess is that Python's behavior is just a hack to make re.match() easier to use with readline(), rather than based on any principled belief about what lines are.
deleted 3y ago
[deleted]
Bjartr 3y ago
The line delimiter is a newline. If you have a file containing `A\nB\nC` in a file, the file is three lines long. I guess it could be argued that a file containing `A\nB\nC\n` has four lines, with the fourth having zero length. That a regex is applying to an in memory string vs a file doesn't feel to me like it should have different semantics. Digging into the history a little, it looks like regexes were popularized in text editors and other file oriented tooling. In those contexts I imagine it would be far more common to want to discard or ignore the trailing zero length line than to process it like every other line in a file.
- akdev1l 3y ago
  Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.
  - wtetzner 3y ago
    So if you have "A" in a file with no newline, there are no lines in that file?
    - jepler 3y ago
      Yes, that is a file with zero lines that ends with an "incomplete line". Processing of such files by standard line-oriented utilities is undefined in the opengroup spec. So, for instance, the effect of "grep"ping such a file is not defined. Heck, even "cat"ting such a file gives non-ideal results, such as colliding with the regular shell prompt. For this reason, a lot of software projects I work on check and correct this condition whenever creating a commit. https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_403 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... ("text file")
      - rovr138 3y ago
        > Yes, that is a file with zero lines that ends with an "incomplete line". It's a file with zero complete lines. But it has 1 line, that's incomplete, right? The file starts empty. Anything in it starts "a line". So it's 1 incomplete line. I hate weird states.