8 ms·

It's kind of driving me nuts that the article says ^ is "start of string" when it's actually "start of line", just like $ is "end of line". \A is apparently "s

It's kind of driving me nuts that the article says ^ is "start of string" when it's actually "start of line", just like $ is "end of line". \A is apparently "start of string" like \Z is "end of string".

masklinn 3y ago
It’s not start of line though, unless the engine is in multiline mode. Here is the documentation for Python’s re for instance: > Matches the start of the string, and in MULTILINE mode also matches immediately after each newline. Or JavaScript: > An input boundary is the start or end of the string; or, if the m flag is set, the start or end of a line. \A and \Z are start/end of input regardless of mode… when they’re available, that’s not the case of all engines.
- eastbound 3y ago
  Probably a vulnerability issue. Programmers would leave multiline mode on by mistake, then validate that some string only contain ^[a-Z]*$… only for the string to have an \n and an SQL injection on the second line.
  - masklinn 3y ago
    > Probably a vulnerability issue. No? It’s a semantics decision.
- danbruc 3y ago
  It is start and end of line. [1] Usually ^ matches only at the beginning of the string, and $ matches only at the end of the string and immediately before the newline (if any) at the end of the string. When this flag is specified, ^ matches at the beginning of the string and at the beginning of each line within the string, immediately following each newline. Similarly, the $ metacharacter matches either at the end of the string and at the end of each line (immediately preceding each newline). In single-line [2] mode, the line starts at the start of the string and ends at the end of the line where the end of the line is either the end of the string if there is no terminating newline or just before the final newline if there is a terminating newline. In multi-line mode a new line starts at the start of the string and after each newline and ends before each newline or at the end of the string if the last line has no terminating newline. The confusion is that people think that they are in string-mode if they are not in multi-line mode but they are not, they are in single-line mode, ^ and $ still use the semantics of lines and a terminating newline, if present, is still not part of the content of the line. With \n\n\n in single-line mode the non-greedy ^(\n+?)$ will capture only two of the newlines, the third one will be eaten by the $. If you make it greedy ^(\n+)$ will capture all three newlines. So arguably the implementations that do not match cat\n with cat$ are the broken ones. [1] https://docs.python.org/3/howto/regex.html#more-metacharacters https://docs.python.org/3/howto/regex.html#more-metacharacte... [2] I am using single-line to mean not multi-line for convenience even though single-line already has a different meaning.
  - masklinn 3y ago
    > It is start and end of line. You seem to have redefined “line” as “not a line”. > The confusion I’m sure redefining “line” as “nothing like what anyone reasonable would interpret as a line” will help a lot and right clear up the confusion.
    - danbruc 3y ago
      The POSIX definition of a line is a sequence of non-newline characters - possibly zero - followed by a newline. Everything that does not end with a newline is not a [complete] line. So strictly speaking it would even be correct that cat$ does not match cat because there is no terminating newline, it should only match cat\n. But as lines missing a terminating newline is a thing, it seems reasonable to be less strict.
      - al_borland 3y ago
        I suppose this is why certain config files are strict about ending with a newline, without it, the last line wouldn’t technically be a line?
      - deleted 3y ago
        [deleted]
      - masklinn 3y ago
        > a line is a sequence of non-newline characters Works for me. How do you square that with your assertion that in your invention of "single-line mode" you implicitly define "line" as matching \n\n?
        danbruc 3y ago
        If you are not in multi-line mode, then a single line is expected and consequently there is at most one newline at the end of the string. You can of course pick an input that violates this, run it against a multi-line string with several newlines in it. cat\n\n will not match cat$ because there is something between cat and the end of the line, it just happens to be a newline but without any special meaning because it is not the last character and you did not say that the input is multi-line.
- f1shy 3y ago
  Matches the EMPTY STRING at the beginning of the line is the correct definition.
tangus 3y ago
That gives the author space for another article ;)
amelius 3y ago
What is driving me nuts is that we have Unicode now, so there is no need to use common characters like $ or ^ to denote special regex state transitions.
- knome 3y ago
  the idea of changing a decades old convention to instead use, as I assume you are implying, some character that requires special entry, is beyond silly.
  - FranOntanaya 3y ago
    I don't think anyone that writes regex would feel specially challenged by using the Alt+ | Ctrl+Shift+u key combos for unicode entry. Having to escape less things in a pattern would be nice.
    - amelius 3y ago
      Also, code is read more often than it is written.
      - cortesoft 3y ago
        People say this all the time, but is it really always true? I have a ton of code that I wrote, that just works, and I never really look at it again, at least not with the level of inspection that requires parsing the regex in my head.
        wccrawford 3y ago
        Even for code I wrote once and then never have to fix, I end up reading it multiple times while I create it and the lines around it. I think it really is always true.
    - cortesoft 3y ago
      I write regexes all the time, and I don't know if I would be CHALLENGED by that, but it would be annoying. Escaping things is trivial, and since you do it all the time it is not anything extra to learn. Having to remember bespoke keystrokes for each character is a lot more to learn.
davidw 3y ago
What with unicode, it'd be fun to have Α and Ω available to make our regexps that much more readable...

masklinn 3y ago

It’s not start of line though, unless the engine is in multiline mode. Here is the documentation for Python’s re for instance: > Matches the start of the string, and in MULTILINE mode also matches immediately after each newline. Or JavaScript: > An input boundary is the start or end of the string; or, if the m flag is set, the start or end of a line. \A and \Z are start/end of input regardless of mode… when they’re available, that’s not the case of all engines.

eastbound 3y ago
Probably a vulnerability issue. Programmers would leave multiline mode on by mistake, then validate that some string only contain ^[a-Z]*$… only for the string to have an \n and an SQL injection on the second line.
- masklinn 3y ago
  > Probably a vulnerability issue. No? It’s a semantics decision.
danbruc 3y ago
It is start and end of line. [1] Usually ^ matches only at the beginning of the string, and $ matches only at the end of the string and immediately before the newline (if any) at the end of the string. When this flag is specified, ^ matches at the beginning of the string and at the beginning of each line within the string, immediately following each newline. Similarly, the $ metacharacter matches either at the end of the string and at the end of each line (immediately preceding each newline). In single-line [2] mode, the line starts at the start of the string and ends at the end of the line where the end of the line is either the end of the string if there is no terminating newline or just before the final newline if there is a terminating newline. In multi-line mode a new line starts at the start of the string and after each newline and ends before each newline or at the end of the string if the last line has no terminating newline. The confusion is that people think that they are in string-mode if they are not in multi-line mode but they are not, they are in single-line mode, ^ and $ still use the semantics of lines and a terminating newline, if present, is still not part of the content of the line. With \n\n\n in single-line mode the non-greedy ^(\n+?)$ will capture only two of the newlines, the third one will be eaten by the $. If you make it greedy ^(\n+)$ will capture all three newlines. So arguably the implementations that do not match cat\n with cat$ are the broken ones. [1] https://docs.python.org/3/howto/regex.html#more-metacharacters https://docs.python.org/3/howto/regex.html#more-metacharacte... [2] I am using single-line to mean not multi-line for convenience even though single-line already has a different meaning.
- masklinn 3y ago
  > It is start and end of line. You seem to have redefined “line” as “not a line”. > The confusion I’m sure redefining “line” as “nothing like what anyone reasonable would interpret as a line” will help a lot and right clear up the confusion.
  - danbruc 3y ago
    The POSIX definition of a line is a sequence of non-newline characters - possibly zero - followed by a newline. Everything that does not end with a newline is not a [complete] line. So strictly speaking it would even be correct that cat$ does not match cat because there is no terminating newline, it should only match cat\n. But as lines missing a terminating newline is a thing, it seems reasonable to be less strict.
    - al_borland 3y ago
      I suppose this is why certain config files are strict about ending with a newline, without it, the last line wouldn’t technically be a line?
    - deleted 3y ago
      [deleted]
    - masklinn 3y ago
      > a line is a sequence of non-newline characters Works for me. How do you square that with your assertion that in your invention of "single-line mode" you implicitly define "line" as matching \n\n?
      - danbruc 3y ago
        If you are not in multi-line mode, then a single line is expected and consequently there is at most one newline at the end of the string. You can of course pick an input that violates this, run it against a multi-line string with several newlines in it. cat\n\n will not match cat$ because there is something between cat and the end of the line, it just happens to be a newline but without any special meaning because it is not the last character and you did not say that the input is multi-line.
f1shy 3y ago
Matches the EMPTY STRING at the beginning of the line is the correct definition.

tangus 3y ago

That gives the author space for another article ;)

amelius 3y ago

What is driving me nuts is that we have Unicode now, so there is no need to use common characters like $ or ^ to denote special regex state transitions.

knome 3y ago
the idea of changing a decades old convention to instead use, as I assume you are implying, some character that requires special entry, is beyond silly.
- FranOntanaya 3y ago
  I don't think anyone that writes regex would feel specially challenged by using the Alt+ | Ctrl+Shift+u key combos for unicode entry. Having to escape less things in a pattern would be nice.
  - amelius 3y ago
    Also, code is read more often than it is written.
    - cortesoft 3y ago
      People say this all the time, but is it really always true? I have a ton of code that I wrote, that just works, and I never really look at it again, at least not with the level of inspection that requires parsing the regex in my head.
      - wccrawford 3y ago
        Even for code I wrote once and then never have to fix, I end up reading it multiple times while I create it and the lines around it. I think it really is always true.
  - cortesoft 3y ago
    I write regexes all the time, and I don't know if I would be CHALLENGED by that, but it would be annoying. Escaping things is trivial, and since you do it all the time it is not anything extra to learn. Having to remember bespoke keystrokes for each character is a lot more to learn.

davidw 3y ago

What with unicode, it'd be fun to have Α and Ω available to make our regexps that much more readable...