24 ms·

Nontrivial regexes should be treated as their own language. I.e., either the team agrees to use them freely, in which case, team members are obligated to actual

by xamuel 8y ago

Nontrivial regexes should be treated as their own language. I.e., either the team agrees to use them freely, in which case, team members are obligated to actually learn them. Or, if not, then team members agree not to use them. If you were working on a Python-only codebase and someone opened a merge request to add a C++ file, you'd smack them down. Complex regexes should be no different.

3pt14159 8y ago
Hmm. Well. I understand where you are coming from, but I don't think it is that 100% that simple. Sometimes a project is 99.99% regex free, but a regex does the exact right thing. For example, say you're working on a project where homomorphic confusion between characters would be a security concern so the charset is restricted but in certain contexts, say, Japanese is expected. A simple: japanese = '([\p{Hiragana}\p{Katakana}]+)' Is all you really need. Concat it with the rest of the strings for the regex and everyone can understand it.
- marcosdumay 8y ago
  This is one complex regex for you to get the context (it doesn't work anymore, as email validation changed since then): http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html Adding that kind of thing to a project requires a calm cost-benefit analysis and very likely can not be done without the input of everybody working on the project.
  - mrfredward 8y ago
    Imagine you need to check if an email address meets RFC822. You can: a) read the 47 page standard and implement every single requirement, one if statement at a time, and end up writing 10s of thousands of lines of code that are nearly unverifiable. b) copy that one regex from the standard, put it in a nice function with a comment on where to find the standard, and move on with your life. Option b will not only save you a week, but also make it much less likely that you screwed up.
    - marcosdumay 8y ago
      What are you building? Almost every time you don't have to do a, you also do not need full email validation. Besides, the regex is wrong, like I said on my comment, so now what do you do? But that's a diversion. You can import a library that will do the same just as well as you can import that regex. Both will have comparable levels of gains and problems, but of very different nature.
      - evilDagmar 8y ago
        Of the two, the regexp from the spec should be preferable, because it is exactly what is in the spec. Some other library would be a once-removed version of it and could actually bring flaws of its own.
    - horsawlarway 8y ago
      Actually, both A and B are incorrect choices. Option A is a complete waste of time. Option B uses a regex that is no longer standards compliant and will tell users that their entirely valid email address is invalid (this is usually a no-no...). You're skipping entirely over option C: c) Send the user an email asking them to validate their email. You'll notice option C is actually the preferred method of almost every company that needs your email to be valid. Option C also does not use a regex at all.
  - 3pt14159 8y ago
    Yeah, I agree with you regarding email validators. My overarching point was that I don't think it is an all-or-nothing decision to include regexes. I shy away from them for most projects, but sometimes they're necessary.
deleted 8y ago
[deleted]
mrfredward 8y ago
I disagree. Regexes can can be done in a neat and modular way. I can write a 3 line function IsValidEmailAddress() with a nasty regex in it, and everyone on the team who doesn't know regexes will understand immediately what I'm up to when I call IsValidEmailAddress() and program around it. The alternative is to write a function hundreds or thousands of lines long that verifies one part of an email string at a time. Of course, that function will be too long and will have lots of duplicated code, so it will be refactored into smaller and more generic functions over and over until I've got a half baked text matching library on my hands. And then I'll have to document the library, and write unit tests for it, and if there is a bug or assumptions change about what to accept as an email, someone will have to read and understand my pages of code. And then someone will want to find email addresses in a file, and there won't be an efficient way to do it. If it's not trivial to do a pattern match with regexes, it's probably not trivial to do it without. The best you can do is keep the ugly implementation hidden in some well named functions that make your intent obvious.
- thrmsforbfast 8y ago
  Regexes get a bad rap mostly for two reasons: 1) they are abused by people who have never written a parser but need to write a parser, often to parse languages that are highly nonregular, and 2) they get thrown at problems where the underlying grammar is actually unknown and iterated on until they're uneditable black boxes. The first problem already has a really good solution: hire people who know the standard undergraduate CS material and you'll never end up with a regex-from-hell when a simple and easy recursive descent parser would've done the trick with ease. The second problem is, of course, more difficult. I've only seen regexes used successfully for a problem like that once. And that's only because the structure was very easy to think about as an NFA. By itself, the regex was a freaking nightmare to look at. But the documentation included a picture of the automata and a great description of what was going on. From there, the monstrous regex was easy to reason about and modify.
  - wahern 8y ago
    hire people who know the standard undergraduate CS material and you'll never end up with a regex-from-hell when a simple and easy recursive descent parser would've done the trick with ease The number of programmers whom I've personally worked with that actually applied proper parsing principles in their work are exceedingly few--count on one hand, few; too few to make generalizations. But suffice it to say not all had CS degrees, nor even a degree of any kind. By contrast, most of the people I've worked with professionally have had a CS degree, and regex abuse is rife. I'm sure the reasons are varied and complex (never really grasped it, unable to recognize appropriate situations, can't apply it in larger code bases, or are simply unable to resist easy fixes when under pressure), but the end result is the same.
rhizome 8y ago
Complex regexes should use the 'x' flag so they can be multiline with comments.

by xamuel 8y ago

3pt14159 8y ago

Hmm. Well. I understand where you are coming from, but I don't think it is that 100% that simple. Sometimes a project is 99.99% regex free, but a regex does the exact right thing. For example, say you're working on a project where homomorphic confusion between characters would be a security concern so the charset is restricted but in certain contexts, say, Japanese is expected. A simple: japanese = '([\p{Hiragana}\p{Katakana}]+)' Is all you really need. Concat it with the rest of the strings for the regex and everyone can understand it.

marcosdumay 8y ago
This is one complex regex for you to get the context (it doesn't work anymore, as email validation changed since then): http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html Adding that kind of thing to a project requires a calm cost-benefit analysis and very likely can not be done without the input of everybody working on the project.
- mrfredward 8y ago
  Imagine you need to check if an email address meets RFC822. You can: a) read the 47 page standard and implement every single requirement, one if statement at a time, and end up writing 10s of thousands of lines of code that are nearly unverifiable. b) copy that one regex from the standard, put it in a nice function with a comment on where to find the standard, and move on with your life. Option b will not only save you a week, but also make it much less likely that you screwed up.
  - marcosdumay 8y ago
    What are you building? Almost every time you don't have to do a, you also do not need full email validation. Besides, the regex is wrong, like I said on my comment, so now what do you do? But that's a diversion. You can import a library that will do the same just as well as you can import that regex. Both will have comparable levels of gains and problems, but of very different nature.
    - evilDagmar 8y ago
      Of the two, the regexp from the spec should be preferable, because it is exactly what is in the spec. Some other library would be a once-removed version of it and could actually bring flaws of its own.
  - horsawlarway 8y ago
    Actually, both A and B are incorrect choices. Option A is a complete waste of time. Option B uses a regex that is no longer standards compliant and will tell users that their entirely valid email address is invalid (this is usually a no-no...). You're skipping entirely over option C: c) Send the user an email asking them to validate their email. You'll notice option C is actually the preferred method of almost every company that needs your email to be valid. Option C also does not use a regex at all.
- 3pt14159 8y ago
  Yeah, I agree with you regarding email validators. My overarching point was that I don't think it is an all-or-nothing decision to include regexes. I shy away from them for most projects, but sometimes they're necessary.

deleted 8y ago

[deleted]

mrfredward 8y ago

I disagree. Regexes can can be done in a neat and modular way. I can write a 3 line function IsValidEmailAddress() with a nasty regex in it, and everyone on the team who doesn't know regexes will understand immediately what I'm up to when I call IsValidEmailAddress() and program around it. The alternative is to write a function hundreds or thousands of lines long that verifies one part of an email string at a time. Of course, that function will be too long and will have lots of duplicated code, so it will be refactored into smaller and more generic functions over and over until I've got a half baked text matching library on my hands. And then I'll have to document the library, and write unit tests for it, and if there is a bug or assumptions change about what to accept as an email, someone will have to read and understand my pages of code. And then someone will want to find email addresses in a file, and there won't be an efficient way to do it. If it's not trivial to do a pattern match with regexes, it's probably not trivial to do it without. The best you can do is keep the ugly implementation hidden in some well named functions that make your intent obvious.

thrmsforbfast 8y ago
Regexes get a bad rap mostly for two reasons: 1) they are abused by people who have never written a parser but need to write a parser, often to parse languages that are highly nonregular, and 2) they get thrown at problems where the underlying grammar is actually unknown and iterated on until they're uneditable black boxes. The first problem already has a really good solution: hire people who know the standard undergraduate CS material and you'll never end up with a regex-from-hell when a simple and easy recursive descent parser would've done the trick with ease. The second problem is, of course, more difficult. I've only seen regexes used successfully for a problem like that once. And that's only because the structure was very easy to think about as an NFA. By itself, the regex was a freaking nightmare to look at. But the documentation included a picture of the automata and a great description of what was going on. From there, the monstrous regex was easy to reason about and modify.
- wahern 8y ago
  hire people who know the standard undergraduate CS material and you'll never end up with a regex-from-hell when a simple and easy recursive descent parser would've done the trick with ease The number of programmers whom I've personally worked with that actually applied proper parsing principles in their work are exceedingly few--count on one hand, few; too few to make generalizations. But suffice it to say not all had CS degrees, nor even a degree of any kind. By contrast, most of the people I've worked with professionally have had a CS degree, and regex abuse is rife. I'm sure the reasons are varied and complex (never really grasped it, unable to recognize appropriate situations, can't apply it in larger code bases, or are simply unable to resist easy fixes when under pressure), but the end result is the same.

rhizome 8y ago

Complex regexes should use the 'x' flag so they can be multiline with comments.