9 ms·

Learn Regex The Hard Way

187 points by QuarkSpark 15y ago

bstar77 15y ago
This is the "Learn X the Hard Way" book I've really been waiting for. Thanks, Zed, for the time and effort on this one. It's a topic that schools seem to completely ignore for some reason, but a skill I find I use on nearly a daily basis.
- redslazer 15y ago
  Its not done yet, its in draft stage and has been available for a while. Im waiting for it to come out fully on a pdf so i can throw my money at him.
- mkramlich 15y ago
  I'm pretty sure there have been plenty of books and documentation on regexps at least as far back as the 90's. I distinctly remember O'Reilly books on the topic, that far back, to be specific. For a certain generation at least, we haven't been waiting to be able to learn them. It's out there. Though having additional angles of approach is not bad.
  - zedshaw 15y ago
    Mastering Regular Expressions is the book you're thinking of.
  - goodweeds 15y ago
    I never liked the O'Reilly approach. At one point I had read 20 of their books and never felt like I learned anything. Some people can learn from references, but I really prefer Zed's "hard way" method which feels a lot like the Programmed Learning methodlogy (http://en.wikipedia.org/wiki/Programmed_learning http://en.wikipedia.org/wiki/Programmed_learning).
    - mkramlich 15y ago
      Understood. Technically there has always been a harder way available to all of us. You acquire a book or some other documentation. Then you sit down at keyboard and write the bare minimum code/syntax to test if you understood how some things work, or just play around it. The terminals on Mac, Windows, Linux have allowed us to do this for decades, as far back as my own childhood, to give one example. So yeah, I think the "hard way" is good, and iterative feedback and experiment is good, and that's been available for a long time even without Zed's things. I'm still learning new things even in the last few days, using this approach, except without some third-party playbook I have to follow. Learn Redis the hard way? Download it. Install it. Start it. Enter client. Type things. See what happens. Repeat. This is fairly obvious. Agreed O'Reilly is probably more famous for their "completeness" rather than effectiveness at teaching.
james2vegas 15y ago
This being zedshaw, no surprise at the lack of Perl in this book, but what is funny is at http://regex.learncodethehardway.org/book/learn-regex-the-hard-wayli3.html#x4-50000.2 http://regex.learncodethehardway.org/book/learn-regex-the-ha... "Imagine if you could write your regular expressions like this: " I don't have to imagine since Perl's regular expression engine has the x modifier to do just that.
- bermanoid 15y ago
  It's funny...when I read that bit, all I could think was, how much nicer would it be if regexes could just be written without the special characters, using sane word-based self-documenting tokens the way we do the rest of our programming, following scoping and quotation rules that actually mesh with the languages we're using them in? Then it occurred to me that plenty of people have probably written libraries to do exactly that, but nobody uses them because we all already have regexes built-in almost everywhere that we want to use them. Hell, I've never even looked for one, even though I choke back a little vomit every time I introduce a new regex into a codebase because of how much future debugging pain I know they can cause (all but the shortest ones force what's essentially a full context-shift in order to parse, and in reality what usually happens is people scan a regex as one chunk and say "Eh, a regex, it's probably right, hopefully my bug is somewhere else..." until they have some concrete reason to think otherwise). Sort of a shame, really, that such a problematically condensed syntax won the prize so early on, and now even those of us that hate it are too comfortable with it to look for something better.
  - zedshaw 15y ago
    Regex aren't difficult to learn, it's just nobody teaches them as a language with a base syntax and words to use. If you just sit down and memorize the names of a few symbols, then learn what each does, then it becomes fairly clear. It's my belief (totally unfounded) that learning a simple symbolic language like regular expressions teaches you how to handle other symbolic languages like mathematics, chemistry, and programming. That's one of the reasons I'm teaching it and trying to get other people to use it. More importantly though, they are damn handy. As long as you don't abuse them in places where a lexer+parser is better, you can get a lot done with very little regex in very short time.
alexjgough 15y ago
Here one should not forget the past, and look to Jeffrey Friedl's excellent "Mastering Regular Expressions".
- theneb 15y ago
  It's actually a good accompliment to "Mastering Regular Expressions" as in that book Python isn't covered.
leeoniya 15y ago
"Exercise 11: This Or That" sadly still suffers from using examples which necessitate mentioning caveats I pointed out months ago. https://news.ycombinator.com/item?id=3297996 https://news.ycombinator.com/item?id=3297996 Even considering the fact that alteration is introduced after the needed concepts of NL and EOL assertions in "Exercise 7: The Beginning And End" no me gusta :(
- zedshaw 15y ago
  I already told you that your reason it was busted is wrong. It's not because of NL or EOL, it's because of the order of precedence of the | operator. If you're going to correct me, be right. In fact, someone also posted a solution in the comments, which you also didn't do.
  - leeoniya 15y ago
    i did not say that you were wrong. you are correct. it is the order of precedence that causes the NL/EOL to be treated as part of the alteration. you are wrong, however, not to mention this alongside examples that specifically display this in an unrelated context, as this is something that is far from obvious. you can 1. easily use different examples or 2. add one line of text that clears this up on the other hand, the title of the book is to learn regex the hard way, so i'm quite possibly the one who is wrong here. i am not trying to troll, trust me, i respect the work and knowledge you have put into this. but as someone who has been confused by this in the past during my own learning, i see no reason why you think i'm an exception and continue to argue against any clarification or change.
    - zedshaw 15y ago
      No, you're wrong still. NL/EOL doesn't factor into it at all. You also are one of those people who thinks the following: 1. "I have been bit in the past by not knowing an obscure fact." 2. "To protect myself, I must learn every obscure fact to prevent this from happening again." 3. "This here has a missing obscure fact and is therefore going to hurt me and everyone around me." This attitude that you have to teach someone everything about something right away is the reason most educational tech books suck. You do not need to teach someone everything right away. You don't even need to teach them everything as long as what you've taught is the foundational elements and those are correct. This attitude is also hyperbolic. The book is not going to destroy the world because you have a problem with one small portion of one exercise that you can't even fix yourself. Finally, you keep saying these things, and you keep asserting you're correct, but I don't see a solution from you. It's in a git repository: http://gitorious.org/learn-regex-the-hard-way/ http://gitorious.org/learn-regex-the-hard-way/ So put your money where your mouth is and send me a patch. If it's soooooo easy to fix and explain then prove me wrong. Until you offer up your supposedly superior world saving solution I have to assume you're just wrong but can't admit it.
riffraff 15y ago
I'm curious to see what ends up in "21: Extensions To Avoid". By the way, from a quick glance, I believe there is no chapter devoted to discuss how the regexp engines work (which is ok) but will there be a section on "when things explode in your face due to exponential behaviour"? I remember when I first noticed a regex I wrote had a bewhaviour like that and it was kind of enlightening, because earlier I had always assumed all the talk about this things was rather academic :)
- zedshaw 15y ago
  Yeah, so I thought about making the 2nd half a walk through building a regex engine by first building a lexer, then a parser, then the engine, but I wasn't sure if that'd work to teach regular expression. I think it'd be a great way to really understand how they work (it's how I figured them out), but I'm not sure if other folks would get the same understanding out of it.
  - tsliwkan 15y ago
    I think this would be awesome. The mini database and the simple object system were really interesting exercises in Learn C the Hard Way and this would be another great exercise/project for one of your books.
  - cschep 15y ago
    Along these lines are there any plans for "learn compilers the hard way" ? Missed that big time in school.
gghh 15y ago
I am looking forward for item 24 "debugging Regex", and I hope it will cover perl's (use re 'debug') or python re.DEBUG flag. I still didn't find a satisfactory document on the web on those features. I'd also like to see something on the eternal diatribe "regexp VS regular grammars", i.e. how the formers substantially differ from the latter. But maybe that's not the right book, having the (sane) "get it done" approach.
jackfoxy 15y ago
Zed, Do you have a timeline in mind for filling out the rest of the exercises in the alpha release? Do you call it a beta at that point? I think you're doing great work, btw. Thanks.
jimmytucson 15y ago
Thanks for this. I learned Perl about 1.5 years ago and recently took up learning Python. The one thing I find less intuitive about Python than Perl is its syntax around regular expressions.

187 points by QuarkSpark 15y ago

This is the "Learn X the Hard Way" book I've really been waiting for. Thanks, Zed, for the time and effort on this one. It's a topic that schools seem to completely ignore for some reason, but a skill I find I use on nearly a daily basis.

redslazer 15y ago
Its not done yet, its in draft stage and has been available for a while. Im waiting for it to come out fully on a pdf so i can throw my money at him.
mkramlich 15y ago
I'm pretty sure there have been plenty of books and documentation on regexps at least as far back as the 90's. I distinctly remember O'Reilly books on the topic, that far back, to be specific. For a certain generation at least, we haven't been waiting to be able to learn them. It's out there. Though having additional angles of approach is not bad.
- zedshaw 15y ago
  Mastering Regular Expressions is the book you're thinking of.
- goodweeds 15y ago
  I never liked the O'Reilly approach. At one point I had read 20 of their books and never felt like I learned anything. Some people can learn from references, but I really prefer Zed's "hard way" method which feels a lot like the Programmed Learning methodlogy (http://en.wikipedia.org/wiki/Programmed_learning http://en.wikipedia.org/wiki/Programmed_learning).
  - mkramlich 15y ago
    Understood. Technically there has always been a harder way available to all of us. You acquire a book or some other documentation. Then you sit down at keyboard and write the bare minimum code/syntax to test if you understood how some things work, or just play around it. The terminals on Mac, Windows, Linux have allowed us to do this for decades, as far back as my own childhood, to give one example. So yeah, I think the "hard way" is good, and iterative feedback and experiment is good, and that's been available for a long time even without Zed's things. I'm still learning new things even in the last few days, using this approach, except without some third-party playbook I have to follow. Learn Redis the hard way? Download it. Install it. Start it. Enter client. Type things. See what happens. Repeat. This is fairly obvious. Agreed O'Reilly is probably more famous for their "completeness" rather than effectiveness at teaching.

james2vegas 15y ago

This being zedshaw, no surprise at the lack of Perl in this book, but what is funny is at http://regex.learncodethehardway.org/book/learn-regex-the-hard-wayli3.html#x4-50000.2 http://regex.learncodethehardway.org/book/learn-regex-the-ha... "Imagine if you could write your regular expressions like this: " I don't have to imagine since Perl's regular expression engine has the x modifier to do just that.

bermanoid 15y ago
It's funny...when I read that bit, all I could think was, how much nicer would it be if regexes could just be written without the special characters, using sane word-based self-documenting tokens the way we do the rest of our programming, following scoping and quotation rules that actually mesh with the languages we're using them in? Then it occurred to me that plenty of people have probably written libraries to do exactly that, but nobody uses them because we all already have regexes built-in almost everywhere that we want to use them. Hell, I've never even looked for one, even though I choke back a little vomit every time I introduce a new regex into a codebase because of how much future debugging pain I know they can cause (all but the shortest ones force what's essentially a full context-shift in order to parse, and in reality what usually happens is people scan a regex as one chunk and say "Eh, a regex, it's probably right, hopefully my bug is somewhere else..." until they have some concrete reason to think otherwise). Sort of a shame, really, that such a problematically condensed syntax won the prize so early on, and now even those of us that hate it are too comfortable with it to look for something better.
- zedshaw 15y ago
  Regex aren't difficult to learn, it's just nobody teaches them as a language with a base syntax and words to use. If you just sit down and memorize the names of a few symbols, then learn what each does, then it becomes fairly clear. It's my belief (totally unfounded) that learning a simple symbolic language like regular expressions teaches you how to handle other symbolic languages like mathematics, chemistry, and programming. That's one of the reasons I'm teaching it and trying to get other people to use it. More importantly though, they are damn handy. As long as you don't abuse them in places where a lexer+parser is better, you can get a lot done with very little regex in very short time.

alexjgough 15y ago

Here one should not forget the past, and look to Jeffrey Friedl's excellent "Mastering Regular Expressions".

theneb 15y ago
It's actually a good accompliment to "Mastering Regular Expressions" as in that book Python isn't covered.

leeoniya 15y ago

"Exercise 11: This Or That" sadly still suffers from using examples which necessitate mentioning caveats I pointed out months ago. https://news.ycombinator.com/item?id=3297996 https://news.ycombinator.com/item?id=3297996 Even considering the fact that alteration is introduced after the needed concepts of NL and EOL assertions in "Exercise 7: The Beginning And End" no me gusta :(

zedshaw 15y ago
I already told you that your reason it was busted is wrong. It's not because of NL or EOL, it's because of the order of precedence of the | operator. If you're going to correct me, be right. In fact, someone also posted a solution in the comments, which you also didn't do.
- leeoniya 15y ago
  i did not say that you were wrong. you are correct. it is the order of precedence that causes the NL/EOL to be treated as part of the alteration. you are wrong, however, not to mention this alongside examples that specifically display this in an unrelated context, as this is something that is far from obvious. you can 1. easily use different examples or 2. add one line of text that clears this up on the other hand, the title of the book is to learn regex the hard way, so i'm quite possibly the one who is wrong here. i am not trying to troll, trust me, i respect the work and knowledge you have put into this. but as someone who has been confused by this in the past during my own learning, i see no reason why you think i'm an exception and continue to argue against any clarification or change.
  - zedshaw 15y ago
    No, you're wrong still. NL/EOL doesn't factor into it at all. You also are one of those people who thinks the following: 1. "I have been bit in the past by not knowing an obscure fact." 2. "To protect myself, I must learn every obscure fact to prevent this from happening again." 3. "This here has a missing obscure fact and is therefore going to hurt me and everyone around me." This attitude that you have to teach someone everything about something right away is the reason most educational tech books suck. You do not need to teach someone everything right away. You don't even need to teach them everything as long as what you've taught is the foundational elements and those are correct. This attitude is also hyperbolic. The book is not going to destroy the world because you have a problem with one small portion of one exercise that you can't even fix yourself. Finally, you keep saying these things, and you keep asserting you're correct, but I don't see a solution from you. It's in a git repository: http://gitorious.org/learn-regex-the-hard-way/ http://gitorious.org/learn-regex-the-hard-way/ So put your money where your mouth is and send me a patch. If it's soooooo easy to fix and explain then prove me wrong. Until you offer up your supposedly superior world saving solution I have to assume you're just wrong but can't admit it.

riffraff 15y ago

I'm curious to see what ends up in "21: Extensions To Avoid". By the way, from a quick glance, I believe there is no chapter devoted to discuss how the regexp engines work (which is ok) but will there be a section on "when things explode in your face due to exponential behaviour"? I remember when I first noticed a regex I wrote had a bewhaviour like that and it was kind of enlightening, because earlier I had always assumed all the talk about this things was rather academic :)

zedshaw 15y ago
Yeah, so I thought about making the 2nd half a walk through building a regex engine by first building a lexer, then a parser, then the engine, but I wasn't sure if that'd work to teach regular expression. I think it'd be a great way to really understand how they work (it's how I figured them out), but I'm not sure if other folks would get the same understanding out of it.
- tsliwkan 15y ago
  I think this would be awesome. The mini database and the simple object system were really interesting exercises in Learn C the Hard Way and this would be another great exercise/project for one of your books.
- cschep 15y ago
  Along these lines are there any plans for "learn compilers the hard way" ? Missed that big time in school.

gghh 15y ago

I am looking forward for item 24 "debugging Regex", and I hope it will cover perl's (use re 'debug') or python re.DEBUG flag. I still didn't find a satisfactory document on the web on those features. I'd also like to see something on the eternal diatribe "regexp VS regular grammars", i.e. how the formers substantially differ from the latter. But maybe that's not the right book, having the (sane) "get it done" approach.

jackfoxy 15y ago

Zed, Do you have a timeline in mind for filling out the rest of the exercises in the alpha release? Do you call it a beta at that point? I think you're doing great work, btw. Thanks.

jimmytucson 15y ago

Thanks for this. I learned Perl about 1.5 years ago and recently took up learning Python. The one thing I find less intuitive about Python than Perl is its syntax around regular expressions.