4 ms·

The thing is, people don't find regexes difficult for any single reason. It depends entirely on context, the goals you're trying to achieve, and your skill leve

by savolai 10y ago

The thing is, people don't find regexes difficult for any single reason. It depends entirely on context, the goals you're trying to achieve, and your skill level. The whole point about this article is that the real issue is regex experts making assumptions about novices' experience based on their own experience. It depends a lot on what level of learning you are on. My hypothesis is that beginners are learning the individual commands and stumble on the escaping and the context. You don't need the mathematics for the mundane everyday simple matching that goes just a tiny bit beyond replace "a" with "b". The popularity of the original article seems to support the hypotheses that to the bulk of novice users (vast majority of whom probably are not HN users) the visual syntax approach would be helpful. -OP/author

brudgers 10y ago
A few years ago, when VerbalExpressions hit HackerNews I became interested in regexes [enough so that I did a Racket Port and put in some time on a building a more complete model]. VerbalExpresions is still popular enough to hit Hacker News from time to time. Anyway, I bought the O'Reilly Mastering Regular Expressions and looked at some StackOverflow questions and bought Ullman's Introduction to Auomata and took his Coursera course and in general thought about the nature of regex's [and regular expressions] off and on. What really highlighted the nature of regular expressions was thinking about regex generators. They popup from time to time and their purpose is to take a set of inputs and produce a regex that matches them. The problem is that given strings [x,y,z] the two best regex's are * and x|y|z. Any system that produces some other regex is by definition more difficult to understand than regex's themselves. Now circling back to novice users, simple code that iterates over strings with a loop or comprehension is probably going to be less prone to bugs than using regex's because regex's are so powerful. It's also going to be simpler to understand because the tools are familiar and purpose built and the abstractions of the code can be matched to the abstractions of the business logic. What makes regex's hard is that the abstractions are mathematical and intuitions about regex's rely on mathematical intuitions about automata [and in the case of Perl type regex's not simply finite automata due to backtracking and capture]. Please don't misunderstand me, I think the article is a neat piece of work. Even though I think that simplifying regex's isn't really possible because of mathematics, the article may introduce someone to the idea of using them. Because regex's are immensely powerful, that's a good thing. Good luck.
- savolai 10y ago
  Thanks for your kind words. You still don't see the point though. I am not simplifying regexes one single bit. The regex syntax is quite simply broken. It is fragmented as hell, and its escaping and context rules stem from a time when all we had was ASCII. This might have been acceptable in the eighties, yes? These have nothing to do with whether or not underlying regex theory is mathematically deep or not. I am not touching the theory. It's simply the syntax that is broken. You can go exactly as deep with Regex UCR as you can with traditional regexes. It's the same regex. You just don't have to torture yourself while doing it. We need to move forward from the single-eyed view that everything must be done in text, even if it means wasting countless hours in man hours and loads of cognitive processing. We need that energy for fruitful work. You are currently just advocating bureaucracy. Your looping example is yet another example of a theoretist making enormous assumptions the context of usage. How do you write a loop in the search/replace field of a text editor or an IDE? How does proneness to bugs matter one single bit if you're just using the tool to avoid a bit of manual work? Which [warning, hypothesis:] seems to be mostly what regexes are actually used anyway.
  - brudgers 10y ago
    After writing my comment, I went for a walk and ate some Chipolte and thought a bit more. And thought about JWZ's: Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. And now I'd put it like this. It doesn't matter what changes are made to the syntax. It doesn't matter what tools we build. The problem with regex's is that they won't balance parenthesis and not balancing parenthesis is counterintuitive to the ideas of matching and patterns. Even worse, there is no way to understand why problems like balancing parenthesis are impossible without a mental model based on regex's mathematical underpinnings. I should probably admit my bias, if I'm in an IDE or a text editor (or even a browser window), the last thing I want to do is reach for a mouse.
  - Tiksi 10y ago
    >and its escaping and context rules stem from a time when all we had was ASCII. My keyboard still only has ~ASCII on it, so having to input anything else is a huge barrier to entry. Regex syntax sucks, I'll be the first to agree with that, but it's more of an annoyance. If I write some regex to match ips(obviously not properly, but quick rough match): ([0-9]\.){1,3}[0-9]{1,3} My thought process isn't "open paren, open bracket, zero" etc, it's more of "Ok, I have a 4 groups of 1 to 3 numbers separated by dots, which can be written as 3 groups followed by a dot followed by a fourth group" and my hands translate it the rest of the way into text. Knowing exactly which character to use may have been a hindrance the first few times I ran into regex, but the real difficulty is in knowing what to match and how to match it. Providing that info to your computer once you know it is the easy part, even if the interface is horrible. It's like driving a car or a bike. You don't think "Ok, time to move my hand to move the steering wheel to the right and that will turn the wheels which will then the car to the right", you just need to turn right and your subconscious takes care of the rest. Improving the syntax would be like trying to improve how well/safely you drive by changing the steering wheel. Sure, it might help a little bit, but that's not really the difficult part of driving well, just an interface to it.

by savolai 10y ago

brudgers 10y ago

A few years ago, when VerbalExpressions hit HackerNews I became interested in regexes [enough so that I did a Racket Port and put in some time on a building a more complete model]. VerbalExpresions is still popular enough to hit Hacker News from time to time. Anyway, I bought the O'Reilly Mastering Regular Expressions and looked at some StackOverflow questions and bought Ullman's Introduction to Auomata and took his Coursera course and in general thought about the nature of regex's [and regular expressions] off and on. What really highlighted the nature of regular expressions was thinking about regex generators. They popup from time to time and their purpose is to take a set of inputs and produce a regex that matches them. The problem is that given strings [x,y,z] the two best regex's are * and x|y|z. Any system that produces some other regex is by definition more difficult to understand than regex's themselves. Now circling back to novice users, simple code that iterates over strings with a loop or comprehension is probably going to be less prone to bugs than using regex's because regex's are so powerful. It's also going to be simpler to understand because the tools are familiar and purpose built and the abstractions of the code can be matched to the abstractions of the business logic. What makes regex's hard is that the abstractions are mathematical and intuitions about regex's rely on mathematical intuitions about automata [and in the case of Perl type regex's not simply finite automata due to backtracking and capture]. Please don't misunderstand me, I think the article is a neat piece of work. Even though I think that simplifying regex's isn't really possible because of mathematics, the article may introduce someone to the idea of using them. Because regex's are immensely powerful, that's a good thing. Good luck.

savolai 10y ago
Thanks for your kind words. You still don't see the point though. I am not simplifying regexes one single bit. The regex syntax is quite simply broken. It is fragmented as hell, and its escaping and context rules stem from a time when all we had was ASCII. This might have been acceptable in the eighties, yes? These have nothing to do with whether or not underlying regex theory is mathematically deep or not. I am not touching the theory. It's simply the syntax that is broken. You can go exactly as deep with Regex UCR as you can with traditional regexes. It's the same regex. You just don't have to torture yourself while doing it. We need to move forward from the single-eyed view that everything must be done in text, even if it means wasting countless hours in man hours and loads of cognitive processing. We need that energy for fruitful work. You are currently just advocating bureaucracy. Your looping example is yet another example of a theoretist making enormous assumptions the context of usage. How do you write a loop in the search/replace field of a text editor or an IDE? How does proneness to bugs matter one single bit if you're just using the tool to avoid a bit of manual work? Which [warning, hypothesis:] seems to be mostly what regexes are actually used anyway.
- brudgers 10y ago
  After writing my comment, I went for a walk and ate some Chipolte and thought a bit more. And thought about JWZ's: Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. And now I'd put it like this. It doesn't matter what changes are made to the syntax. It doesn't matter what tools we build. The problem with regex's is that they won't balance parenthesis and not balancing parenthesis is counterintuitive to the ideas of matching and patterns. Even worse, there is no way to understand why problems like balancing parenthesis are impossible without a mental model based on regex's mathematical underpinnings. I should probably admit my bias, if I'm in an IDE or a text editor (or even a browser window), the last thing I want to do is reach for a mouse.
- Tiksi 10y ago
  >and its escaping and context rules stem from a time when all we had was ASCII. My keyboard still only has ~ASCII on it, so having to input anything else is a huge barrier to entry. Regex syntax sucks, I'll be the first to agree with that, but it's more of an annoyance. If I write some regex to match ips(obviously not properly, but quick rough match): ([0-9]\.){1,3}[0-9]{1,3} My thought process isn't "open paren, open bracket, zero" etc, it's more of "Ok, I have a 4 groups of 1 to 3 numbers separated by dots, which can be written as 3 groups followed by a dot followed by a fourth group" and my hands translate it the rest of the way into text. Knowing exactly which character to use may have been a hindrance the first few times I ran into regex, but the real difficulty is in knowing what to match and how to match it. Providing that info to your computer once you know it is the easy part, even if the interface is horrible. It's like driving a car or a bike. You don't think "Ok, time to move my hand to move the steering wheel to the right and that will turn the wheels which will then the car to the right", you just need to turn right and your subconscious takes care of the rest. Improving the syntax would be like trying to improve how well/safely you drive by changing the steering wheel. Sure, it might help a little bit, but that's not really the difficult part of driving well, just an interface to it.