5 ms·

I'm generally very sympathetic to regex bugs (especially in a language like JavaScript where you don't get nice expanded multiline regexes with comments), but I

I'm generally very sympathetic to regex bugs (especially in a language like JavaScript where you don't get nice expanded multiline regexes with comments), but I am wondering why they went with a regex in the first place. Did they decide `document.location.host` was too brittle for some reason?

taneq 10y ago
I'm not sympathetic to regex bugs where they're being used to parse untrusted user input which is then later used to do something important (like, say, pick which URL to submit credentials to). They're way too easy to cock up for anything security-related.
techdragon 10y ago
I'd agree if there wasn't an extremely good solution to this problem. Verbal Expressions - It's an extremely good higher level interface to the underlying regular expressions tools, in MANY languages. Including: JavaScript - https://github.com/VerbalExpressions/JSVerbalExpressions https://github.com/VerbalExpressions/JSVerbalExpressions ActionScript 3 - https://github.com/VerbalExpressions/AS3VerbalExpressions https://github.com/VerbalExpressions/AS3VerbalExpressions Clojure - https://github.com/VerbalExpressions/ClojureVerbalExpressions https://github.com/VerbalExpressions/ClojureVerbalExpression... C++ - https://github.com/VerbalExpressions/CppVerbalExpressions https://github.com/VerbalExpressions/CppVerbalExpressions C# - https://github.com/VerbalExpressions/CSharpVerbalExpressions https://github.com/VerbalExpressions/CSharpVerbalExpressions Dart - https://github.com/VerbalExpressions/DartVerbalExpressions https://github.com/VerbalExpressions/DartVerbalExpressions Elixir - https://github.com/VerbalExpressions/ElixirVerbalExpressions https://github.com/VerbalExpressions/ElixirVerbalExpressions Elm - https://github.com/VerbalExpressions/elm-verbal-expressions https://github.com/VerbalExpressions/elm-verbal-expressions Erlang - https://github.com/VerbalExpressions/ErlangVerbalExpressions https://github.com/VerbalExpressions/ErlangVerbalExpressions FreeBasic - https://github.com/VerbalExpressions/FreeBasicVerbalExpressions https://github.com/VerbalExpressions/FreeBasicVerbalExpressi... F# - https://github.com/VerbalExpressions/FSharpVerbalExpressions https://github.com/VerbalExpressions/FSharpVerbalExpressions Go - https://github.com/VerbalExpressions/GoVerbalExpressions https://github.com/VerbalExpressions/GoVerbalExpressions Groovy - https://github.com/VerbalExpressions/GroovyVerbalExpressions https://github.com/VerbalExpressions/GroovyVerbalExpressions Haskell - https://github.com/VerbalExpressions/HaskellVerbalExpressions https://github.com/VerbalExpressions/HaskellVerbalExpression... Haxe - https://github.com/VerbalExpressions/HaxeVerbalExpressions https://github.com/VerbalExpressions/HaxeVerbalExpressions Java - https://github.com/VerbalExpressions/JavaVerbalExpressions https://github.com/VerbalExpressions/JavaVerbalExpressions Lua - https://github.com/VerbalExpressions/LuaVerbalExpressions https://github.com/VerbalExpressions/LuaVerbalExpressions Objective C - https://github.com/VerbalExpressions/ObjectiveCVerbalExpressions https://github.com/VerbalExpressions/ObjectiveCVerbalExpress... Perl - https://github.com/VerbalExpressions/PerlVerbalExpressions https://github.com/VerbalExpressions/PerlVerbalExpressions PHP - https://github.com/VerbalExpressions/PHPVerbalExpressions https://github.com/VerbalExpressions/PHPVerbalExpressions PowerShell - https://github.com/VerbalExpressions/PowerShellVerbalExpressions https://github.com/VerbalExpressions/PowerShellVerbalExpress... PureScript - https://github.com/VerbalExpressions/purescript-verbal-expressions https://github.com/VerbalExpressions/purescript-verbal-expre... Python - https://github.com/VerbalExpressions/PythonVerbalExpressions https://github.com/VerbalExpressions/PythonVerbalExpressions Racket - https://github.com/VerbalExpressions/RacketVerbalExpressions https://github.com/VerbalExpressions/RacketVerbalExpressions Ruby - https://github.com/VerbalExpressions/RubyVerbalExpressions https://github.com/VerbalExpressions/RubyVerbalExpressions Rust - https://github.com/VerbalExpressions/RustVerbalExpressions https://github.com/VerbalExpressions/RustVerbalExpressions Scala - https://github.com/VerbalExpressions/ScalaVerbalExpressions https://github.com/VerbalExpressions/ScalaVerbalExpressions Swift - https://github.com/VerbalExpressions/SwiftVerbalExpressions https://github.com/VerbalExpressions/SwiftVerbalExpressions Vala - https://github.com/VerbalExpressions/ValaVerbalExpressions https://github.com/VerbalExpressions/ValaVerbalExpressions And probably more, but that's just the "official" implementations.
- obsurveyor 10y ago
  You could have just linked to http://verbalexpressions.github.io http://verbalexpressions.github.io instead of spamming all the repositories. Also, about half of them are out of date by 3 or more years.
  - elktea 10y ago
    The point being it's available for a wide range of languages
    - techdragon 10y ago
      Exactly... I probably could have posted less of them but in an era of TLDR it was easy enough to just post the list and let the uninterested scroll past and ignore it. That I have down votes for pointing out how practically no developers are without an option to guard against poorly written regular expressions, feels somewhat overkill.
- dpark 10y ago
  I don't see how this would have prevented this problem. The issue was not that regular expressions were "too hard" for the lastpass team, but that URLs are hard to parse correctly. To put it another way, if you don't know how to parse a URL correctly, you'll probably write an incorrect parser no matter what parsing tool you use. That's why you generally shouldn't write parsers for URLs.
K0nserv 10y ago
Not using `document.location.host` stood out to me too. I think the takeaway here is don't use regex unless you absolutely have to and don't use it to parse things that have rigorous standards describing them. Emails, phone numbers, URLs come to mind.
- gorhill 10y ago
  > don't use it to parse things that have rigorous standards describing them Where a regex must be used, there is a reference regex for parsing URL: https://tools.ietf.org/html/rfc3986#appendix-B https://tools.ietf.org/html/rfc3986#appendix-B Edit: a permalink to demonstrate the above reference regex: https://regex101.com/r/yJ5nU4/1 https://regex101.com/r/yJ5nU4/1 -- would have prevented the LastPass bug.
  - sebcat 10y ago
    I once came across a bug in a codebase I inherited that parsed URI-references from HTML documents. That bug was caused by using that very same regular expression. The string "foo" correctly gets parsed as a path (group 5). However the string "foo;key:value" gets parsed as a "foo;key" scheme (group 1,2) and a "bar" path (group 5) by the regular expression, but as a path if you follow the grammar. scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) vs. ^(([^:\/?#]+):)? :; are reserved characters, but they do not need to be encoded when they are a part of the path. So yeah, don't use regular expressions for parsing.

taneq 10y ago

I'm not sympathetic to regex bugs where they're being used to parse untrusted user input which is then later used to do something important (like, say, pick which URL to submit credentials to). They're way too easy to cock up for anything security-related.

techdragon 10y ago

I'd agree if there wasn't an extremely good solution to this problem. Verbal Expressions - It's an extremely good higher level interface to the underlying regular expressions tools, in MANY languages. Including: JavaScript - https://github.com/VerbalExpressions/JSVerbalExpressions https://github.com/VerbalExpressions/JSVerbalExpressions ActionScript 3 - https://github.com/VerbalExpressions/AS3VerbalExpressions https://github.com/VerbalExpressions/AS3VerbalExpressions Clojure - https://github.com/VerbalExpressions/ClojureVerbalExpressions https://github.com/VerbalExpressions/ClojureVerbalExpression... C++ - https://github.com/VerbalExpressions/CppVerbalExpressions https://github.com/VerbalExpressions/CppVerbalExpressions C# - https://github.com/VerbalExpressions/CSharpVerbalExpressions https://github.com/VerbalExpressions/CSharpVerbalExpressions Dart - https://github.com/VerbalExpressions/DartVerbalExpressions https://github.com/VerbalExpressions/DartVerbalExpressions Elixir - https://github.com/VerbalExpressions/ElixirVerbalExpressions https://github.com/VerbalExpressions/ElixirVerbalExpressions Elm - https://github.com/VerbalExpressions/elm-verbal-expressions https://github.com/VerbalExpressions/elm-verbal-expressions Erlang - https://github.com/VerbalExpressions/ErlangVerbalExpressions https://github.com/VerbalExpressions/ErlangVerbalExpressions FreeBasic - https://github.com/VerbalExpressions/FreeBasicVerbalExpressions https://github.com/VerbalExpressions/FreeBasicVerbalExpressi... F# - https://github.com/VerbalExpressions/FSharpVerbalExpressions https://github.com/VerbalExpressions/FSharpVerbalExpressions Go - https://github.com/VerbalExpressions/GoVerbalExpressions https://github.com/VerbalExpressions/GoVerbalExpressions Groovy - https://github.com/VerbalExpressions/GroovyVerbalExpressions https://github.com/VerbalExpressions/GroovyVerbalExpressions Haskell - https://github.com/VerbalExpressions/HaskellVerbalExpressions https://github.com/VerbalExpressions/HaskellVerbalExpression... Haxe - https://github.com/VerbalExpressions/HaxeVerbalExpressions https://github.com/VerbalExpressions/HaxeVerbalExpressions Java - https://github.com/VerbalExpressions/JavaVerbalExpressions https://github.com/VerbalExpressions/JavaVerbalExpressions Lua - https://github.com/VerbalExpressions/LuaVerbalExpressions https://github.com/VerbalExpressions/LuaVerbalExpressions Objective C - https://github.com/VerbalExpressions/ObjectiveCVerbalExpressions https://github.com/VerbalExpressions/ObjectiveCVerbalExpress... Perl - https://github.com/VerbalExpressions/PerlVerbalExpressions https://github.com/VerbalExpressions/PerlVerbalExpressions PHP - https://github.com/VerbalExpressions/PHPVerbalExpressions https://github.com/VerbalExpressions/PHPVerbalExpressions PowerShell - https://github.com/VerbalExpressions/PowerShellVerbalExpressions https://github.com/VerbalExpressions/PowerShellVerbalExpress... PureScript - https://github.com/VerbalExpressions/purescript-verbal-expressions https://github.com/VerbalExpressions/purescript-verbal-expre... Python - https://github.com/VerbalExpressions/PythonVerbalExpressions https://github.com/VerbalExpressions/PythonVerbalExpressions Racket - https://github.com/VerbalExpressions/RacketVerbalExpressions https://github.com/VerbalExpressions/RacketVerbalExpressions Ruby - https://github.com/VerbalExpressions/RubyVerbalExpressions https://github.com/VerbalExpressions/RubyVerbalExpressions Rust - https://github.com/VerbalExpressions/RustVerbalExpressions https://github.com/VerbalExpressions/RustVerbalExpressions Scala - https://github.com/VerbalExpressions/ScalaVerbalExpressions https://github.com/VerbalExpressions/ScalaVerbalExpressions Swift - https://github.com/VerbalExpressions/SwiftVerbalExpressions https://github.com/VerbalExpressions/SwiftVerbalExpressions Vala - https://github.com/VerbalExpressions/ValaVerbalExpressions https://github.com/VerbalExpressions/ValaVerbalExpressions And probably more, but that's just the "official" implementations.

obsurveyor 10y ago
You could have just linked to http://verbalexpressions.github.io http://verbalexpressions.github.io instead of spamming all the repositories. Also, about half of them are out of date by 3 or more years.
- elktea 10y ago
  The point being it's available for a wide range of languages
  - techdragon 10y ago
    Exactly... I probably could have posted less of them but in an era of TLDR it was easy enough to just post the list and let the uninterested scroll past and ignore it. That I have down votes for pointing out how practically no developers are without an option to guard against poorly written regular expressions, feels somewhat overkill.
dpark 10y ago
I don't see how this would have prevented this problem. The issue was not that regular expressions were "too hard" for the lastpass team, but that URLs are hard to parse correctly. To put it another way, if you don't know how to parse a URL correctly, you'll probably write an incorrect parser no matter what parsing tool you use. That's why you generally shouldn't write parsers for URLs.

K0nserv 10y ago

Not using `document.location.host` stood out to me too. I think the takeaway here is don't use regex unless you absolutely have to and don't use it to parse things that have rigorous standards describing them. Emails, phone numbers, URLs come to mind.

gorhill 10y ago
> don't use it to parse things that have rigorous standards describing them Where a regex must be used, there is a reference regex for parsing URL: https://tools.ietf.org/html/rfc3986#appendix-B https://tools.ietf.org/html/rfc3986#appendix-B Edit: a permalink to demonstrate the above reference regex: https://regex101.com/r/yJ5nU4/1 https://regex101.com/r/yJ5nU4/1 -- would have prevented the LastPass bug.
- sebcat 10y ago
  I once came across a bug in a codebase I inherited that parsed URI-references from HTML documents. That bug was caused by using that very same regular expression. The string "foo" correctly gets parsed as a path (group 5). However the string "foo;key:value" gets parsed as a "foo;key" scheme (group 1,2) and a "bar" path (group 5) by the regular expression, but as a path if you follow the grammar. scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) vs. ^(([^:\/?#]+):)? :; are reserved characters, but they do not need to be encoded when they are a part of the path. So yeah, don't use regular expressions for parsing.