3 ms·

> I have not confirmed this but I have been told the regexp portion is tightly controlled by Google, more than most parts of the spec. When v8's regexp implemen

> I have not confirmed this but I have been told the regexp portion is tightly controlled by Google, more than most parts of the spec. When v8's regexp implementation is found to deviate from the spec, the spec gets changed. This makes sense (why risk breaking websites), but it illustrates how little weight the regexp spec actually has. A few responses: 1. This was not at all my experience working with the V8 team. For example, when I noticed that Irregexp did not properly implement the Canonicalize algorithm for non-unicode case-insensitive comparisons [1], they were happy to accept my patches [2][3]. Nobody ever suggested changing the spec. 2. Nothing changes in the JS spec without consensus from all the major stakeholders. See, for example, the second sentence in the TC39 process document here [4]: "The committee operates by consensus and has discretion to alter the specification as it sees fit." Mozilla can block (and has blocked) proposals that it feels are bad for the web platform. Google has no special power to bend the committee to its will. 3. Another way of saying "when V8's regexp implementation is found to deviate from the spec, the spec gets changed" is "when the spec is found to deviate from the consensus of implementations, we correct the spec". The Chakra team's post notes: "In practice, all browsers accept regular expressions such as `/]/` and web developers write them." The purpose of the spec is to make the web work better by maximizing compatibility between implementations. In cases where all implementations agree, and real websites depend on that behaviour, the spec should change to match reality. Don't get me wrong: Chromium monoculture is a real problem. If we thought that Google was going to add a bunch of new regexp features without going through the standards process, or if we had plans to prototype regexp features that we feared Google wouldn't accept upstream, we might have made a different decision. But regexps are not a plausible battleground, and the compatibility benefits of sharing code outweigh the harms. PS: regress looks really sweet, and I am excited to see how it turns out. [1] https://tc39.es/ecma262/#sec-runtime-semantics-canonicalize-ch https://tc39.es/ecma262/#sec-runtime-semantics-canonicalize-... [2] https://github.com/v8/v8/commit/3fab9d05cf34a7f0bc0e9405729ab8b78c0671ac https://github.com/v8/v8/commit/3fab9d05cf34a7f0bc0e9405729a... [3] https://github.com/v8/v8/commit/b65fcfe92566415c8aae373d5fb5a9b16bed21ce https://github.com/v8/v8/commit/b65fcfe92566415c8aae373d5fb5... [4] https://tc39.es/process-document/ https://tc39.es/process-document/

IainIreland 6y ago
tl;dr: The spec exists for the web; the web does not exists for the spec.
- hajile 6y ago
  Or another reading is that Google is yet again abusing their dominant position to alter the spec to suit their needs instead of the need of people using non-dominant browsers.
ridiculous_fish 6y ago
Great reply, thank you for sharing your perspective. First, I confess to some skepticism of the standardization process. For example, consider ES2018 lookbehinds. At the point of standardization, only one browser implemented them: v8. And in the standardized form (arbitrary width) they are very difficult to retrofit onto existing engines. Moddable had to throw their engine out and start over [1]. JSC's YARR still doesn't support them, neither does SpiderMonkey. Did all stakeholders really agree on this feature and then not implement it? Or was this just v8 in the driver's seat? Second, regexps actually do have a role in driving monoculture. You can't polyfill regexp syntax. At one point, Steam's website only worked on Chrome because only v8 implemented lookbehinds [2] - again because it's hard to retrofit. Lastly, it remains a problem that one cannot implement a conforming JS regexp engine from the spec alone. The rest of JavaScript does not have that problem, only regexp. And I disagree with the "real websites depend on that behaviour" qualifier. No website depended on /{1}/ being invalid syntax, yet it was made so in ES8. Incidentally, it's amusing that v8's Canonicalize was buggy. Non-Unicode case-insensitive char ranges is THE ugliest part of regexp (QuickJS doesn't even try [3]) and I just assumed it was codifying v8's implementation. Guess it was someone else's, hah. 1: https://blog.moddable.com/blog/regexp/ https://blog.moddable.com/blog/regexp/ 2: https://github.com/webcompat/web-bugs/issues/51385 https://github.com/webcompat/web-bugs/issues/51385 3: https://github.com/ldarren/QuickJS/blob/mod/libregexp.c#L259 https://github.com/ldarren/QuickJS/blob/mod/libregexp.c#L259
- IainIreland 6y ago
  The TC39 minutes are all online [1]. You can see the initial introduction of lookbehind assertions [2], the meeting where they reached stage 3 [3], a status update while waiting for more implementations [4], and then the eventual agreement to move them to stage 4 (full standardization)[5]. They reached that point because: a) everybody agreed that they were a good feature, with precedent in other languages; b) the spec text had been scrutinized by the people who like to spend time scrutinizing spec text; and c) two implementations (Moddable and V8) had successfully implemented them. That's the process, and it was followed to the letter. Does that put a burden on other engines to keep up? Sure. But that's what we signed up for. The webcompat bug you linked is from April 2020, more than two years after lookbehind assertions were standardized. That's nobody's fault but ours, and we did this project so that we don't end up in a similar situation in the future. I'm not sure what your source is for JS regexps being impossible to implement from the spec, but if you have specific points of concern, you should open an issue [6]. Writing a spec is hard work, and things get missed, but the people working on this are genuinely trying their best. (As far as I can tell, Canonicalize is what everybody thought they were implementing. V8 tried to get clever with ICU and ran facefirst into some of the dark corners of Unicode.) 1: https://github.com/tc39/notes/tree/master/meetings https://github.com/tc39/notes/tree/master/meetings 2: https://github.com/tc39/notes/blob/master/meetings/2016-11/nov-30.md#12ivc-regexp-lookbehind https://github.com/tc39/notes/blob/master/meetings/2016-11/n... 3: https://github.com/tc39/notes/blob/master/meetings/2017-03/mar-21.md#10ib-regexp-lookbehind-assertions https://github.com/tc39/notes/blob/master/meetings/2017-03/m... 4: https://github.com/tc39/notes/blob/master/meetings/2017-05/may-23.md#16ic-status-update-on-regexp-proposals-lookbehind-unicode-properties-dotall-flag-and-named-groups-status-update https://github.com/tc39/notes/blob/master/meetings/2017-05/m... 5: https://github.com/tc39/notes/blob/master/meetings/2018-01/jan-24.md#13iij-regexp-lookbehind-assertions-for-stage-4 https://github.com/tc39/notes/blob/master/meetings/2018-01/j... 6: https://github.com/tc39/ecma262/issues https://github.com/tc39/ecma262/issues

IainIreland 6y ago

tl;dr: The spec exists for the web; the web does not exists for the spec.

hajile 6y ago
Or another reading is that Google is yet again abusing their dominant position to alter the spec to suit their needs instead of the need of people using non-dominant browsers.

ridiculous_fish 6y ago

Great reply, thank you for sharing your perspective. First, I confess to some skepticism of the standardization process. For example, consider ES2018 lookbehinds. At the point of standardization, only one browser implemented them: v8. And in the standardized form (arbitrary width) they are very difficult to retrofit onto existing engines. Moddable had to throw their engine out and start over [1]. JSC's YARR still doesn't support them, neither does SpiderMonkey. Did all stakeholders really agree on this feature and then not implement it? Or was this just v8 in the driver's seat? Second, regexps actually do have a role in driving monoculture. You can't polyfill regexp syntax. At one point, Steam's website only worked on Chrome because only v8 implemented lookbehinds [2] - again because it's hard to retrofit. Lastly, it remains a problem that one cannot implement a conforming JS regexp engine from the spec alone. The rest of JavaScript does not have that problem, only regexp. And I disagree with the "real websites depend on that behaviour" qualifier. No website depended on /{1}/ being invalid syntax, yet it was made so in ES8. Incidentally, it's amusing that v8's Canonicalize was buggy. Non-Unicode case-insensitive char ranges is THE ugliest part of regexp (QuickJS doesn't even try [3]) and I just assumed it was codifying v8's implementation. Guess it was someone else's, hah. 1: https://blog.moddable.com/blog/regexp/ https://blog.moddable.com/blog/regexp/ 2: https://github.com/webcompat/web-bugs/issues/51385 https://github.com/webcompat/web-bugs/issues/51385 3: https://github.com/ldarren/QuickJS/blob/mod/libregexp.c#L259 https://github.com/ldarren/QuickJS/blob/mod/libregexp.c#L259

IainIreland 6y ago
The TC39 minutes are all online [1]. You can see the initial introduction of lookbehind assertions [2], the meeting where they reached stage 3 [3], a status update while waiting for more implementations [4], and then the eventual agreement to move them to stage 4 (full standardization)[5]. They reached that point because: a) everybody agreed that they were a good feature, with precedent in other languages; b) the spec text had been scrutinized by the people who like to spend time scrutinizing spec text; and c) two implementations (Moddable and V8) had successfully implemented them. That's the process, and it was followed to the letter. Does that put a burden on other engines to keep up? Sure. But that's what we signed up for. The webcompat bug you linked is from April 2020, more than two years after lookbehind assertions were standardized. That's nobody's fault but ours, and we did this project so that we don't end up in a similar situation in the future. I'm not sure what your source is for JS regexps being impossible to implement from the spec, but if you have specific points of concern, you should open an issue [6]. Writing a spec is hard work, and things get missed, but the people working on this are genuinely trying their best. (As far as I can tell, Canonicalize is what everybody thought they were implementing. V8 tried to get clever with ICU and ran facefirst into some of the dark corners of Unicode.) 1: https://github.com/tc39/notes/tree/master/meetings https://github.com/tc39/notes/tree/master/meetings 2: https://github.com/tc39/notes/blob/master/meetings/2016-11/nov-30.md#12ivc-regexp-lookbehind https://github.com/tc39/notes/blob/master/meetings/2016-11/n... 3: https://github.com/tc39/notes/blob/master/meetings/2017-03/mar-21.md#10ib-regexp-lookbehind-assertions https://github.com/tc39/notes/blob/master/meetings/2017-03/m... 4: https://github.com/tc39/notes/blob/master/meetings/2017-05/may-23.md#16ic-status-update-on-regexp-proposals-lookbehind-unicode-properties-dotall-flag-and-named-groups-status-update https://github.com/tc39/notes/blob/master/meetings/2017-05/m... 5: https://github.com/tc39/notes/blob/master/meetings/2018-01/jan-24.md#13iij-regexp-lookbehind-assertions-for-stage-4 https://github.com/tc39/notes/blob/master/meetings/2018-01/j... 6: https://github.com/tc39/ecma262/issues https://github.com/tc39/ecma262/issues