3 ms·

I've never used a language mode that worked properly in all cases. That's why I've stopped using them, aside from very clever and extraordinarily useful ones li

by mbrock 11y ago

I've never used a language mode that worked properly in all cases. That's why I've stopped using them, aside from very clever and extraordinarily useful ones like paredit. It terrifies me to think about how these language modes contain regexps for matching regexp syntax. Of course those are wrong.

masklinn 11y ago
> I've never used a language mode that worked properly in all cases. You need a full parser for that to happen, most mode authors don't bother[0]. JS2-mode (https://github.com/mooz/js2-mode/ https://github.com/mooz/js2-mode/) does that (or attempts to). Most syntactic colorisers are souped-up tokenisers, not actual parsers. It suffices in 99% of cases, but breaks badly elsewhere. [0] it would help if more languages made fast parsing machinery externally available as a convenient API. ALAS, that's rarely the case, even less so for fast parsing machinery which doesn't throw the information you need for syntactic coloration out the window.
- hvis 11y ago
  I'm fairly sure that Emacs's go-mode, which doesn't use a full-blown parser, won't make the mistake of treating a paren inside a string as if it were outside either. Or if it did, that would be easy to fix. Emacs has a generic lightweight parsing facility that can tell you if you're inside a string, or a comment, or how to skip over a string, or a pair of matching parens/braces/brackets, and so on.
- mbrock 11y ago
  The issue tracker for js2-mode illustrates that such a parsing approach is no guarantee of real-world accuracy either. https://github.com/mooz/js2-mode/issues https://github.com/mooz/js2-mode/issues The goal just shifts from having a semi-accurate regexp tokenizer, to having the language mode's complex parser match the complex parser of the language it targets—which will pretty much never happen, unless the language is exquisitely simple, like maybe restricted dialects of Scheme... or Brainfuck.
  - hvis 11y ago
    js2-mode supports ES6 fairly accurately. But of course, that's not enough, people keep asking for new things: JavaScript mixed with HTML, Facebook Flow support, JSX, new ES7 (still unstable) syntax extensions, and so on. It would be easier with a more stable language. I.e. not JavaScript, which is still going through growing pains. Even so, "25 Open, 204 Closed" seems pretty good to me.
    - mbrock 11y ago
      fundamental-mode hasn't failed me even once, is what I mean. :) I'm not criticizing js2-mode, it's just that I have personally given up hope for accurate language modes in general, and realized that they are a lot of effort and IMO relatively little use.
- softawre 11y ago
  [0] - typescript does this I believe
- nv-vn 11y ago
  Does Atom have an API for syntax highlighting with full parsers? I'm pretty sure it only supports bix trees of regexes for that.
omaranto 11y ago
The Textadept editor uses PEGs for its language modes. I haven't tested them but that sounds like it has a chance of being pretty good.

by mbrock 11y ago

masklinn 11y ago

> I've never used a language mode that worked properly in all cases. You need a full parser for that to happen, most mode authors don't bother[0]. JS2-mode (https://github.com/mooz/js2-mode/ https://github.com/mooz/js2-mode/) does that (or attempts to). Most syntactic colorisers are souped-up tokenisers, not actual parsers. It suffices in 99% of cases, but breaks badly elsewhere. [0] it would help if more languages made fast parsing machinery externally available as a convenient API. ALAS, that's rarely the case, even less so for fast parsing machinery which doesn't throw the information you need for syntactic coloration out the window.

hvis 11y ago
I'm fairly sure that Emacs's go-mode, which doesn't use a full-blown parser, won't make the mistake of treating a paren inside a string as if it were outside either. Or if it did, that would be easy to fix. Emacs has a generic lightweight parsing facility that can tell you if you're inside a string, or a comment, or how to skip over a string, or a pair of matching parens/braces/brackets, and so on.
mbrock 11y ago
The issue tracker for js2-mode illustrates that such a parsing approach is no guarantee of real-world accuracy either. https://github.com/mooz/js2-mode/issues https://github.com/mooz/js2-mode/issues The goal just shifts from having a semi-accurate regexp tokenizer, to having the language mode's complex parser match the complex parser of the language it targets—which will pretty much never happen, unless the language is exquisitely simple, like maybe restricted dialects of Scheme... or Brainfuck.
- hvis 11y ago
  js2-mode supports ES6 fairly accurately. But of course, that's not enough, people keep asking for new things: JavaScript mixed with HTML, Facebook Flow support, JSX, new ES7 (still unstable) syntax extensions, and so on. It would be easier with a more stable language. I.e. not JavaScript, which is still going through growing pains. Even so, "25 Open, 204 Closed" seems pretty good to me.
  - mbrock 11y ago
    fundamental-mode hasn't failed me even once, is what I mean. :) I'm not criticizing js2-mode, it's just that I have personally given up hope for accurate language modes in general, and realized that they are a lot of effort and IMO relatively little use.
softawre 11y ago
[0] - typescript does this I believe
nv-vn 11y ago
Does Atom have an API for syntax highlighting with full parsers? I'm pretty sure it only supports bix trees of regexes for that.

omaranto 11y ago

The Textadept editor uses PEGs for its language modes. I haven't tested them but that sounds like it has a chance of being pretty good.