43 ms·

Making bracket pair colorization faster

711 points by feross 5y ago

sesuximo 5y ago
Great news!
- curiousgal 5y ago
  Not for the extension developer though. Good thing it was not monetized.
  - sparsely 5y ago
    Well, he no longer has to maintain it, and an even better version of the feature now exists
  - Igelau 5y ago
    https://github.com/microsoft/vscode/issues/128465#issuecomment-900311495 https://github.com/microsoft/vscode/issues/128465#issuecomme...
  - deleted 5y ago
    [deleted]
  - jffry 5y ago
    Igelau linked to this thread: https://github.com/microsoft/vscode/issues/128465#issuecomment-900311495 https://github.com/microsoft/vscode/issues/128465#issuecomme... In it, CoenraadS (the extension developer) writes "I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it." and "If it could that would be great, and my extension could be deprecated completely." There was also discussion with the author about updating the extension to prompt users to switch to the native functionality, and it appears CoenraadS was also involved in the process/design of implementing the feature in VS code. This appears to be a shining example of doing it right, and giving the original extension author credit.
DenisM 5y ago
Background code analysis in Visual Studio grinds my windows VM to a halt after every few keystrokes. Yet they spend time speeding up colorization of bracket pairs, which worked just fine for the last 10 releases or so.
- schrute 5y ago
  > Yet they spend time speeding up colorization of bracket pairs, which worked just fine for the last 10 releases or so. You must have missed the part where it is now 10,000x faster. Given your complaint about performance this should excite you!
- freemint 5y ago
  If VS is to slow, you need to use out more for the Background code analysis to amortize.
  - DenisM 5y ago
    I don't get it.
- SquareWheel 5y ago
  Are you mixing up Visual Studio with VS Code? This pertains to the latter.
  - DenisM 5y ago
    Seems so. Is there no background code analysis in VS code? That would be a reason to switch.
    - int_19h 5y ago
      In both VS and VSCode, semantic analysis is handled by specific language extensions (some of which might ship in the box, as C# is in VS and TypeScript is in VSCode). So it's impossible to answer this question without knowing which language you're using.
gpderetta 5y ago
Interestingly rainbow-delimiters in emacs seems to be instantaneous even with elisp being an slow interpreted language. For better or worse emacs does run "extensions" synchronously; this allows extremely fine grained control of the buffer, but a slow extension can kill interactive performance.
- tempodox 5y ago
  Similar with `lisp_rainbow` via the slimv plugin in vim. It uses vim's built-in syntax coloring mechanism and doesn't slow down the editor.
  - gowld 5y ago
    That's what VSCode team did too -- moved the computational work from a plugin to core.
- gowld 5y ago
  Have you tried rainbow-delimiters on a 42 kilo-line file? That's the benchmark in the OP.
  - gpderetta 5y ago
    Just tried. Colorization seems still almost instantaneous. Emacs does generally not like such a huge file though (reindenting the whole buffer is slow for example).
- dspillett 5y ago
  > elisp being an slow interpreted language The issue isn't about the speed of the language, but the code not having access to information it needs from the extension API so having to go long ways around to work it out. The new version may be in a faster base language, but the majority of its speed boost comes from having access to information that the editor already knows without having to mess around re-deriving it.
unwind 5y ago
That sounds awesome for VS Code users in general, but I guess it's sad news for the independent developer (CoenraadS) who made the pair colorization extension popular and famous. With the editor's native performance 10,000 times better, I suepect the number of installs of their extension will plummet. It's hard to compete with 1st party powers, as all the hoopla around app stores (and before that, the famous "Embrace, extend & extinguish" [1] strategy) show all the time. That said, I'm not saying it's a hostile move, it's just ... interesting. [1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
- 1_player 5y ago
  What? If the independent developer didn't want his technology to be used, he would have patented it or made it closed source. This is not what EEE is about.
- Gehinnn 5y ago
  Author of the blog post here. We openly discussed various approaches with CoenraadS and other extension authors here: https://github.com/microsoft/vscode/issues/128465#issuecomment-879089188 https://github.com/microsoft/vscode/issues/128465#issuecomme...
  - rattray 5y ago
    Thanks for your awesome work! Can't wait to try it.
  - gnrlst 5y ago
    Quoting CoenraadS: > Author of Bracket Pair Colorizer here. > I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it. > If there are some quick wins, I can still apply them, but I think my extension is so hacky it is easier to do 1.b or 1.c Seems CoenraadS is completely in favor of the native implementation (and probably could have been called out in the blog post as few people will find this context).
- cstuder 5y ago
therealmarv 5y ago
Deactivated the "Bracket Pair Colorizer" extension from CoenraadS which I've first installed years ago. Activated the internal Bracket Pair Colorizer. First world problems: Bracket colors are different now and especially XML looks off... seems I need to fine tune that some day (but not today).
- rattray 5y ago
  Or open an issue on GitHub with a few screenshots? Vs code devs are iterating!
- pachico 5y ago
  As a colour blind, I distinguish colours are different but I don't care/wouldn't know what they are: perfect!
- spartanatreyu 5y ago
  You can change the colors in the settings.
naranha 5y ago
> Brackets are queried when rendering the viewport and thus querying them has to be really fast. Then by querying them only when the document changes could make it another 10,000x faster ;)
- antris 5y ago
  Great improvement of performance for those who like to code blind!
414owen 5y ago
So, what's the plan? Reimplement every popular extension that doesn't have good performance? > the asynchronous communication between the renderer and the extension-host severely limits how fast bracket pair colorization can be when implemented as an extension. This limit cannot be overcome Nope, I'm not convinced. If it can be done internally, then you should expose more internals until it's possible to do it with the public API. I bet this could be made really performant if vscode had incremental parsing, like tree-sitter+Atom.
- LewisVerstappen 5y ago
  There were additional issues too, like access to token information discussed here > This is another challenge of the Bracket Pair Colorization extension that affects performance negatively: it does not have access to these tokens and has to recompute them on its own. We thought long about how we could efficiently and reliably expose token information to extensions, but came to the conclusion that we cannot do this without a lot of implementation details leaking into the extension API. Because the extension still has to send over a list of color decorations for each bracket in the document, such an API alone would not even solve the performance problem.
  - TeMPOraL 5y ago
    > access to token information This is perhaps the core difference between VS Code and Emacs (which a lot of people believe is being superseded by VS Code as the "extensible editor") - in Emacs, there is no such thing as limiting access to information. Outside of things that get hidden accidentally[0], any bit of elisp code can access everything. It's not just a practical difference, but also a philosophical one: Emacs plugins are not designed to expose a narrow API, because it's impossible to enforce anyway. There's a structure to it, so nothing prevents one from creating good abstractions - those abstractions just have to be designed with extensibility and interoperability in mind[1], because the users (including other package developers) always have an option to just hook into, advise, override or replace any piece of code in your package. Performance-wise, the impact of it varies. On the one hand, this level of flexibility prevents Emacs from making important breaking changes[2]. On the other hand, nothing ever has to wait for a better API design - if there's a way to make a feature faster by hooking to a dependency's internal, people will do just that, and keep doing that until the API blesses the use case. -- [0] - Like state the C core doesn't expose, or some implicit state shared by a bunch of closures - though you can get at the latter if you override whatever is hiding the state. [1] - E.g. by offering hooks as a blessed, well-defined and stable way to interact with internals to cover 90% of interoperability needs. [2] - Like doing proper multithreading, or replacing Emacs Lisp with a more polished Lisp - though myself I feel ELisp is good enough as a language.
EMM_386 5y ago
VS Code is an amazing piece of technology, despite all the "Electron hate" (although changing to WebView2). I recently commented on it here: https://news.ycombinator.com/item?id=28556588 https://news.ycombinator.com/item?id=28556588 Microsoft engineers are top-notch and this is another feature I'll be using daily. Good stuff!
- TeMPOraL 5y ago
  VS Code is the benchmark of what can be done with Electron - if you really, really care. But it's an extreme outlier. The hate for Electron comes from how the average Electron application works. Not only almost nobody cares as much as VS Code team does, the very choice of using Electron itself is usually an act of not caring.
  - flohofwoe 5y ago
    But this is also why simply switching to OS-native APIs and compiled languages wouldn't help much for the average case. A team that doesn't care about performance in Electron also wouldn't care about peformance in native applications, and performance isn't some magic pixie dust that automatically appears when chosing a different UI framework or programming language, it needs to be actively worked towards (some YMMV of course).
    - IshKebab 5y ago
      > performance isn't some magic pixie dust that automatically appears when chosing a different UI framework or programming language That's a little misleading. If you write the same program in idiomatic C++ and Python then it's almost guaranteed that the C++ version will be much much faster even before you have done any profiling or performance optimisation. So there is some magic pixie dust.
      - goldenkey 5y ago
        C++ doesn't give you asymptotic Big(O) algorithmic superpowers. No language does.
rhexs 5y ago
Awesome! When is the leetcode problem coming so I can practice it for my next interview loop?
PragmaticPulp 5y ago
The original author of the bracket colorizer extension gets a great shout-out in this post and was also involved in the discussions of building the feature. Moreover, he has stated that he was tired of maintaining the extension and seems to be fully in favor of this move: > Author of Bracket Pair Colorizer here. > I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it. The comments here suggesting that the Visual Studio Code team was somehow wrong for making this improvement or that they otherwise wronged the extension author are not reflective of the actual process nor even the extension author’s own feelings. The blog post does a great job of describing why bracket colorization isn’t appropriate for the plug-in architecture anyway. Making it fast can only be accomplished within the core of the editor code which has more direct access to syntax parsing. This is a win for everyone all around. The Code team did a great job with this feature and the associated blog post.
- duxup 5y ago
  We see a lot of situations where things sort of die or fade away when the author(s) just don't have time. This seems like the ideal situation where a highly capable team takes over and the author is happy to see it.
  - int_19h 5y ago
    The single most popular extension for VSCode - the one that adds Python support (https://marketplace.visualstudio.com/items?itemName=ms-python.python https://marketplace.visualstudio.com/items?itemName=ms-pytho...) - is somewhat similar: it was originally written by Don Jayamanne, and later acquired by Microsoft. Although in that case, Don was hired by Microsoft as well, and continued working on it: https://devblogs.microsoft.com/python/don-jayamanne-joins-microsoft/ https://devblogs.microsoft.com/python/don-jayamanne-joins-mi... This can still be seen on GitHub - if you look closely, the official repo at https://github.com/microsoft/vscode-python https://github.com/microsoft/vscode-python is a fork! Curiously, some bits of code went kinda full circle in the process - vscode-python reused the (open source) Python debugger written at Microsoft for Python Tools for Visual Studio.
orliesaurus 5y ago
> The feature can be enabled by adding the setting "editor.bracketPairColorization.enabled": true. Interesting this NEEDS to be enabled. ...or will it be set to `true` on new installs? Seems like a super useful feature that should be on by default to me..
- mmis1000 5y ago
  I think change user's layout suddenly without explaining is definitely a way to speedrun 'how to get all user's hate' in 10 seconds. It should be either 'prompt and let you decide' or 'use new defaults on new install'
  - easton 5y ago
    They’ve been prompting the user on new installs to set their preferences and themes, it’ll probably get rolled into that.
- karaterobot 5y ago
  I think it's a good choice, since developers do not like people messing with their environments without asking. Especially if those people are Microsoft. The strategy of writing a really good, technical article explaining the feature, and getting it marketed on forums where developers can see it and get excited enough to try it seems savvy to me. People who are resistant to change (and likely to complain) will just not be affected. I wouldn't turn it on for existing users, but as you say, having it on for new installs does make sense though!
beermonster 5y ago
For those just wanting to try it out the feature can be enabled by adding the setting "editor.bracketPairColorization.enabled": true.”
- emmelaich 5y ago
  There's a gui option for this but it's a bit mysterious. It shows enabled, but immediately under has an unchecked checkbox.
  - nmeofthestate 5y ago
    Looks like it formats the underlying setting attribute name in the UI by emboldening the final dot-separated word editor.bracketPairColorization.enabled
sorenjan 5y ago
This is a well written and in depth article, it must have taken a lot of time and effort to write. Big thanks to the author and Microsoft for releasing it.
bern4444 5y ago
I wonder why VSCode hasn't yet adopted TreeSitter for syntax parsing. Seems like it solves at least part of the performance issue via incremental parsing
- Gehinnn 5y ago
  (author of the blog post here, personal opinion) Tree-sitter is much more general (and I guess way more complex) and most likely cannot use some tricks we use for "simple" bracket pair parsing. For example, we can almost always re-use nested bracket pairs when characters are inserted/deleted, because the bracket pair language is so simple and it does not matter where a bracket pair is in the AST. But when parsing C# and adding a single opening bracket at the beginning of the file, I doubt namespace/class declarations stay namespace/class declarations. Also, long lists in Tree-sitter seem to cause linearly growing incremental parsing time - that's why we use balanced (2,3) trees. You can try it out on their playground [1] by selecting JavaScript and adding some 100k `{}`s. On my machine, adding a single character takes 50ms. When there are 200k bracket pairs, it takes 100ms. When all these brackets are contained in a single bracket pair however, adding characters after this single root pair is fast again (<1ms). But to be honest, Tree-sitter is still mind boggling fast. [1] https://tree-sitter.github.io/tree-sitter/playground https://tree-sitter.github.io/tree-sitter/playground
  - cormacrelf 5y ago
    On the first point, one idea would be to implement the simple bracket pairs language as a TS grammar, stand-alone and independent of any other syntax highlighting. The C# problem of making the syntax invalid and killing the brace highlights disappears. The linear scanning behaviour you describe is due to the change in the left and right parse context for all of those pairs. Yes, it’s linear when you invalidate a subtree, but in the case of a simple bracket pairs language, the damage is limited to scanning through the children of top level brace pairs, and each child subtree is trivial to check. From the IGLR paper: > In a state-matching implementation, each node representing a nonterminal symbol contains a record of the configuration of the pushdown automaton (the ‘parse state’) when the node was shifted onto the stack. A subtree can be reused when both its left and right context are unchanged … Imagine { is prepended to abc(def, {ghi}). I believe the “abc” needs re-parsing, and so does the () subtree as their left parse state is now the “looking for }” state instead of “looking for ({[“ and “looking for )” respectively. But it’s limited to one level deeper — after you split the () subtree, every subtree inside it is still in the “looking for )” configuration on both sides. Specifically, the “def”, “,” and “{ghi}” are fully reused. There are only three possible parse states, so generally you get a lot of subtree reuse. You get even better subtree skipping performance by making long sequences without an brace into a single node, instead of eg tokenising by word and not grouping them. (In this case, “def, “ instead of splitting that.) So realistically for your C# example, only the namespace node is split, and the linear scan is through all the top level brace pairs within the namespace but no deeper. You’ve described IGLR’s best case scenario for incremental parsing an initial brace insertion. Languages that don’t have namespaces would be worse off (but still not too bad). For more realistic languages than {}{}{}{}{}{}{}{}.js I think this approach would work very well. If I liked brace pairs (I don’t) then I would implement this dead simple new grammar and turn it into a Neovim plugin. The existing nvim-ts-rainbow plugin uses queries on existing languages, so is very good at giving perfect/correct pairs and customising per-language, but exhibits the invalid syntax problem and also performance issues which appear to be resolved for most people, but may be back with bigger documents. (Edit — you would need a few variants to account for comments and strings. That makes it a bit more annoying.)
iaml 5y ago
Seems interesting they chose to limit it to a maximum of 6 colors. Is there some technical reason for that? I'm currently using "Bracket Pair Colorizer" with 8 colors and even with that I can remember a couple places where I've seen this limit got exceeded and coloring started from beginning.
- svat 5y ago
  > Seems interesting they chose to limit it to a maximum of 6 colors. Where do you see this? The article talks about nesting levels being bounded by O(log N) levels where N is the length of the document, and from what I can tell this is just for analysis and it supports arbitrary nesting levels… Oh, is your question about the actual colours used for the brackets, i.e. arbitrary levels of matching are supported, but the colours repeat every 6 levels (e.g. brackets at levels i and i+6 have the same colour)? If so this is a UX choice (I can't see any technical reason) and I imagine the considerations might be something like: - There are only finitely many colours that can be usefully distinguished visually by the average person, so we need to pick some limit L, - The limit L needs to be small enough that all L colours are easily distinguishable and memorable, but large enough that it would never be confusing whether a certain coloured bracket is at nesting level i or i ± L. I guess the thinking may have been that 6 is large enough typically, e.g. it's likely to be clear from context whether a certain bracket (say blue) is at (say) level 2 or level 8.
  - Gehinnn 5y ago
    If there is demand, please file an issue and we can discuss increasing this limit. However, these colors are themeable and thus there can only be finitely many. There is no technical reason for this limit of 6.
    - iaml 5y ago
      I see, I'll definitely make one.
  - iaml 5y ago
    It's in the release notes [0]. > All colors are themeable and up to six colors can be configured. [0] https://code.visualstudio.com/updates/v1_60#_high-performance-bracket-pair-colorization https://code.visualstudio.com/updates/v1_60#_high-performanc...
svat 5y ago
> The core idea is to use a recursive descent parser to build an abstract syntax tree (AST) that describes the structure of all bracket pairs. This is a really nice and clear blog post on an algorithmic challenge wonderfully addressed to a satisfactory conclusion (the rebalancing and node reuse etc work out so perfectly); must have been fun, thanks for sharing! I don't know / haven't thought much about text editors so a question for the authors or anyone who understands: from the mention of "the bracket pair AST", it seems that a separate AST is created just for this bracket-pair problem, is that right? I imagine there was/is already (at least one) another AST already being computed in the editor, for other reasons? If so, how did you decide between trying to do this with the same AST (making that one more complex), versus computing an additional AST just for this problem?
- Gehinnn 5y ago
  > it seems that a separate AST is created just for this bracket-pair problem, is that right? That is right. > I imagine there was/is already (at least one) another AST already being computed in the editor, for other reasons Not in the renderer process. The point is that more concrete ASTs that model more features of the language are less likely to have reusable nodes. When you have just bracket pairs, you can reuse any bracket pair that was not modified! However, when prepending a JS file with `[`, a class declaration probably does not parse as class declaration anymore, so you cannot just reuse it and make it child of the array expression (what you could do if it was just another bracket pair).
derefr 5y ago
I’ve always wanted something like this extension, but where the brackets aren’t colorized, but rather change size depending on their nesting level, with the outermost brackets becoming increasingly large as the total nesting level of the expression increases — exactly the way brackets do in maths. Ideally, unlike in TeX, additions to nesting levels wouldn’t require a re-flow (i.e. typing a bracket wouldn’t make the code constantly shimmy around on the screen), but rather these would just be changes in the perceived size of these characters, while keeping them where they are on a monospace grid (since a lot of code assumes a pure monospace layout for indentation et al.) In my imagination, this would work by just having a set of bracket/paren/brace/etc. graphemes in each font, that have increasingly-long ascenders/descenders, while the base size of of the “functional” part of the grapheme remains constant. As if there were five versions of the letter g, with an increasing length of the stem on which the lower hook rests. As such, these magnified brackets would be rendered similarly to the way emoji are rendered in terminals: the grapheme would eventually “leak out” of its monospace box (in the emoji case to the sides; in this case above and below), to the point of eventually overlapping surrounding graphemes, rather than re-spacing the text to give it room.
- rory 5y ago
  The challenge here would be that nesting levels can be added inside or outside the previous level, so to prevent reflow, bracket size would be dependent on order of addition and could vary wildly between different "nests".
  - snovv_crash 5y ago
    You could just adjust the height, not width of the bracket. There is usually plenty of space between lines to support this.
- cscharenberg 5y ago
  I have a strong memory of using an IDE that behaved like this - using size and font to communicate structure - at a job around 2005. I can't currently remember the name but think it it started with 'source'? If someone else remembers this and the name I'd appreciate it. I could be remembering wrong too about how far it went with such decoration.
chubot 5y ago
Hm so it's reusing the syntax highlighter's knowledge of comments and string literals? So then is the recursive descent parser mentioned language-independent or language-dependent? This is cool although I feel like the caveat of the ambiguity of < and > could isn't the best experience, especially for TypeScript and C++ I imagine.
- Gehinnn 5y ago
  > So then is the recursive descent parser mentioned language-independent or language-dependent? The parser is language-independent, but the tokenizer used by the parser looks up the tokens of syntax highlighting to decide if a bracket character is an opening/closing bracket or just text.
pbiggar 5y ago
I think people don't spend enough time thinking how many of the problems they have are caused by the fact that programming languages are just text that need to be parsed by every single tool they use. If languages were (eg) AST-based, this would be a fairly trivial problem.
- Gehinnn 5y ago
  (Personal opinion) Constructing or incrementally updating an AST is not cheap. In particular, most ASTs are very fragile and there are characters when inserted at the wrong position invalidate the entire AST (e.g. prepending a C-like document with `/*`). Even prepending documents with `{` could render all nodes of the AST invalid, as they might get a different meaning (a class declaration might now be parsed as block statement). The AST for the language of well-formed bracket pairs is very robust. There is no character that invalidates the entire AST. There are characters though that invalidate all tokens, but there is a separate asynchronous (slow) method that updates them in the background and only then incrementally the bracket pair AST.
  - pbiggar 5y ago
    > Constructing or incrementally updating an AST is not cheap. In particular, most ASTs are very fragile and there are characters when inserted at the wrong position invalidate the entire AST (e.g. prepending a C-like document with `/*`). Even prepending documents with `{` could render all nodes of the AST invalid, as they might get a different meaning (a class declaration might now be parsed as block statement). I think you have a worldview where text is being edited and ASTs are constantly being replaced via parsing. In an AST-based language, it doesn't necessarily have meaning to add some random character at some random spot, in the way that it does with text. With text, you can only recreate the entire AST by parsing, and discover that the parse is broken. With an AST-based language (such as https://darklang.com https://darklang.com), the editor could inform you that that character cannot be added in that place. Or it could allow you to put it there, informing you that that particular piece of text has no valid meaning, while the rest of the AST is still valid. (Darklang uses both approaches).
  - sweetsocks21 5y ago
lennoff 5y ago
Slightly related question: is there a plug-in that can color my code based in scope instead of syntax?
- srcreigh 5y ago
  There is one for emacs. Could be good inspo if someone wanted to make a VSCode version. https://github.com/alphapapa/prism.el https://github.com/alphapapa/prism.el
- kdmytro 5y ago
  Not exactly what you asked for, bun indent-rainbow colors the indentation space per nestedness level. Also, I think the old bracket colorimg extension has an option to underline the entire section of code between the brackets pair you're currently in.
jamesfisher 5y ago
Nice, but I really want matching _variable_ colorization. Here is a beautiful demo [1]. I've yet to see it implemented in any serious editor. [1]: https://evanbrooks.info/syntax-highlight/v2/ https://evanbrooks.info/syntax-highlight/v2/
- fooey 5y ago
  Do you mean what VSCode calls semantic highlighting? https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide https://code.visualstudio.com/api/language-extensions/semant...
  - kevingadd 5y ago
    No, the GP is talking about a unique color being assigned to each name (local, parameter etc) in a scope so that typos become visible at a glance. I used a Sublime Text extension for this once but haven't seen it built into an editor either.
    - dr_kiszonka 5y ago
      I wonder if it would be possible to combine the two, e.g., have parameters always displayed in italics with each parameter having its distinct color.
  - deleted 5y ago
    [deleted]
- matsemann 5y ago
  Jetbrains editors have that, called semantic highlighting. https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-your-variables-at-a-glance-with-semantic-highlighting/ https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-you... per variable basis, so apparently different than the vs code thingy with the same name. Also lots of extensions for similar stuff, often named "rainbow" something.
thallavajhula 5y ago
This was such a joy to read. I rarely read every line of a long article. I just sift through the longer articles and this wasn't the case. It is written in a very interesting way, describing the -- past, known issues, and the solution. Kudos to the team behind this implementation, the original author, and the people who put forth this blog post!
choeger 5y ago
> the checker.ts file of the TypeScript project, which has more than 42k lines of code What the actual f** ? 42k lines of code and they mention it casually without even thinking about explaining the reason behind this monstrosity? And yes, I fully expect the comments below to contain stuff like "This is nothing, back in the day we had a 1M loc Java class."
- pnt12 5y ago
  They just picked a big real file and used it for benchmarking. Apparently it's not even from the same project, why would they comment on it?
- npigrounet 5y ago
  This is nothing, back in the day we had a 1M loc Java class.
- sellyme 5y ago
  Why on earth is it their job to explain the reason behind a large file in a completely different piece of software?
- int_19h 5y ago
  The file is linked from the article, so you can look at it and judge for yourself. https://raw.githubusercontent.com/microsoft/TypeScript/8362a0f929d74ff46828016ec67c05744a8dbb3c/src/compiler/checker.ts https://raw.githubusercontent.com/microsoft/TypeScript/8362a...
xg15 5y ago
I'm just amazed how much technical genius is needed just to work around their architecture - to archive almost the same responsiveness that this feature had in emacs 20 years ago.
- kevingadd 5y ago
  How is it implemented in emacs?
  - gpderetta 5y ago
    If parent is referring to rainbow-identifiers, it seems that it uses so called syntax tables, which are used for syntactic parsing of documents, they are created by the buffer major mode and are shared between modules. They are used for all kind of things, like indenting, cursor movement, etc. At least that's my understanding, I'm a long time emacs user, but I don't have a very deep knowledge of the internals.
zh3 5y ago
It's amazing that modern software techniques are so slow that speeding up rainbow braces is necessary (let alone worthy of an article). Get off my lawn (while I water it with my tears).
- zh3 5y ago
  Going to incur even more downvotes from the MS crowd here, but seriously this is insane (and looks like the PR team are on my case).
fuadnafiz98 5y ago
This document is a prove of why you need to learn algorithms, algorithm complexity, compiler, parsers :)
- tantalor 5y ago
  If you work on internals of an compiler/editor, yeah. For everybody else, no.
  - ulrikrasmussen 5y ago
    Basic knowledge making you able to do rudimentary big O analysis is always useful. Unintended polynomial running time where linear is possible is a very common performance regression.
    - neonbones 5y ago
      This year I started my leetcode grinding for interview purposes. But when I look back, I can tell that's all this practice and analysis of solutions made me a much better engineer than before. The most impactful thing in my 6+ year career was my understanding of how to think effectively about my code. Before that, I got some basic knowledge about CS theory, master theorem, asymptotic, all these things, and was a typical skeptical developer who was thinking like, "yeah, that's cool, but you don't actually need it. At all, basic knowledge is enough." Today, I will recommend that anyone go through basic algo and ds courses (Sedgwick book or course, Skiena book too) and try to practice them. Now I'm even trying to participate in leetcode and codeforces contests. In the end, after the first months of frustration about how stupid I was and how I couldn't find a solution to easy problems, it's started to be fun, and I loved it.
baby 5y ago
tl;dr: look for "editor.bracketPairColorization.enabled" in your user settings.
co_dh 5y ago
In q/kdb, bracket for unary function application is optional, that is, f[x] is the same as f x. I found it very convenient after a while, and wish other language follows. It is very convenient to write f g h x, than f[g[h[x]]]
- int_19h 5y ago
  This is the usual arrangement in functional programming languages - ML, Haskell etc all use juxtaposition to denote function application. If you want something along these lines, but a bit closer to mainstream, with easy access to popular libraries etc, that would probably be F#.
turpialito 5y ago
Not to de-merit the creator. Looks really useful. But I tend to code using Allman-style bracketing. I find Allman bracketing and corresponding indentation makes my code more readable, without the need for much color.
hidden-spyder 5y ago
Towards the end, they say: > Even though JavaScript might not be the best language to write high performance code Why not?
- cztomsik 5y ago
  Performance is not predictable, and basically everything is cache-miss. I love JS but this statement is absolutely true.
hoseja 5y ago
Great illustration of how API-based extension schemes are fundamentally crippled. Also, "Feel free to skip the sections on algorithmic complexities." repeated so many times sounds rather patronizing.
speps 5y ago
Does anyone know of an equivalent for Visual Studio (not VSCode)?
Tomis02 5y ago
Some commenters are upset but can't quite express why. I'll explain. The article's title is "Bracket pair colorization 10,000x faster", which implies that the previous implementation was using a slow algorithmic approach and someone made it 10k faster. Sure, the article explains the situation in detail, but the way it's laid out kinda screams (at least to me and some other people) "look at me, I'm so good that I improved the original implementation 10k times", when in reality the 10k speedup has nothing to do with algorithms but with the extension API. A more modest and fair approach would have been to name the article "Bracket pair colourization is now in core VS Code" or similar, and leave the 10k speedup as a footnote, rather than having the entire article revolve around it. It's just a matter of etiquette - it's not a situation where boasting is justified.

711 points by feross 5y ago

sesuximo 5y ago

Great news!

curiousgal 5y ago
Not for the extension developer though. Good thing it was not monetized.
- sparsely 5y ago
  Well, he no longer has to maintain it, and an even better version of the feature now exists
- Igelau 5y ago
  https://github.com/microsoft/vscode/issues/128465#issuecomment-900311495 https://github.com/microsoft/vscode/issues/128465#issuecomme...
- deleted 5y ago
  [deleted]
- jffry 5y ago
  Igelau linked to this thread: https://github.com/microsoft/vscode/issues/128465#issuecomment-900311495 https://github.com/microsoft/vscode/issues/128465#issuecomme... In it, CoenraadS (the extension developer) writes "I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it." and "If it could that would be great, and my extension could be deprecated completely." There was also discussion with the author about updating the extension to prompt users to switch to the native functionality, and it appears CoenraadS was also involved in the process/design of implementing the feature in VS code. This appears to be a shining example of doing it right, and giving the original extension author credit.

DenisM 5y ago

Background code analysis in Visual Studio grinds my windows VM to a halt after every few keystrokes. Yet they spend time speeding up colorization of bracket pairs, which worked just fine for the last 10 releases or so.

schrute 5y ago
> Yet they spend time speeding up colorization of bracket pairs, which worked just fine for the last 10 releases or so. You must have missed the part where it is now 10,000x faster. Given your complaint about performance this should excite you!
freemint 5y ago
If VS is to slow, you need to use out more for the Background code analysis to amortize.
- DenisM 5y ago
  I don't get it.
SquareWheel 5y ago
Are you mixing up Visual Studio with VS Code? This pertains to the latter.
- DenisM 5y ago
  Seems so. Is there no background code analysis in VS code? That would be a reason to switch.
  - int_19h 5y ago
    In both VS and VSCode, semantic analysis is handled by specific language extensions (some of which might ship in the box, as C# is in VS and TypeScript is in VSCode). So it's impossible to answer this question without knowing which language you're using.

gpderetta 5y ago

Interestingly rainbow-delimiters in emacs seems to be instantaneous even with elisp being an slow interpreted language. For better or worse emacs does run "extensions" synchronously; this allows extremely fine grained control of the buffer, but a slow extension can kill interactive performance.

tempodox 5y ago
Similar with `lisp_rainbow` via the slimv plugin in vim. It uses vim's built-in syntax coloring mechanism and doesn't slow down the editor.
- gowld 5y ago
  That's what VSCode team did too -- moved the computational work from a plugin to core.
gowld 5y ago
Have you tried rainbow-delimiters on a 42 kilo-line file? That's the benchmark in the OP.
- gpderetta 5y ago
  Just tried. Colorization seems still almost instantaneous. Emacs does generally not like such a huge file though (reindenting the whole buffer is slow for example).
dspillett 5y ago
> elisp being an slow interpreted language The issue isn't about the speed of the language, but the code not having access to information it needs from the extension API so having to go long ways around to work it out. The new version may be in a faster base language, but the majority of its speed boost comes from having access to information that the editor already knows without having to mess around re-deriving it.

unwind 5y ago

That sounds awesome for VS Code users in general, but I guess it's sad news for the independent developer (CoenraadS) who made the pair colorization extension popular and famous. With the editor's native performance 10,000 times better, I suepect the number of installs of their extension will plummet. It's hard to compete with 1st party powers, as all the hoopla around app stores (and before that, the famous "Embrace, extend & extinguish" [1] strategy) show all the time. That said, I'm not saying it's a hostile move, it's just ... interesting. [1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

1_player 5y ago
What? If the independent developer didn't want his technology to be used, he would have patented it or made it closed source. This is not what EEE is about.
Gehinnn 5y ago
Author of the blog post here. We openly discussed various approaches with CoenraadS and other extension authors here: https://github.com/microsoft/vscode/issues/128465#issuecomment-879089188 https://github.com/microsoft/vscode/issues/128465#issuecomme...
- rattray 5y ago
  Thanks for your awesome work! Can't wait to try it.
- gnrlst 5y ago
  Quoting CoenraadS: > Author of Bracket Pair Colorizer here. > I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it. > If there are some quick wins, I can still apply them, but I think my extension is so hacky it is easier to do 1.b or 1.c Seems CoenraadS is completely in favor of the native implementation (and probably could have been called out in the blog post as few people will find this context).
cstuder 5y ago

therealmarv 5y ago

Deactivated the "Bracket Pair Colorizer" extension from CoenraadS which I've first installed years ago. Activated the internal Bracket Pair Colorizer. First world problems: Bracket colors are different now and especially XML looks off... seems I need to fine tune that some day (but not today).

rattray 5y ago
Or open an issue on GitHub with a few screenshots? Vs code devs are iterating!
pachico 5y ago
As a colour blind, I distinguish colours are different but I don't care/wouldn't know what they are: perfect!
spartanatreyu 5y ago
You can change the colors in the settings.

naranha 5y ago

> Brackets are queried when rendering the viewport and thus querying them has to be really fast. Then by querying them only when the document changes could make it another 10,000x faster ;)

antris 5y ago
Great improvement of performance for those who like to code blind!

414owen 5y ago

So, what's the plan? Reimplement every popular extension that doesn't have good performance? > the asynchronous communication between the renderer and the extension-host severely limits how fast bracket pair colorization can be when implemented as an extension. This limit cannot be overcome Nope, I'm not convinced. If it can be done internally, then you should expose more internals until it's possible to do it with the public API. I bet this could be made really performant if vscode had incremental parsing, like tree-sitter+Atom.

LewisVerstappen 5y ago
There were additional issues too, like access to token information discussed here > This is another challenge of the Bracket Pair Colorization extension that affects performance negatively: it does not have access to these tokens and has to recompute them on its own. We thought long about how we could efficiently and reliably expose token information to extensions, but came to the conclusion that we cannot do this without a lot of implementation details leaking into the extension API. Because the extension still has to send over a list of color decorations for each bracket in the document, such an API alone would not even solve the performance problem.
- TeMPOraL 5y ago
  > access to token information This is perhaps the core difference between VS Code and Emacs (which a lot of people believe is being superseded by VS Code as the "extensible editor") - in Emacs, there is no such thing as limiting access to information. Outside of things that get hidden accidentally[0], any bit of elisp code can access everything. It's not just a practical difference, but also a philosophical one: Emacs plugins are not designed to expose a narrow API, because it's impossible to enforce anyway. There's a structure to it, so nothing prevents one from creating good abstractions - those abstractions just have to be designed with extensibility and interoperability in mind[1], because the users (including other package developers) always have an option to just hook into, advise, override or replace any piece of code in your package. Performance-wise, the impact of it varies. On the one hand, this level of flexibility prevents Emacs from making important breaking changes[2]. On the other hand, nothing ever has to wait for a better API design - if there's a way to make a feature faster by hooking to a dependency's internal, people will do just that, and keep doing that until the API blesses the use case. -- [0] - Like state the C core doesn't expose, or some implicit state shared by a bunch of closures - though you can get at the latter if you override whatever is hiding the state. [1] - E.g. by offering hooks as a blessed, well-defined and stable way to interact with internals to cover 90% of interoperability needs. [2] - Like doing proper multithreading, or replacing Emacs Lisp with a more polished Lisp - though myself I feel ELisp is good enough as a language.

EMM_386 5y ago

VS Code is an amazing piece of technology, despite all the "Electron hate" (although changing to WebView2). I recently commented on it here: https://news.ycombinator.com/item?id=28556588 https://news.ycombinator.com/item?id=28556588 Microsoft engineers are top-notch and this is another feature I'll be using daily. Good stuff!

TeMPOraL 5y ago
VS Code is the benchmark of what can be done with Electron - if you really, really care. But it's an extreme outlier. The hate for Electron comes from how the average Electron application works. Not only almost nobody cares as much as VS Code team does, the very choice of using Electron itself is usually an act of not caring.
- flohofwoe 5y ago
  But this is also why simply switching to OS-native APIs and compiled languages wouldn't help much for the average case. A team that doesn't care about performance in Electron also wouldn't care about peformance in native applications, and performance isn't some magic pixie dust that automatically appears when chosing a different UI framework or programming language, it needs to be actively worked towards (some YMMV of course).
  - IshKebab 5y ago
    > performance isn't some magic pixie dust that automatically appears when chosing a different UI framework or programming language That's a little misleading. If you write the same program in idiomatic C++ and Python then it's almost guaranteed that the C++ version will be much much faster even before you have done any profiling or performance optimisation. So there is some magic pixie dust.
    - goldenkey 5y ago
      C++ doesn't give you asymptotic Big(O) algorithmic superpowers. No language does.

rhexs 5y ago

Awesome! When is the leetcode problem coming so I can practice it for my next interview loop?

PragmaticPulp 5y ago

The original author of the bracket colorizer extension gets a great shout-out in this post and was also involved in the discussions of building the feature. Moreover, he has stated that he was tired of maintaining the extension and seems to be fully in favor of this move: > Author of Bracket Pair Colorizer here. > I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it. The comments here suggesting that the Visual Studio Code team was somehow wrong for making this improvement or that they otherwise wronged the extension author are not reflective of the actual process nor even the extension author’s own feelings. The blog post does a great job of describing why bracket colorization isn’t appropriate for the plug-in architecture anyway. Making it fast can only be accomplished within the core of the editor code which has more direct access to syntax parsing. This is a win for everyone all around. The Code team did a great job with this feature and the associated blog post.

duxup 5y ago
We see a lot of situations where things sort of die or fade away when the author(s) just don't have time. This seems like the ideal situation where a highly capable team takes over and the author is happy to see it.
- int_19h 5y ago
  The single most popular extension for VSCode - the one that adds Python support (https://marketplace.visualstudio.com/items?itemName=ms-python.python https://marketplace.visualstudio.com/items?itemName=ms-pytho...) - is somewhat similar: it was originally written by Don Jayamanne, and later acquired by Microsoft. Although in that case, Don was hired by Microsoft as well, and continued working on it: https://devblogs.microsoft.com/python/don-jayamanne-joins-microsoft/ https://devblogs.microsoft.com/python/don-jayamanne-joins-mi... This can still be seen on GitHub - if you look closely, the official repo at https://github.com/microsoft/vscode-python https://github.com/microsoft/vscode-python is a fork! Curiously, some bits of code went kinda full circle in the process - vscode-python reused the (open source) Python debugger written at Microsoft for Python Tools for Visual Studio.

orliesaurus 5y ago

> The feature can be enabled by adding the setting "editor.bracketPairColorization.enabled": true. Interesting this NEEDS to be enabled. ...or will it be set to `true` on new installs? Seems like a super useful feature that should be on by default to me..

mmis1000 5y ago
I think change user's layout suddenly without explaining is definitely a way to speedrun 'how to get all user's hate' in 10 seconds. It should be either 'prompt and let you decide' or 'use new defaults on new install'
- easton 5y ago
  They’ve been prompting the user on new installs to set their preferences and themes, it’ll probably get rolled into that.
karaterobot 5y ago
I think it's a good choice, since developers do not like people messing with their environments without asking. Especially if those people are Microsoft. The strategy of writing a really good, technical article explaining the feature, and getting it marketed on forums where developers can see it and get excited enough to try it seems savvy to me. People who are resistant to change (and likely to complain) will just not be affected. I wouldn't turn it on for existing users, but as you say, having it on for new installs does make sense though!

beermonster 5y ago

For those just wanting to try it out the feature can be enabled by adding the setting "editor.bracketPairColorization.enabled": true.”

emmelaich 5y ago
There's a gui option for this but it's a bit mysterious. It shows enabled, but immediately under has an unchecked checkbox.
- nmeofthestate 5y ago
  Looks like it formats the underlying setting attribute name in the UI by emboldening the final dot-separated word editor.bracketPairColorization.enabled

sorenjan 5y ago

This is a well written and in depth article, it must have taken a lot of time and effort to write. Big thanks to the author and Microsoft for releasing it.

bern4444 5y ago

I wonder why VSCode hasn't yet adopted TreeSitter for syntax parsing. Seems like it solves at least part of the performance issue via incremental parsing

Gehinnn 5y ago
(author of the blog post here, personal opinion) Tree-sitter is much more general (and I guess way more complex) and most likely cannot use some tricks we use for "simple" bracket pair parsing. For example, we can almost always re-use nested bracket pairs when characters are inserted/deleted, because the bracket pair language is so simple and it does not matter where a bracket pair is in the AST. But when parsing C# and adding a single opening bracket at the beginning of the file, I doubt namespace/class declarations stay namespace/class declarations. Also, long lists in Tree-sitter seem to cause linearly growing incremental parsing time - that's why we use balanced (2,3) trees. You can try it out on their playground [1] by selecting JavaScript and adding some 100k `{}`s. On my machine, adding a single character takes 50ms. When there are 200k bracket pairs, it takes 100ms. When all these brackets are contained in a single bracket pair however, adding characters after this single root pair is fast again (<1ms). But to be honest, Tree-sitter is still mind boggling fast. [1] https://tree-sitter.github.io/tree-sitter/playground https://tree-sitter.github.io/tree-sitter/playground
- cormacrelf 5y ago
  On the first point, one idea would be to implement the simple bracket pairs language as a TS grammar, stand-alone and independent of any other syntax highlighting. The C# problem of making the syntax invalid and killing the brace highlights disappears. The linear scanning behaviour you describe is due to the change in the left and right parse context for all of those pairs. Yes, it’s linear when you invalidate a subtree, but in the case of a simple bracket pairs language, the damage is limited to scanning through the children of top level brace pairs, and each child subtree is trivial to check. From the IGLR paper: > In a state-matching implementation, each node representing a nonterminal symbol contains a record of the configuration of the pushdown automaton (the ‘parse state’) when the node was shifted onto the stack. A subtree can be reused when both its left and right context are unchanged … Imagine { is prepended to abc(def, {ghi}). I believe the “abc” needs re-parsing, and so does the () subtree as their left parse state is now the “looking for }” state instead of “looking for ({[“ and “looking for )” respectively. But it’s limited to one level deeper — after you split the () subtree, every subtree inside it is still in the “looking for )” configuration on both sides. Specifically, the “def”, “,” and “{ghi}” are fully reused. There are only three possible parse states, so generally you get a lot of subtree reuse. You get even better subtree skipping performance by making long sequences without an brace into a single node, instead of eg tokenising by word and not grouping them. (In this case, “def, “ instead of splitting that.) So realistically for your C# example, only the namespace node is split, and the linear scan is through all the top level brace pairs within the namespace but no deeper. You’ve described IGLR’s best case scenario for incremental parsing an initial brace insertion. Languages that don’t have namespaces would be worse off (but still not too bad). For more realistic languages than {}{}{}{}{}{}{}{}.js I think this approach would work very well. If I liked brace pairs (I don’t) then I would implement this dead simple new grammar and turn it into a Neovim plugin. The existing nvim-ts-rainbow plugin uses queries on existing languages, so is very good at giving perfect/correct pairs and customising per-language, but exhibits the invalid syntax problem and also performance issues which appear to be resolved for most people, but may be back with bigger documents. (Edit — you would need a few variants to account for comments and strings. That makes it a bit more annoying.)

iaml 5y ago

Seems interesting they chose to limit it to a maximum of 6 colors. Is there some technical reason for that? I'm currently using "Bracket Pair Colorizer" with 8 colors and even with that I can remember a couple places where I've seen this limit got exceeded and coloring started from beginning.

svat 5y ago
> Seems interesting they chose to limit it to a maximum of 6 colors. Where do you see this? The article talks about nesting levels being bounded by O(log N) levels where N is the length of the document, and from what I can tell this is just for analysis and it supports arbitrary nesting levels… Oh, is your question about the actual colours used for the brackets, i.e. arbitrary levels of matching are supported, but the colours repeat every 6 levels (e.g. brackets at levels i and i+6 have the same colour)? If so this is a UX choice (I can't see any technical reason) and I imagine the considerations might be something like: - There are only finitely many colours that can be usefully distinguished visually by the average person, so we need to pick some limit L, - The limit L needs to be small enough that all L colours are easily distinguishable and memorable, but large enough that it would never be confusing whether a certain coloured bracket is at nesting level i or i ± L. I guess the thinking may have been that 6 is large enough typically, e.g. it's likely to be clear from context whether a certain bracket (say blue) is at (say) level 2 or level 8.
- Gehinnn 5y ago
  If there is demand, please file an issue and we can discuss increasing this limit. However, these colors are themeable and thus there can only be finitely many. There is no technical reason for this limit of 6.
  - iaml 5y ago
    I see, I'll definitely make one.
- iaml 5y ago
  It's in the release notes [0]. > All colors are themeable and up to six colors can be configured. [0] https://code.visualstudio.com/updates/v1_60#_high-performance-bracket-pair-colorization https://code.visualstudio.com/updates/v1_60#_high-performanc...

svat 5y ago

> The core idea is to use a recursive descent parser to build an abstract syntax tree (AST) that describes the structure of all bracket pairs. This is a really nice and clear blog post on an algorithmic challenge wonderfully addressed to a satisfactory conclusion (the rebalancing and node reuse etc work out so perfectly); must have been fun, thanks for sharing! I don't know / haven't thought much about text editors so a question for the authors or anyone who understands: from the mention of "the bracket pair AST", it seems that a separate AST is created just for this bracket-pair problem, is that right? I imagine there was/is already (at least one) another AST already being computed in the editor, for other reasons? If so, how did you decide between trying to do this with the same AST (making that one more complex), versus computing an additional AST just for this problem?

Gehinnn 5y ago
> it seems that a separate AST is created just for this bracket-pair problem, is that right? That is right. > I imagine there was/is already (at least one) another AST already being computed in the editor, for other reasons Not in the renderer process. The point is that more concrete ASTs that model more features of the language are less likely to have reusable nodes. When you have just bracket pairs, you can reuse any bracket pair that was not modified! However, when prepending a JS file with `[`, a class declaration probably does not parse as class declaration anymore, so you cannot just reuse it and make it child of the array expression (what you could do if it was just another bracket pair).

derefr 5y ago

I’ve always wanted something like this extension, but where the brackets aren’t colorized, but rather change size depending on their nesting level, with the outermost brackets becoming increasingly large as the total nesting level of the expression increases — exactly the way brackets do in maths. Ideally, unlike in TeX, additions to nesting levels wouldn’t require a re-flow (i.e. typing a bracket wouldn’t make the code constantly shimmy around on the screen), but rather these would just be changes in the perceived size of these characters, while keeping them where they are on a monospace grid (since a lot of code assumes a pure monospace layout for indentation et al.) In my imagination, this would work by just having a set of bracket/paren/brace/etc. graphemes in each font, that have increasingly-long ascenders/descenders, while the base size of of the “functional” part of the grapheme remains constant. As if there were five versions of the letter g, with an increasing length of the stem on which the lower hook rests. As such, these magnified brackets would be rendered similarly to the way emoji are rendered in terminals: the grapheme would eventually “leak out” of its monospace box (in the emoji case to the sides; in this case above and below), to the point of eventually overlapping surrounding graphemes, rather than re-spacing the text to give it room.

rory 5y ago
The challenge here would be that nesting levels can be added inside or outside the previous level, so to prevent reflow, bracket size would be dependent on order of addition and could vary wildly between different "nests".
- snovv_crash 5y ago
  You could just adjust the height, not width of the bracket. There is usually plenty of space between lines to support this.
cscharenberg 5y ago
I have a strong memory of using an IDE that behaved like this - using size and font to communicate structure - at a job around 2005. I can't currently remember the name but think it it started with 'source'? If someone else remembers this and the name I'd appreciate it. I could be remembering wrong too about how far it went with such decoration.

chubot 5y ago

Hm so it's reusing the syntax highlighter's knowledge of comments and string literals? So then is the recursive descent parser mentioned language-independent or language-dependent? This is cool although I feel like the caveat of the ambiguity of < and > could isn't the best experience, especially for TypeScript and C++ I imagine.

Gehinnn 5y ago
> So then is the recursive descent parser mentioned language-independent or language-dependent? The parser is language-independent, but the tokenizer used by the parser looks up the tokens of syntax highlighting to decide if a bracket character is an opening/closing bracket or just text.

pbiggar 5y ago

I think people don't spend enough time thinking how many of the problems they have are caused by the fact that programming languages are just text that need to be parsed by every single tool they use. If languages were (eg) AST-based, this would be a fairly trivial problem.

Gehinnn 5y ago
(Personal opinion) Constructing or incrementally updating an AST is not cheap. In particular, most ASTs are very fragile and there are characters when inserted at the wrong position invalidate the entire AST (e.g. prepending a C-like document with `/*`). Even prepending documents with `{` could render all nodes of the AST invalid, as they might get a different meaning (a class declaration might now be parsed as block statement). The AST for the language of well-formed bracket pairs is very robust. There is no character that invalidates the entire AST. There are characters though that invalidate all tokens, but there is a separate asynchronous (slow) method that updates them in the background and only then incrementally the bracket pair AST.
- pbiggar 5y ago
  > Constructing or incrementally updating an AST is not cheap. In particular, most ASTs are very fragile and there are characters when inserted at the wrong position invalidate the entire AST (e.g. prepending a C-like document with `/*`). Even prepending documents with `{` could render all nodes of the AST invalid, as they might get a different meaning (a class declaration might now be parsed as block statement). I think you have a worldview where text is being edited and ASTs are constantly being replaced via parsing. In an AST-based language, it doesn't necessarily have meaning to add some random character at some random spot, in the way that it does with text. With text, you can only recreate the entire AST by parsing, and discover that the parse is broken. With an AST-based language (such as https://darklang.com https://darklang.com), the editor could inform you that that character cannot be added in that place. Or it could allow you to put it there, informing you that that particular piece of text has no valid meaning, while the rest of the AST is still valid. (Darklang uses both approaches).
- sweetsocks21 5y ago

lennoff 5y ago

Slightly related question: is there a plug-in that can color my code based in scope instead of syntax?

srcreigh 5y ago
There is one for emacs. Could be good inspo if someone wanted to make a VSCode version. https://github.com/alphapapa/prism.el https://github.com/alphapapa/prism.el
kdmytro 5y ago
Not exactly what you asked for, bun indent-rainbow colors the indentation space per nestedness level. Also, I think the old bracket colorimg extension has an option to underline the entire section of code between the brackets pair you're currently in.

jamesfisher 5y ago

Nice, but I really want matching _variable_ colorization. Here is a beautiful demo [1]. I've yet to see it implemented in any serious editor. [1]: https://evanbrooks.info/syntax-highlight/v2/ https://evanbrooks.info/syntax-highlight/v2/

fooey 5y ago
Do you mean what VSCode calls semantic highlighting? https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide https://code.visualstudio.com/api/language-extensions/semant...
- kevingadd 5y ago
  No, the GP is talking about a unique color being assigned to each name (local, parameter etc) in a scope so that typos become visible at a glance. I used a Sublime Text extension for this once but haven't seen it built into an editor either.
  - dr_kiszonka 5y ago
    I wonder if it would be possible to combine the two, e.g., have parameters always displayed in italics with each parameter having its distinct color.
- deleted 5y ago
  [deleted]
matsemann 5y ago
Jetbrains editors have that, called semantic highlighting. https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-your-variables-at-a-glance-with-semantic-highlighting/ https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-you... per variable basis, so apparently different than the vs code thingy with the same name. Also lots of extensions for similar stuff, often named "rainbow" something.

thallavajhula 5y ago

This was such a joy to read. I rarely read every line of a long article. I just sift through the longer articles and this wasn't the case. It is written in a very interesting way, describing the -- past, known issues, and the solution. Kudos to the team behind this implementation, the original author, and the people who put forth this blog post!

choeger 5y ago

> the checker.ts file of the TypeScript project, which has more than 42k lines of code What the actual f** ? 42k lines of code and they mention it casually without even thinking about explaining the reason behind this monstrosity? And yes, I fully expect the comments below to contain stuff like "This is nothing, back in the day we had a 1M loc Java class."

pnt12 5y ago
They just picked a big real file and used it for benchmarking. Apparently it's not even from the same project, why would they comment on it?
npigrounet 5y ago
This is nothing, back in the day we had a 1M loc Java class.
sellyme 5y ago
Why on earth is it their job to explain the reason behind a large file in a completely different piece of software?
int_19h 5y ago
The file is linked from the article, so you can look at it and judge for yourself. https://raw.githubusercontent.com/microsoft/TypeScript/8362a0f929d74ff46828016ec67c05744a8dbb3c/src/compiler/checker.ts https://raw.githubusercontent.com/microsoft/TypeScript/8362a...

xg15 5y ago

I'm just amazed how much technical genius is needed just to work around their architecture - to archive almost the same responsiveness that this feature had in emacs 20 years ago.

kevingadd 5y ago
How is it implemented in emacs?
- gpderetta 5y ago
  If parent is referring to rainbow-identifiers, it seems that it uses so called syntax tables, which are used for syntactic parsing of documents, they are created by the buffer major mode and are shared between modules. They are used for all kind of things, like indenting, cursor movement, etc. At least that's my understanding, I'm a long time emacs user, but I don't have a very deep knowledge of the internals.

zh3 5y ago

It's amazing that modern software techniques are so slow that speeding up rainbow braces is necessary (let alone worthy of an article). Get off my lawn (while I water it with my tears).

zh3 5y ago
Going to incur even more downvotes from the MS crowd here, but seriously this is insane (and looks like the PR team are on my case).

fuadnafiz98 5y ago

This document is a prove of why you need to learn algorithms, algorithm complexity, compiler, parsers :)

tantalor 5y ago
If you work on internals of an compiler/editor, yeah. For everybody else, no.
- ulrikrasmussen 5y ago
  Basic knowledge making you able to do rudimentary big O analysis is always useful. Unintended polynomial running time where linear is possible is a very common performance regression.
  - neonbones 5y ago
    This year I started my leetcode grinding for interview purposes. But when I look back, I can tell that's all this practice and analysis of solutions made me a much better engineer than before. The most impactful thing in my 6+ year career was my understanding of how to think effectively about my code. Before that, I got some basic knowledge about CS theory, master theorem, asymptotic, all these things, and was a typical skeptical developer who was thinking like, "yeah, that's cool, but you don't actually need it. At all, basic knowledge is enough." Today, I will recommend that anyone go through basic algo and ds courses (Sedgwick book or course, Skiena book too) and try to practice them. Now I'm even trying to participate in leetcode and codeforces contests. In the end, after the first months of frustration about how stupid I was and how I couldn't find a solution to easy problems, it's started to be fun, and I loved it.

baby 5y ago

tl;dr: look for "editor.bracketPairColorization.enabled" in your user settings.

co_dh 5y ago

In q/kdb, bracket for unary function application is optional, that is, f[x] is the same as f x. I found it very convenient after a while, and wish other language follows. It is very convenient to write f g h x, than f[g[h[x]]]

int_19h 5y ago
This is the usual arrangement in functional programming languages - ML, Haskell etc all use juxtaposition to denote function application. If you want something along these lines, but a bit closer to mainstream, with easy access to popular libraries etc, that would probably be F#.

turpialito 5y ago

Not to de-merit the creator. Looks really useful. But I tend to code using Allman-style bracketing. I find Allman bracketing and corresponding indentation makes my code more readable, without the need for much color.

hidden-spyder 5y ago

Towards the end, they say: > Even though JavaScript might not be the best language to write high performance code Why not?

cztomsik 5y ago
Performance is not predictable, and basically everything is cache-miss. I love JS but this statement is absolutely true.

hoseja 5y ago

Great illustration of how API-based extension schemes are fundamentally crippled. Also, "Feel free to skip the sections on algorithmic complexities." repeated so many times sounds rather patronizing.

speps 5y ago

Does anyone know of an equivalent for Visual Studio (not VSCode)?

Tomis02 5y ago

Some commenters are upset but can't quite express why. I'll explain. The article's title is "Bracket pair colorization 10,000x faster", which implies that the previous implementation was using a slow algorithmic approach and someone made it 10k faster. Sure, the article explains the situation in detail, but the way it's laid out kinda screams (at least to me and some other people) "look at me, I'm so good that I improved the original implementation 10k times", when in reality the 10k speedup has nothing to do with algorithms but with the extension API. A more modest and fair approach would have been to name the article "Bracket pair colourization is now in core VS Code" or similar, and leave the 10k speedup as a footnote, rather than having the entire article revolve around it. It's just a matter of etiquette - it's not a situation where boasting is justified.