10 ms·

Syntax highlighting is a waste of an information channel

196 points by adenadel 6y ago

Symbiote 6y ago
I don't think this is suggested: highlight all uses of a field in a class, or a variable in a method. IntelliJ does this for the field/variable under the cursor, with a background colour and a margin colour. I'd like to see what it would be like with all variables in their individual colours.
- recursivecaveat 6y ago
  There are plugins for various editors using that kind of highlighting scheme if you'd like to try it. Usually it is called "semantic highlighting".
perl4ever 6y ago
I find the unhighlighted circle to be about as quick to identify as the red one. Also, I work for someone who is color blind. This whole attitude of "we must do something really important with color because it's so powerful" goes against the idea that it has to be optional.
- deleted 6y ago
  [deleted]
- jarofgreen 6y ago
  > Also, I work for someone who is color blind. Jethrains also does things with outlines, etc - a bit more limiting maybe but worth considering. But yes, we much make sure we don't end up causing problems here - what about someone using a text-to-speech tool?
- GlennS 6y ago
  Not convinced. It's not exclusionary for people with good colour vision to want to use it effectively. It is exclusionary not to provide good alternatives for people who don't have good colour vision (or various other disabilities).
- Finnucane 6y ago
  I am color blind, and some of the examples here just didn’t work for me. Even normal syntax highlighting is of limited value, and I need to tweak color schemes. Some thing like this could be useful as long as it didn’t get overwhelming (selectable context switching) and the user had some control over the colors used.
duskwuff 6y ago
I don't buy it. 1. I personally find syntax highlighting incredibly helpful as a way of setting off syntactic elements, and incidentally as a way of calling attention to typos (like a missing quote or misspelled keyword). The author seems to dismiss this without any analysis. 2. Using color for syntax highlighting doesn't mean you can't also use color to highlight other useful things (like search results or errors), as the author suggests. Many editors which support syntax highlighting do exactly this! 3. The author's examples aren't particularly compelling. For example, using colors to denote levels of nesting will make code flash between colors as you edit it. Highlighting variables by type feels like a form of syntax highlighting, not something new, and most of the examples of highlighting code which meets specific conditions (variables which are assigned to multiple times, long functions, functions without documentation, etc) all feel like they could be summarized as "linter warnings/failures".
- em-bee 6y ago
  using colors to denote levels of nesting will make code flash between colors as you edit it. why would that be a problem? the nesting wouldn't change with every keystroke, but only as you type the specific characters that define the nesting. and depending on your editor or your typing habit you may type opening and closing characters first before filling in the content, thus you wouldn't break the nesting structure either. the pharo smalltalk code editor highlights nesting brackets and parentheses in different colors, and i don't see any flashing of colors.
- mercer 6y ago
  I think it's important to separate the various suggestions the author offers from the actual implementation. Currently, when my cursor is over a bracket, the matching one is already highlighted. I'd love it when in this situation his suggestion to color the brackets would 'activate'. Or when my cursor (or mouse) is on a variable, I'd like to see the highlighting by type. I can also imagine having a 'highlighting' bar somewhere on screen where I can switch between different forms of highlighting. When I'm doing a code review, highlight based on interesting conditions (long functions, lack of documentations, etc.).
  - 6y ago
waf 6y ago
Visual Studio does semantic highlighting for at least C# (they call it enhanced highlighting). Static classes, parameters, internal structure of regex strings, etc. Basically anything returned by the Roslyn Classification API. I personally like it, but I think other people feel that syntax highlighting plus semantic highlighting leads to rainbow soup.
- maxmouchet 6y ago
  Semshi [1] does it for Python and I find it very useful for identifying errors such as missing imports, misspelled variables, etc. [1] https://github.com/numirias/semshi https://github.com/numirias/semshi
  - deadbunny 6y ago
    Oooh, this is very nice. Thanks for sharing!
adrianmonk 6y ago
Too much color ends up overwhelming. I can find the red circle super quick because it's the only red thing in a sea of black and white. But I like the idea of going beyond syntax. Maybe a way to get the best of both worlds would be to have keyboard shortcuts that toggle cycle through different views. Press this shortcut to color-code types, press this other one to color-code error handling, etc. Also, maybe other visual depictions might be useful. Maybe parentheses could start off very tall and get slightly shorter as you go a layer inward. Or add a 3D/depth shading effect so that code each level of parens literally looks like a level. Or whatever else you can think of, the point being that it doesn't have to be color.
- avianes 6y ago
  > Too much color ends up overwhelming. I can find the red circle super quick because it's the only red thing in a sea of black and white. > But I like the idea of going beyond syntax. Maybe a way to get the best of both worlds would be to have keyboard shortcuts that toggle cycle through different views. I couldn't agree more. When a line has too many different colors I have a really hard time to intuitively organizing the information. It's kind of a visual cacophony to me. So I use a minimalist syntax highlight to avoid that. I believe we could come up with a more clever and interactive highlighting system. A simple example could be rainbow parenthesis which is only enabled when many parentheses are found in a chunk of code. (if it already exists, I'd be glad to know about it)
megameter 6y ago
This kind of nibbles at the edges of the longstanding question: Whether source code is best represented as a linear document. Because historically, that was true for the obvious reasons of pragmatism; you could put it on a punchcard, print it to paper, save it to a file all using the same character encoding and rendering methods, and when we went to interactive editing that didn't change: half of the joy of plaintext is that you can select, copy, paste, search and modify in mostly standard ways across many tools. Syntax highlighting is incrementally in line with this paradigm since there's only one syntax for the language(syntax extension and mixed-language documents aside) and it too follows a linear order, although it also recognizes that an AST exists. But as you get into stuff like code folding, Intellisense and other IDE-like features, deviations emerge. More of the selections deal with symbolic meanings(declarations, types) and not really the tokens or the characters they consist of, and only incidentally the delimiters of the AST. But - when we try to fully re-envision it and express these other concepts directly we tend to end up with something that is unfamiliar, a bit clunky and alienating(most graphical languages). So the accepted developments always seem to come in these little ideas that are mostly-optional add-ons. Chuck Moore did use semantic color as a language feature in ColorForth, but his specific goals lean towards a holistic bootstrapping minimalism, so I haven't seen it taken up elsewhere.
- XorNot 6y ago
  You can extend this further: git is the wrong representation model for code (text). It's a useful representation model, but really only because programming languages are mostly designed to be line-oriented (though not enough: i.e. Python should disallow multiple imports on one line because it lightly breaks diff-viewing). Git is the dumbest possible thing that can work, and it does work, but it's also wrong. What we really need is a generic way to store graph-structured data (which is what an AST really is) into something like git. Because then we could finally get rid of the notion of code-formatting as anything but a user experience peculiarity (the rise of "push button and blackbox fixes code for commit" elucidates why this should be). But more importantly, it would mean we could reasonably start representing commits with some notion of what they are doing to the control flow and structure of the code. Think git's rename detection but it notices when a function is renamed, or can represent refactoring in a diff by showing that all the control linkages of one block of code have moved to a particular function (and by extension, diffs would now implicitly show when new links or dependencies are added). The trouble of course is doing any of this generically, is an unsolved problem. I have an idea that you could probably do something interesting like this with git's existing structure and a language like Golang, where you decompose the program into a bunch of files representing the hierarchical AST and commit that rather then actual text maybe.
Animats 6y ago
Syntax highlighting shows something that's already visible. What could be highlighted that isn't already visible? Undefined variables? Those really should get a squiggly underline, as with spell checkers. Some IDE probably does that already. Anything more subtle? A key point in language design and program tooling is detecting that thing A, which needs to be consistent with thing B far away in some other module, isn't. Most hard to find defects involve some distant relationship. What could be done with color or other automatic annotation to help?
- em-bee 6y ago
  the code editor in pharo smalltalk does that
- alternatetwo 6y ago
  Visual Studio gives use before assign code pieces a green underline in C++.
- greggman3 6y ago
  Undefined variables is already a thing in most IDEs? I know VSCode does it. I can't imagine the others don't. One interesting thing I've found in JavaScript (and I'm guessing it would be useful in Python) is the CodeSpell plugin for VSCode https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker https://marketplace.visualstudio.com/items?itemName=streetsi... It spell checks code by separating camelCaseWords and identifiers_with_underscores. Sure it's not "smart" but for dynamic languages, at least for me, it helps prevent bugs. A static language the IDE would already have the info it needs to highlight a non-existent property but in a dynamic language generally not so I've found it quite useful.
erikbye 6y ago
This is awful, a mess. I prefer little to no syntax highlighting, just a grayscale theme with some bold and italics. Actual code clarity (naming, etc.) and structure (indentation, whitespace, etc.) is what is important. Recently there was an article here about what makes a good photograph, one element was few colors, bringing focus to what matters. That is what code should look like, like a good photograph. Not like a rainbow exploded.
- yesenadam 6y ago
  It seems everyone prefers a different colouring to their syntax highlighting, which you get used to, and other peoples' looks weird! But code should look like what you personally want it to, surely. There's no one way. Sure, you could write a manifesto promoting your opinion to a prescription for everyone, but why? You don't have to look at theirs, they don't have to look at yours. Yes, maybe for code sharing, low- or no-colour is more like a common denominator, and not strange/disorienting for anyone. I find one of the main benefits of syntax highlighting is it's less tiring on the eyes. And once you get used to it what your colours mean, much faster to read. Having it look attractive as well is just a side-benefit, though not a small one, if you spend a lot of your life looking at code!
jarofgreen 6y ago
A really good post, but problem 2 is the biggie here: > What if something needs to be colored two things for two different reasons? As another commenter points out, JetBrains already do a lot of the ideas this person has raised - but they do it in response to actions. Eg. Put the cursor next to one bracket, the matching bracket is highlighted. I think the key to breaking this open is an editor that does different highlights at different times in response to user actions; the key will be getting the right user actions so this is seamless and easy to use.
mdf 6y ago
Good points in this post, although I'd add the word "Only" in front of the headline. IMO, syntax highlighting definitely has its uses, but why limit ourselves to only using colors for that? I'd like to see a future, where depending on whether I'm, for example: - sketching the structure for a completely new program, - implementing a new feature to an existing program, - refactoring an existing program, - debugging a problem in my program, - writing tests, or - trying to find something for reference I'd be able to seamlessly switch between different highlighting schemes. I don't (yet) know what I'd prefer for each of these different modes of software development. The author of TFA lists some ideas for some different ways of doing semantic highlighting. Maybe over the years I'd grow to like some of them for some of these tasks.
reportgunner 6y ago
I don't get it. Author is a programmer, why doesn't he just develop this? I don't need this and I don't want it bloating my text editor.
heavenlyblue 6y ago
Written by an author who had never seen IntelliJ IDEAs highlighting framework. Yes, it does syntax highlighting, but it also manages to mix it up with local variables, etc.
pragmatick 6y ago
I use https://plugins.jetbrains.com/plugin/10080-rainbow-brackets https://plugins.jetbrains.com/plugin/10080-rainbow-brackets for IntelliJ to color matching brackets.
- LandR 6y ago
  I find this plugin invaluable for Clojure!
gnicholas 6y ago
Some people are reacting to the (clickbait-y) title, which is a more extreme statement than what the author ends up advocating. The ultimate argument is that it's ok to syntax highlight sometimes, but sometimes it might be helpful to use color in other ways. Personally, I find syntax highlighting to be very useful, but I also agree that using color in other ways can be more useful at times. As it happens, syntax highlighting was one of the inspirations for the color gradient text pattern I developed to make reading on screen easier. [1] The existence of syntax highlighting helped me to think about color in text as an information channel that can be used to make certain tasks more efficient, so I'm very grateful for that! 1: https://chrome.google.com/webstore/detail/beeline-reader/ifjafammaookpiajfbedmacfldaiamgg?hl=en https://chrome.google.com/webstore/detail/beeline-reader/ifj...
smt88 6y ago
JetBrains IDEs do almost all of these already, mostly without plugins. Other IDEs probably do, too.
matharmin 6y ago
VSCode semantic highlighting is mentioned by the author, and it does support these type of use cases. It works by first doing normal syntax highlighting, then asynchronously getting the semantic info for semantic highlighting, since it's a much slower process. The current language-specific implementations do use it to just augment the syntax highlighting, since that is what is what most people would expect. But out of the box it already supports some use cases the author mentioned, such as different highlighting for imports or constants. The full AST and type info can be used for this (depending on the language server), so pretty-much anything is possible. What I personally find very useful (and had to hack experimental support for) is highlighting of TypeScript async/Promise values and functions. This is something that a language server can identify easier than humans, and can make a big difference in coding.
raxxorrax 6y ago
This is somewhat true, although I think keywords should still be highlighted. I once wrote a plugin that highlighted all labels in different colors. Even with a good palette it was really bad... The scope or block highlights could be useful. Difficult for some languages probably, but I would give it a shot.
heapslip 6y ago
I think too much syntax color is an easy way to shoot yourself in the foot as a developer. When everything is highlighted, and everything is important, nothing is important. We need to choose what is important, I like to highlight language keywords as grey, function names as blue and numeric values / strings as purple / green. Everything else is white, and it works great. tree-sitter[0] should end regex-based syntax highlighting, check this out for nvim: https://github.com/nvim-treesitter/nvim-treesitter https://github.com/nvim-treesitter/nvim-treesitter I really want to see tree-sitter used to power an awesome linter, that works across languages.
- jayjader 6y ago
  I find it interesting that your comment (to me) seems to caution against "too much" syntactic highlighting, while the comparison screenshots in the README for the project you mention seem to indicate an extensive use of colors (compared to "traditional" highlighting, as per the readme). On a personal note I do prefer the 2nd screenshot to the first (ie I prefer tree-sitter's highlighting than whatever they used to generate the "traditional" image). In the end I think it comes down to colors being useful to highlight _meaningful_ differences between code snippets, be it semantic, syntactic, or whichever other distinction is useful to the person reading/editing the code at that time. As mentioned elsewhere in these comments, in a language that distinguishes between synchronous and asynchronous functions (à la async/await in Typescript or Python) it could very well be useful to have a different color for each. Then again, if you're "just" opening up your code files to edit some string "constants" here and there, there's almost no point in having any other highlighting (at that moment/for that action/activity) than "what is string, what is not?". So the holy grail is some kind of highlighting that is aware of your intent (as in, why you are viewing/editing this piece of code), and can adapt to changes in said intent.
  - saurik 6y ago
    Interestingly, I strongly prefer the one on the left, as the things the one on the right is coloring extra don't really matter but calling attention to the %s in the formatted strings on the left is extremely useful.
  - 6y ago
minikomi 6y ago
I've experimented with using various "semantic depth" color schemes when writing clojure. Depending on the code it can be useful. Some I tried: - https://github.com/istib/rainbow-blocks https://github.com/istib/rainbow-blocks - rainbow blocks, the "OG" - https://github.com/seanirby/rainbow-blocks-bg https://github.com/seanirby/rainbow-blocks-bg - rainbow blocks but background style - https://github.com/alphapapa/prism.el https://github.com/alphapapa/prism.el - Prism, a slightly more "friendly" rainbow blocks implementation
gsliepen 6y ago
Syntax highlighting is very useful, if not then it wouldn't be so commonplace. The other highlighting methods are also useful. But there is not one objective best highlighting method, it actually changes from moment to moment depending on what you are doing. And many editors in fact already change how they highlight things based on the context! For example, when I search for something, my editor highlights all matches. If my cursor is on an opening bracket, it will start highlighting the matching closing bracket. Some of the highlighting methods shown in the article also seem a bit useless to me. Assigning colors to indentation levels? Surely the indentation itself is a huge visual clue already that doesn't need colors to be wasted. The rainbow parentheses example also shows that there are limits to how many obiously distinct colors you can actually have on screen at the same time, and the colors don't uniquely show nesting levels.
- rbanffy 6y ago
  > Surely the indentation itself is a huge visual clue Apple had a security bug that stemmed from indented C code that was indented but not within brackets. The second line of an if was not under the condition, but was indented the same as the first line. I myself prefer to use type to convey structure and color to convey semantics. Having keywords boldfaced helps structure to stand out while coloring parameters, locals and globals differently would help meaning to surface more evidently.
  - alblue 6y ago
    For what it’s worth, it was introduced as a failure of an automated merge process and not direct programmer entry. Had braces been used the error would likely not have been caused (or would have compilation errors) but the fact that it appeared indented you the same level is not because a programmer made a mistake. It is perhaps more telling that the code was compiled without an error indicating dead code which was the net effect of this merge, but sadly warning free code isn’t a goal.
    - rbanffy 6y ago
      True, but just by looking at the code it looked right. An autoformat tool would have made the mistake more visible. I wonder if popular linters would pick it up (not only the lack of braces as a rule, but the misleading indentation)
the_other 6y ago
I've got a muddled response to this. On one hand, it sounds really helpful and exciting to have tools assist us with these problems e.g. I've read the "show nesting" idea before and wondered why we don't have it yet. Yet on the other hand, I think the post misses the mark in some ways. Most IDEs and editors working with most typed languages already do more than colouring in types: they typically hint ahead of entering characters what type of thing should go in each place where a typed thing can go. I think this is more useful than colour, and we already have it. Taking this further, if our editor has toggles for the categories listed, and thus if it has ASTs (or equivalent things) to work over for semantics, then it probably could do even more than highlighting. Some editors already help you refactor: renaming identifiers across a project, pulling functions out etc. It's not just colouring, it's active involvement with the code, its' types and behaviours. Further, the author deployed an overly simplistic view of colour and our visual perception. In my case, the "red" circle wasn't much easier to find because it was red, but because it was filled in. If it had the same colour as the rounded squares, and was filled in, I would have found it just as fast. The real trick in the grid examples is that some tool (in this case the author) had identified the significant information and done something about it, before the reader even got to apprehend the situation. Tooling which does this would absolutely be more powerful than tooling which doesn't. We already have tools which do some of these things. The drive shouldn't be "we need more colour?" the drive should be "what features are most helpful to coders?"
mmgutz 6y ago
Egads ... that rainbow parentheses suggestion is horrific to my eyes. Most editors already do a good job of showing matched braces without shouting. The problems he mentions are not tied to syntax. We need better CONTEXT aware highlighting is the gist of the article.
mmmkkaaayy 6y ago
If you haven't before, I'd recommending giving Intellij's semantic highlight a look: https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-your-variables-at-a-glance-with-semantic-highlighting/ https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-you... Every variable name is assigned its own color, so you can track the data dependencies in your function using color. Similar to what the author suggests as "Type Highlighting" Some might consider it distracting/overstimulating, but I find it both useful and it makes the code look a bit more 'artistic'.
- account42 6y ago
  KDevelop had this for a long time and I agree it is useful.
- lstamour 6y ago
  Many IDEs, IntelliJ and Visual Studio Code among them, also have plugins for "Rainbow brackets" (the ability to use different colours for different pairs of open and close brackets). WebStorm takes this a step farther and clearly highlights open and close tags in HTML, etc., if I recall correctly.
JohnBooty 6y ago
I'd like to see a truly flexible code display system that treats things as a matrix. As the author describes, you have a number of possible concepts we might like to visually denote: - Language constructs (ie, traditional syntax highlighting) - Parentheses/bracket pairs - Context highlighting - etc etc etc. The author considers a single "information channel", text color. There are other possible, largely orthogonal, information channels. - Text background color - Font weight (light/bold/regular) - Font face - Font italic / upright - Font underlines (single, double, dashed, bold, squiggle, etc) - Font underline color? - Perhaps font effects (shadow, blur, etc) - The editor gutter (Sublime and others expose this) - Hover state (ie, tooltips) I'd like a text display system that lets me mix and match inputs and outputs, matrix style. Perhaps I'd like to use font weights to denote one concept, and underlines for another. Perhaps I'd like to use standard text-color choice for language constructs ala traditional syntax highlighting, and text background color for "context" or "semantic" highlighting (where each variable gets a unique color within its scope) It would most certainly be able to create some real visual nightmares with this level of control, but applied judiciously, it could be very valuable.
sradman 6y ago
The use cases demonstrated are compelling but there seems to be an underlying assumption that current syntax highlighting techniques are broken. After reading the article I'd argue that there is a case to be made for enhanced but temporary syntax highlighting modes where most highlighting classes are muted except the focus of your attention, such as imported package variables. Consistency is important, both for style and syntax highlighting. It is hard to anticipate what a developer is mentally focused on as they scan code.
Konohamaru 6y ago
The author raised an interesting point with respect to syntax highlighting for types. I find the convention that literals be typed (ints, strings, etc...) to be inappropriate. That is very low-level and these should always be inferred. Where typing should exist is at the level the author suggests, such as * Iterators * List-like structures * Functions that return option types and other more abstract structures.
falcolas 6y ago
What’s amusing to me is how few suggestions around highlighting ignores the vast amounts of whitespace that surrounds every character. Borders (partial and full), underlines, strikethrough, background colors, font faces... so many options, yet are we’re constantly limiting ourselves to the color of a letter and the occasional red squiggle. The one valid excuse I can think of is “my editor doesn’t support it”, to which I say BS. VIM, in a terminal, can do most of those.
dragonwriter 6y ago
> Why aren’t things this way? They...are. With proper extensions, for example, VSCode does many of those things, and can switch which of the supported things it is doing, on top of also doing syntax highlighting, and some of the changes are automatic and context-driven. And while VSCode is my current daily driver, that's been true of code editors for quite some time.
jnxx 6y ago
He somehow missed rainbow identifiers, which is quite useful to represent the data flow in a function: https://www.emacswiki.org/emacs/ColorIdentifiersMode https://www.emacswiki.org/emacs/ColorIdentifiersMode
Already__Taken 6y ago
I don't know how to make it but I always wanted a vscode extension to text background colour what scope you were in for teaching. The rainbow vomit some beautifully indented student python would be...

196 points by adenadel 6y ago

I don't think this is suggested: highlight all uses of a field in a class, or a variable in a method. IntelliJ does this for the field/variable under the cursor, with a background colour and a margin colour. I'd like to see what it would be like with all variables in their individual colours.

recursivecaveat 6y ago
There are plugins for various editors using that kind of highlighting scheme if you'd like to try it. Usually it is called "semantic highlighting".

perl4ever 6y ago

I find the unhighlighted circle to be about as quick to identify as the red one. Also, I work for someone who is color blind. This whole attitude of "we must do something really important with color because it's so powerful" goes against the idea that it has to be optional.

deleted 6y ago
[deleted]
jarofgreen 6y ago
> Also, I work for someone who is color blind. Jethrains also does things with outlines, etc - a bit more limiting maybe but worth considering. But yes, we much make sure we don't end up causing problems here - what about someone using a text-to-speech tool?
GlennS 6y ago
Not convinced. It's not exclusionary for people with good colour vision to want to use it effectively. It is exclusionary not to provide good alternatives for people who don't have good colour vision (or various other disabilities).
Finnucane 6y ago
I am color blind, and some of the examples here just didn’t work for me. Even normal syntax highlighting is of limited value, and I need to tweak color schemes. Some thing like this could be useful as long as it didn’t get overwhelming (selectable context switching) and the user had some control over the colors used.

duskwuff 6y ago

I don't buy it. 1. I personally find syntax highlighting incredibly helpful as a way of setting off syntactic elements, and incidentally as a way of calling attention to typos (like a missing quote or misspelled keyword). The author seems to dismiss this without any analysis. 2. Using color for syntax highlighting doesn't mean you can't also use color to highlight other useful things (like search results or errors), as the author suggests. Many editors which support syntax highlighting do exactly this! 3. The author's examples aren't particularly compelling. For example, using colors to denote levels of nesting will make code flash between colors as you edit it. Highlighting variables by type feels like a form of syntax highlighting, not something new, and most of the examples of highlighting code which meets specific conditions (variables which are assigned to multiple times, long functions, functions without documentation, etc) all feel like they could be summarized as "linter warnings/failures".

em-bee 6y ago
using colors to denote levels of nesting will make code flash between colors as you edit it. why would that be a problem? the nesting wouldn't change with every keystroke, but only as you type the specific characters that define the nesting. and depending on your editor or your typing habit you may type opening and closing characters first before filling in the content, thus you wouldn't break the nesting structure either. the pharo smalltalk code editor highlights nesting brackets and parentheses in different colors, and i don't see any flashing of colors.
mercer 6y ago
I think it's important to separate the various suggestions the author offers from the actual implementation. Currently, when my cursor is over a bracket, the matching one is already highlighted. I'd love it when in this situation his suggestion to color the brackets would 'activate'. Or when my cursor (or mouse) is on a variable, I'd like to see the highlighting by type. I can also imagine having a 'highlighting' bar somewhere on screen where I can switch between different forms of highlighting. When I'm doing a code review, highlight based on interesting conditions (long functions, lack of documentations, etc.).
- 6y ago

waf 6y ago

Visual Studio does semantic highlighting for at least C# (they call it enhanced highlighting). Static classes, parameters, internal structure of regex strings, etc. Basically anything returned by the Roslyn Classification API. I personally like it, but I think other people feel that syntax highlighting plus semantic highlighting leads to rainbow soup.

maxmouchet 6y ago
Semshi [1] does it for Python and I find it very useful for identifying errors such as missing imports, misspelled variables, etc. [1] https://github.com/numirias/semshi https://github.com/numirias/semshi
- deadbunny 6y ago
  Oooh, this is very nice. Thanks for sharing!

adrianmonk 6y ago

Too much color ends up overwhelming. I can find the red circle super quick because it's the only red thing in a sea of black and white. But I like the idea of going beyond syntax. Maybe a way to get the best of both worlds would be to have keyboard shortcuts that toggle cycle through different views. Press this shortcut to color-code types, press this other one to color-code error handling, etc. Also, maybe other visual depictions might be useful. Maybe parentheses could start off very tall and get slightly shorter as you go a layer inward. Or add a 3D/depth shading effect so that code each level of parens literally looks like a level. Or whatever else you can think of, the point being that it doesn't have to be color.

avianes 6y ago
> Too much color ends up overwhelming. I can find the red circle super quick because it's the only red thing in a sea of black and white. > But I like the idea of going beyond syntax. Maybe a way to get the best of both worlds would be to have keyboard shortcuts that toggle cycle through different views. I couldn't agree more. When a line has too many different colors I have a really hard time to intuitively organizing the information. It's kind of a visual cacophony to me. So I use a minimalist syntax highlight to avoid that. I believe we could come up with a more clever and interactive highlighting system. A simple example could be rainbow parenthesis which is only enabled when many parentheses are found in a chunk of code. (if it already exists, I'd be glad to know about it)

megameter 6y ago

This kind of nibbles at the edges of the longstanding question: Whether source code is best represented as a linear document. Because historically, that was true for the obvious reasons of pragmatism; you could put it on a punchcard, print it to paper, save it to a file all using the same character encoding and rendering methods, and when we went to interactive editing that didn't change: half of the joy of plaintext is that you can select, copy, paste, search and modify in mostly standard ways across many tools. Syntax highlighting is incrementally in line with this paradigm since there's only one syntax for the language(syntax extension and mixed-language documents aside) and it too follows a linear order, although it also recognizes that an AST exists. But as you get into stuff like code folding, Intellisense and other IDE-like features, deviations emerge. More of the selections deal with symbolic meanings(declarations, types) and not really the tokens or the characters they consist of, and only incidentally the delimiters of the AST. But - when we try to fully re-envision it and express these other concepts directly we tend to end up with something that is unfamiliar, a bit clunky and alienating(most graphical languages). So the accepted developments always seem to come in these little ideas that are mostly-optional add-ons. Chuck Moore did use semantic color as a language feature in ColorForth, but his specific goals lean towards a holistic bootstrapping minimalism, so I haven't seen it taken up elsewhere.

XorNot 6y ago
You can extend this further: git is the wrong representation model for code (text). It's a useful representation model, but really only because programming languages are mostly designed to be line-oriented (though not enough: i.e. Python should disallow multiple imports on one line because it lightly breaks diff-viewing). Git is the dumbest possible thing that can work, and it does work, but it's also wrong. What we really need is a generic way to store graph-structured data (which is what an AST really is) into something like git. Because then we could finally get rid of the notion of code-formatting as anything but a user experience peculiarity (the rise of "push button and blackbox fixes code for commit" elucidates why this should be). But more importantly, it would mean we could reasonably start representing commits with some notion of what they are doing to the control flow and structure of the code. Think git's rename detection but it notices when a function is renamed, or can represent refactoring in a diff by showing that all the control linkages of one block of code have moved to a particular function (and by extension, diffs would now implicitly show when new links or dependencies are added). The trouble of course is doing any of this generically, is an unsolved problem. I have an idea that you could probably do something interesting like this with git's existing structure and a language like Golang, where you decompose the program into a bunch of files representing the hierarchical AST and commit that rather then actual text maybe.

Animats 6y ago

Syntax highlighting shows something that's already visible. What could be highlighted that isn't already visible? Undefined variables? Those really should get a squiggly underline, as with spell checkers. Some IDE probably does that already. Anything more subtle? A key point in language design and program tooling is detecting that thing A, which needs to be consistent with thing B far away in some other module, isn't. Most hard to find defects involve some distant relationship. What could be done with color or other automatic annotation to help?

em-bee 6y ago
the code editor in pharo smalltalk does that
alternatetwo 6y ago
Visual Studio gives use before assign code pieces a green underline in C++.
greggman3 6y ago
Undefined variables is already a thing in most IDEs? I know VSCode does it. I can't imagine the others don't. One interesting thing I've found in JavaScript (and I'm guessing it would be useful in Python) is the CodeSpell plugin for VSCode https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker https://marketplace.visualstudio.com/items?itemName=streetsi... It spell checks code by separating camelCaseWords and identifiers_with_underscores. Sure it's not "smart" but for dynamic languages, at least for me, it helps prevent bugs. A static language the IDE would already have the info it needs to highlight a non-existent property but in a dynamic language generally not so I've found it quite useful.

erikbye 6y ago

This is awful, a mess. I prefer little to no syntax highlighting, just a grayscale theme with some bold and italics. Actual code clarity (naming, etc.) and structure (indentation, whitespace, etc.) is what is important. Recently there was an article here about what makes a good photograph, one element was few colors, bringing focus to what matters. That is what code should look like, like a good photograph. Not like a rainbow exploded.

yesenadam 6y ago
It seems everyone prefers a different colouring to their syntax highlighting, which you get used to, and other peoples' looks weird! But code should look like what you personally want it to, surely. There's no one way. Sure, you could write a manifesto promoting your opinion to a prescription for everyone, but why? You don't have to look at theirs, they don't have to look at yours. Yes, maybe for code sharing, low- or no-colour is more like a common denominator, and not strange/disorienting for anyone. I find one of the main benefits of syntax highlighting is it's less tiring on the eyes. And once you get used to it what your colours mean, much faster to read. Having it look attractive as well is just a side-benefit, though not a small one, if you spend a lot of your life looking at code!

jarofgreen 6y ago

A really good post, but problem 2 is the biggie here: > What if something needs to be colored two things for two different reasons? As another commenter points out, JetBrains already do a lot of the ideas this person has raised - but they do it in response to actions. Eg. Put the cursor next to one bracket, the matching bracket is highlighted. I think the key to breaking this open is an editor that does different highlights at different times in response to user actions; the key will be getting the right user actions so this is seamless and easy to use.

mdf 6y ago

Good points in this post, although I'd add the word "Only" in front of the headline. IMO, syntax highlighting definitely has its uses, but why limit ourselves to only using colors for that? I'd like to see a future, where depending on whether I'm, for example: - sketching the structure for a completely new program, - implementing a new feature to an existing program, - refactoring an existing program, - debugging a problem in my program, - writing tests, or - trying to find something for reference I'd be able to seamlessly switch between different highlighting schemes. I don't (yet) know what I'd prefer for each of these different modes of software development. The author of TFA lists some ideas for some different ways of doing semantic highlighting. Maybe over the years I'd grow to like some of them for some of these tasks.

reportgunner 6y ago

I don't get it. Author is a programmer, why doesn't he just develop this? I don't need this and I don't want it bloating my text editor.

heavenlyblue 6y ago

Written by an author who had never seen IntelliJ IDEAs highlighting framework. Yes, it does syntax highlighting, but it also manages to mix it up with local variables, etc.

pragmatick 6y ago

I use https://plugins.jetbrains.com/plugin/10080-rainbow-brackets https://plugins.jetbrains.com/plugin/10080-rainbow-brackets for IntelliJ to color matching brackets.

LandR 6y ago
I find this plugin invaluable for Clojure!

gnicholas 6y ago

Some people are reacting to the (clickbait-y) title, which is a more extreme statement than what the author ends up advocating. The ultimate argument is that it's ok to syntax highlight sometimes, but sometimes it might be helpful to use color in other ways. Personally, I find syntax highlighting to be very useful, but I also agree that using color in other ways can be more useful at times. As it happens, syntax highlighting was one of the inspirations for the color gradient text pattern I developed to make reading on screen easier. [1] The existence of syntax highlighting helped me to think about color in text as an information channel that can be used to make certain tasks more efficient, so I'm very grateful for that! 1: https://chrome.google.com/webstore/detail/beeline-reader/ifjafammaookpiajfbedmacfldaiamgg?hl=en https://chrome.google.com/webstore/detail/beeline-reader/ifj...

smt88 6y ago

JetBrains IDEs do almost all of these already, mostly without plugins. Other IDEs probably do, too.

matharmin 6y ago

VSCode semantic highlighting is mentioned by the author, and it does support these type of use cases. It works by first doing normal syntax highlighting, then asynchronously getting the semantic info for semantic highlighting, since it's a much slower process. The current language-specific implementations do use it to just augment the syntax highlighting, since that is what is what most people would expect. But out of the box it already supports some use cases the author mentioned, such as different highlighting for imports or constants. The full AST and type info can be used for this (depending on the language server), so pretty-much anything is possible. What I personally find very useful (and had to hack experimental support for) is highlighting of TypeScript async/Promise values and functions. This is something that a language server can identify easier than humans, and can make a big difference in coding.

raxxorrax 6y ago

This is somewhat true, although I think keywords should still be highlighted. I once wrote a plugin that highlighted all labels in different colors. Even with a good palette it was really bad... The scope or block highlights could be useful. Difficult for some languages probably, but I would give it a shot.

heapslip 6y ago

I think too much syntax color is an easy way to shoot yourself in the foot as a developer. When everything is highlighted, and everything is important, nothing is important. We need to choose what is important, I like to highlight language keywords as grey, function names as blue and numeric values / strings as purple / green. Everything else is white, and it works great. tree-sitter[0] should end regex-based syntax highlighting, check this out for nvim: https://github.com/nvim-treesitter/nvim-treesitter https://github.com/nvim-treesitter/nvim-treesitter I really want to see tree-sitter used to power an awesome linter, that works across languages.

jayjader 6y ago
I find it interesting that your comment (to me) seems to caution against "too much" syntactic highlighting, while the comparison screenshots in the README for the project you mention seem to indicate an extensive use of colors (compared to "traditional" highlighting, as per the readme). On a personal note I do prefer the 2nd screenshot to the first (ie I prefer tree-sitter's highlighting than whatever they used to generate the "traditional" image). In the end I think it comes down to colors being useful to highlight _meaningful_ differences between code snippets, be it semantic, syntactic, or whichever other distinction is useful to the person reading/editing the code at that time. As mentioned elsewhere in these comments, in a language that distinguishes between synchronous and asynchronous functions (à la async/await in Typescript or Python) it could very well be useful to have a different color for each. Then again, if you're "just" opening up your code files to edit some string "constants" here and there, there's almost no point in having any other highlighting (at that moment/for that action/activity) than "what is string, what is not?". So the holy grail is some kind of highlighting that is aware of your intent (as in, why you are viewing/editing this piece of code), and can adapt to changes in said intent.
- saurik 6y ago
  Interestingly, I strongly prefer the one on the left, as the things the one on the right is coloring extra don't really matter but calling attention to the %s in the formatted strings on the left is extremely useful.
- 6y ago

minikomi 6y ago

I've experimented with using various "semantic depth" color schemes when writing clojure. Depending on the code it can be useful. Some I tried: - https://github.com/istib/rainbow-blocks https://github.com/istib/rainbow-blocks - rainbow blocks, the "OG" - https://github.com/seanirby/rainbow-blocks-bg https://github.com/seanirby/rainbow-blocks-bg - rainbow blocks but background style - https://github.com/alphapapa/prism.el https://github.com/alphapapa/prism.el - Prism, a slightly more "friendly" rainbow blocks implementation

gsliepen 6y ago

Syntax highlighting is very useful, if not then it wouldn't be so commonplace. The other highlighting methods are also useful. But there is not one objective best highlighting method, it actually changes from moment to moment depending on what you are doing. And many editors in fact already change how they highlight things based on the context! For example, when I search for something, my editor highlights all matches. If my cursor is on an opening bracket, it will start highlighting the matching closing bracket. Some of the highlighting methods shown in the article also seem a bit useless to me. Assigning colors to indentation levels? Surely the indentation itself is a huge visual clue already that doesn't need colors to be wasted. The rainbow parentheses example also shows that there are limits to how many obiously distinct colors you can actually have on screen at the same time, and the colors don't uniquely show nesting levels.

rbanffy 6y ago
> Surely the indentation itself is a huge visual clue Apple had a security bug that stemmed from indented C code that was indented but not within brackets. The second line of an if was not under the condition, but was indented the same as the first line. I myself prefer to use type to convey structure and color to convey semantics. Having keywords boldfaced helps structure to stand out while coloring parameters, locals and globals differently would help meaning to surface more evidently.
- alblue 6y ago
  For what it’s worth, it was introduced as a failure of an automated merge process and not direct programmer entry. Had braces been used the error would likely not have been caused (or would have compilation errors) but the fact that it appeared indented you the same level is not because a programmer made a mistake. It is perhaps more telling that the code was compiled without an error indicating dead code which was the net effect of this merge, but sadly warning free code isn’t a goal.
  - rbanffy 6y ago
    True, but just by looking at the code it looked right. An autoformat tool would have made the mistake more visible. I wonder if popular linters would pick it up (not only the lack of braces as a rule, but the misleading indentation)

the_other 6y ago

I've got a muddled response to this. On one hand, it sounds really helpful and exciting to have tools assist us with these problems e.g. I've read the "show nesting" idea before and wondered why we don't have it yet. Yet on the other hand, I think the post misses the mark in some ways. Most IDEs and editors working with most typed languages already do more than colouring in types: they typically hint ahead of entering characters what type of thing should go in each place where a typed thing can go. I think this is more useful than colour, and we already have it. Taking this further, if our editor has toggles for the categories listed, and thus if it has ASTs (or equivalent things) to work over for semantics, then it probably could do even more than highlighting. Some editors already help you refactor: renaming identifiers across a project, pulling functions out etc. It's not just colouring, it's active involvement with the code, its' types and behaviours. Further, the author deployed an overly simplistic view of colour and our visual perception. In my case, the "red" circle wasn't much easier to find because it was red, but because it was filled in. If it had the same colour as the rounded squares, and was filled in, I would have found it just as fast. The real trick in the grid examples is that some tool (in this case the author) had identified the significant information and done something about it, before the reader even got to apprehend the situation. Tooling which does this would absolutely be more powerful than tooling which doesn't. We already have tools which do some of these things. The drive shouldn't be "we need more colour?" the drive should be "what features are most helpful to coders?"

mmgutz 6y ago

Egads ... that rainbow parentheses suggestion is horrific to my eyes. Most editors already do a good job of showing matched braces without shouting. The problems he mentions are not tied to syntax. We need better CONTEXT aware highlighting is the gist of the article.

mmmkkaaayy 6y ago

If you haven't before, I'd recommending giving Intellij's semantic highlight a look: https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-your-variables-at-a-glance-with-semantic-highlighting/ https://blog.jetbrains.com/pycharm/2017/01/make-sense-of-you... Every variable name is assigned its own color, so you can track the data dependencies in your function using color. Similar to what the author suggests as "Type Highlighting" Some might consider it distracting/overstimulating, but I find it both useful and it makes the code look a bit more 'artistic'.

account42 6y ago
KDevelop had this for a long time and I agree it is useful.
lstamour 6y ago
Many IDEs, IntelliJ and Visual Studio Code among them, also have plugins for "Rainbow brackets" (the ability to use different colours for different pairs of open and close brackets). WebStorm takes this a step farther and clearly highlights open and close tags in HTML, etc., if I recall correctly.

JohnBooty 6y ago

I'd like to see a truly flexible code display system that treats things as a matrix. As the author describes, you have a number of possible concepts we might like to visually denote: - Language constructs (ie, traditional syntax highlighting) - Parentheses/bracket pairs - Context highlighting - etc etc etc. The author considers a single "information channel", text color. There are other possible, largely orthogonal, information channels. - Text background color - Font weight (light/bold/regular) - Font face - Font italic / upright - Font underlines (single, double, dashed, bold, squiggle, etc) - Font underline color? - Perhaps font effects (shadow, blur, etc) - The editor gutter (Sublime and others expose this) - Hover state (ie, tooltips) I'd like a text display system that lets me mix and match inputs and outputs, matrix style. Perhaps I'd like to use font weights to denote one concept, and underlines for another. Perhaps I'd like to use standard text-color choice for language constructs ala traditional syntax highlighting, and text background color for "context" or "semantic" highlighting (where each variable gets a unique color within its scope) It would most certainly be able to create some real visual nightmares with this level of control, but applied judiciously, it could be very valuable.

sradman 6y ago

The use cases demonstrated are compelling but there seems to be an underlying assumption that current syntax highlighting techniques are broken. After reading the article I'd argue that there is a case to be made for enhanced but temporary syntax highlighting modes where most highlighting classes are muted except the focus of your attention, such as imported package variables. Consistency is important, both for style and syntax highlighting. It is hard to anticipate what a developer is mentally focused on as they scan code.

Konohamaru 6y ago

The author raised an interesting point with respect to syntax highlighting for types. I find the convention that literals be typed (ints, strings, etc...) to be inappropriate. That is very low-level and these should always be inferred. Where typing should exist is at the level the author suggests, such as * Iterators * List-like structures * Functions that return option types and other more abstract structures.

falcolas 6y ago

What’s amusing to me is how few suggestions around highlighting ignores the vast amounts of whitespace that surrounds every character. Borders (partial and full), underlines, strikethrough, background colors, font faces... so many options, yet are we’re constantly limiting ourselves to the color of a letter and the occasional red squiggle. The one valid excuse I can think of is “my editor doesn’t support it”, to which I say BS. VIM, in a terminal, can do most of those.

dragonwriter 6y ago

> Why aren’t things this way? They...are. With proper extensions, for example, VSCode does many of those things, and can switch which of the supported things it is doing, on top of also doing syntax highlighting, and some of the changes are automatic and context-driven. And while VSCode is my current daily driver, that's been true of code editors for quite some time.

jnxx 6y ago

He somehow missed rainbow identifiers, which is quite useful to represent the data flow in a function: https://www.emacswiki.org/emacs/ColorIdentifiersMode https://www.emacswiki.org/emacs/ColorIdentifiersMode

Already__Taken 6y ago

I don't know how to make it but I always wanted a vscode extension to text background colour what scope you were in for teaching. The rainbow vomit some beautifully indented student python would be...