4 ms·

One thing I’ve learned in my years of working on compilers & programming language tech, which at first is counterintuitive, is that for the best user experience

One thing I’ve learned in my years of working on compilers & programming language tech, which at first is counterintuitive, is that for the best user experience in a compiler, often you want your parser to be “total”, and not to reject any inputs. Instead of bailing out at the first error, you parse invalid portions of the input and sync up to the next point where you can recover and keep parsing, annotating the resulting token stream or parse tree with error indicators such as “invalid number literal token” or “missing bracket”. This lets you report all errors in a source file at once, so they can all be highlighted in an IDE for example, regardless of errors earlier in the file. The more semantic information you have about the correct parts of the program, the better error messages you can produce about the incorrect parts. You can extend this to all areas of the compilation pipeline—for example, proceeding to resolve names and typecheck the definitions in a program that are syntactically correct, while ignoring the others and leaving them as error markers.

Zalastax 8y ago
This philosophy is an integral part to the good error reports in TypeScript: https://www.infoq.com/news/2016/05/anders-hejlsberg-compiler https://www.infoq.com/news/2016/05/anders-hejlsberg-compiler
tom_mellior 8y ago
> Instead of bailing out at the first error, you parse invalid portions of the input and sync up to the next point where you can recover and keep parsing People like saying that, but I've never seen it work well or provide any benefits. Because these tools are necessarily imperfect, you will get false positives, i.e., an earlier error will confuse the system sufficiently to generate later errors that aren't really errors in the program. So you have to fix errors in order anyway. So there is no benefit to producing further messages. You just risk producing several pages of spurious diagnostics for a simple typo. Of course this is not completely true, but it's true enough. Or if it isn't, I'd like to hear about examples of systems that never produce such false positives, and where for some reason it makes sense (and is actually beneficial) to change something later in the file first. Until then, just give me the first error. I won't look at the rest anyway, so no need for the compiler to be "clever" about it.
- ufo 8y ago
  This works better in an IDE than if you interact with the compiler directly. If you look at the compiler output directly the false positive drown out the actual errors so it is hard to find out what you need to fix. But in an IDE the false positives just become red squiggly lines further down the file, and don't obfuscate the important red squigglys at the top.
- evincarofautumn 8y ago
  It works best in an IDE, or in my case in Emacs where I can jump to each error in turn. On the command line or in an interactive mode (REPL, notebook, &c.) I usually just want one error at a time. I’ve had fairly good luck with this in C# in Visual Studio or Xamarin Studio with Roslyn, although I don’t actually use .NET much—even though I worked on Mono for a few years, hah. It also really depends on the grammar of the language how well you can do recovery. Ideally a language should be designed from the beginning to support it. Simple example: if a language allows nested function definitions with the same notation as top-level definitions, then you can’t reliably insert a missing closing bracket for a definition, because you can’t assume that the next one is a new top-level; it could be nested. (Although of course you can use heuristics like indentation to get around this.) Anyway, the point of successful recovery is that the diagnostics later in the file are not spurious. And if the compiler supports recoverable parsing, you can still have a mode where it just bails out after the first error if that’s what you prefer.
  - tom_mellior 8y ago
    > Anyway, the point of successful recovery is that the diagnostics later in the file are not spurious. True if you only consider superficial syntax things. False in the real world, where compilers also check whether identifiers are declared, and with what type. If you have an error (syntax or semantic) and decide to press on, in general your symbol tables will be in a messed-up state, and you will generate spurious type or "undeclared identifier" errors. Also, in the real world, the compilers I have experience with do not only continue if they can be 100% sure that it makes sense, but they invariably continue further because they try to be "smart". The IDE point is a semi-valid point since it doesn't scroll to the bottom-most, most useless errors like a terminal does. But I'm still not convinced that it adds value. That it enables situations where it's better to fix the second error before the first.

Zalastax 8y ago

This philosophy is an integral part to the good error reports in TypeScript: https://www.infoq.com/news/2016/05/anders-hejlsberg-compiler https://www.infoq.com/news/2016/05/anders-hejlsberg-compiler

tom_mellior 8y ago

> Instead of bailing out at the first error, you parse invalid portions of the input and sync up to the next point where you can recover and keep parsing People like saying that, but I've never seen it work well or provide any benefits. Because these tools are necessarily imperfect, you will get false positives, i.e., an earlier error will confuse the system sufficiently to generate later errors that aren't really errors in the program. So you have to fix errors in order anyway. So there is no benefit to producing further messages. You just risk producing several pages of spurious diagnostics for a simple typo. Of course this is not completely true, but it's true enough. Or if it isn't, I'd like to hear about examples of systems that never produce such false positives, and where for some reason it makes sense (and is actually beneficial) to change something later in the file first. Until then, just give me the first error. I won't look at the rest anyway, so no need for the compiler to be "clever" about it.

ufo 8y ago
This works better in an IDE than if you interact with the compiler directly. If you look at the compiler output directly the false positive drown out the actual errors so it is hard to find out what you need to fix. But in an IDE the false positives just become red squiggly lines further down the file, and don't obfuscate the important red squigglys at the top.
evincarofautumn 8y ago
It works best in an IDE, or in my case in Emacs where I can jump to each error in turn. On the command line or in an interactive mode (REPL, notebook, &c.) I usually just want one error at a time. I’ve had fairly good luck with this in C# in Visual Studio or Xamarin Studio with Roslyn, although I don’t actually use .NET much—even though I worked on Mono for a few years, hah. It also really depends on the grammar of the language how well you can do recovery. Ideally a language should be designed from the beginning to support it. Simple example: if a language allows nested function definitions with the same notation as top-level definitions, then you can’t reliably insert a missing closing bracket for a definition, because you can’t assume that the next one is a new top-level; it could be nested. (Although of course you can use heuristics like indentation to get around this.) Anyway, the point of successful recovery is that the diagnostics later in the file are not spurious. And if the compiler supports recoverable parsing, you can still have a mode where it just bails out after the first error if that’s what you prefer.
- tom_mellior 8y ago
  > Anyway, the point of successful recovery is that the diagnostics later in the file are not spurious. True if you only consider superficial syntax things. False in the real world, where compilers also check whether identifiers are declared, and with what type. If you have an error (syntax or semantic) and decide to press on, in general your symbol tables will be in a messed-up state, and you will generate spurious type or "undeclared identifier" errors. Also, in the real world, the compilers I have experience with do not only continue if they can be 100% sure that it makes sense, but they invariably continue further because they try to be "smart". The IDE point is a semi-valid point since it doesn't scroll to the bottom-most, most useless errors like a terminal does. But I'm still not convinced that it adds value. That it enables situations where it's better to fix the second error before the first.