4 ms·

I know you have decades of experience with compilers, but are you sure 0.1% isn't a bit low? I can imagine 0.1% to be true if you have experience writing parser

I know you have decades of experience with compilers, but are you sure 0.1% isn't a bit low? I can imagine 0.1% to be true if you have experience writing parsers, and just make a shitty parser for a batch compiler, and spend a lot of time on the semantics on the language. You can also emit some shitty stack machine bytecode with a single AST pass, and write a simple interpreter for that bytecode, and the effort would be about the same as writing the simple parser by hand. And you can spend a lot of time making a parser actually robust, threading file positions through the pipeline, handle errors nicely, allow for efficient incremental parsing, and so on. I could probably spend man-months and man-years on such a parser.

pizlonator 7y ago
It’s probably true. You write the parser once and then eventually forget about it except if there is a language change. Then you spend a small amount of time adding the syntax for that change to the parser and again forget about it. On the other hand, _lots_ of folks work on the optimization passes, instruction selection, register allocation and the core of the IR. Since instruction selection and register allocation have CPU specific features usually, they have some logic whose complexity (in terms of size of the code) is proportional to the number of CPUs targeted. So 0.1% is probably pretty close to the true number.
- pdpi 7y ago
  Inversely with LLVM around (and the JVM, sort of), people now get to focus on building the frontend of the compiler and forget about all those optimisation passes.
  - ernst_klim 7y ago
    >now get to focus on building the frontend of the compiler Although LLVM is cool, it's just a backend, so besides frontend you need some runtime, type checker, type inference, and, if your lang is sufficiently complex, you still need a huge middle end layer which knows something about semantics of your language and how to optimize it properly. So if your language is just a very simple featureless language with procedures and simple data types, like Pascal, you could just write a direct code generator. But when you have a language with GC, exceptions, modules and all that kind of stuff, you need a middle end, and a complex one.
  - pizlonator 7y ago
    Only suckers forget about optimization passes. ;-) Don't forget that: - Optimization work on llvm and other compilers is still happening in anger. - Frontend is a relative term. For example, the "frontend" to LLVM when WebKit used LLVM as a "backend" was a full-blown compiler with >100KLoC of code. Since then that "frontend" grew its own backend and dropped LLVM, but point is, just because you're a frontend doesn't mean you don't have optimizations. Frontend doesn't just mean parser. - Rust's or Haskell's "frontends" to LLVM are seriously impressive compilers and the complexity is not to do with parsing AFAICT but all the other stuff (type checking comes to mind but there's also really impressive lowering and optimization). - Backends aren't just complex because of optimization passes. The register allocator and instruction selector aren't so much optimization passes as necessary transformations for turning a compiler's IR into machine code. Those things just grow and grow. - The whispers in the wind that I'm hearing from compiler folks looking to make bank is that there's shitloads of work to be done porting compilers to new targets. It's much more work to port a compiler to a new target than it is to write a parser. That said, writing a frontend for a language as complex as Rust/Haskell/etc is probably on the same order of complexity as parting a backend or maybe even more complex - but that's not because of parsing; it's usually either because of System F type shit (if you have a statically typed language) or dynamic type inference (if you have a dynamically typed language).
- einpoklum 7y ago
  No, not true at all. It depends on the language of course.
voldacar 7y ago
>You can also emit some shitty stack machine bytecode with a single AST pass This is what the earliest C compilers did, right?
chrisseaton 7y ago
> I know you have decades of experience with compilers, but are you sure 0.1% isn't a bit low? I've been working professionally on my compiler since 2013. That's about 2500 days. 0.1% of that is 2.5 days. Yes, that's about how much of my time I've spent working on parsing issues. Parsing, for compilers, is pretty much a solved problem. We know exactly how to do it. Parsing for editors and things might be more interesting still.
WalterBright 7y ago
> but are you sure 0.1% isn't a bit low? It's probably way too high. The reason is parsing is a small, simple, very well defined problem, and only rarely changes. This is quite unlike anything else in the compiler. Unless, of course, you're talking about the C preprocessor. Prepare to spend an ungodly amount of time on that.

pizlonator 7y ago

It’s probably true. You write the parser once and then eventually forget about it except if there is a language change. Then you spend a small amount of time adding the syntax for that change to the parser and again forget about it. On the other hand, _lots_ of folks work on the optimization passes, instruction selection, register allocation and the core of the IR. Since instruction selection and register allocation have CPU specific features usually, they have some logic whose complexity (in terms of size of the code) is proportional to the number of CPUs targeted. So 0.1% is probably pretty close to the true number.

pdpi 7y ago
Inversely with LLVM around (and the JVM, sort of), people now get to focus on building the frontend of the compiler and forget about all those optimisation passes.
- ernst_klim 7y ago
  >now get to focus on building the frontend of the compiler Although LLVM is cool, it's just a backend, so besides frontend you need some runtime, type checker, type inference, and, if your lang is sufficiently complex, you still need a huge middle end layer which knows something about semantics of your language and how to optimize it properly. So if your language is just a very simple featureless language with procedures and simple data types, like Pascal, you could just write a direct code generator. But when you have a language with GC, exceptions, modules and all that kind of stuff, you need a middle end, and a complex one.
- pizlonator 7y ago
  Only suckers forget about optimization passes. ;-) Don't forget that: - Optimization work on llvm and other compilers is still happening in anger. - Frontend is a relative term. For example, the "frontend" to LLVM when WebKit used LLVM as a "backend" was a full-blown compiler with >100KLoC of code. Since then that "frontend" grew its own backend and dropped LLVM, but point is, just because you're a frontend doesn't mean you don't have optimizations. Frontend doesn't just mean parser. - Rust's or Haskell's "frontends" to LLVM are seriously impressive compilers and the complexity is not to do with parsing AFAICT but all the other stuff (type checking comes to mind but there's also really impressive lowering and optimization). - Backends aren't just complex because of optimization passes. The register allocator and instruction selector aren't so much optimization passes as necessary transformations for turning a compiler's IR into machine code. Those things just grow and grow. - The whispers in the wind that I'm hearing from compiler folks looking to make bank is that there's shitloads of work to be done porting compilers to new targets. It's much more work to port a compiler to a new target than it is to write a parser. That said, writing a frontend for a language as complex as Rust/Haskell/etc is probably on the same order of complexity as parting a backend or maybe even more complex - but that's not because of parsing; it's usually either because of System F type shit (if you have a statically typed language) or dynamic type inference (if you have a dynamically typed language).
einpoklum 7y ago
No, not true at all. It depends on the language of course.

voldacar 7y ago

>You can also emit some shitty stack machine bytecode with a single AST pass This is what the earliest C compilers did, right?

chrisseaton 7y ago

> I know you have decades of experience with compilers, but are you sure 0.1% isn't a bit low? I've been working professionally on my compiler since 2013. That's about 2500 days. 0.1% of that is 2.5 days. Yes, that's about how much of my time I've spent working on parsing issues. Parsing, for compilers, is pretty much a solved problem. We know exactly how to do it. Parsing for editors and things might be more interesting still.

WalterBright 7y ago

> but are you sure 0.1% isn't a bit low? It's probably way too high. The reason is parsing is a small, simple, very well defined problem, and only rarely changes. This is quite unlike anything else in the compiler. Unless, of course, you're talking about the C preprocessor. Prepare to spend an ungodly amount of time on that.