5 ms·

I don't think parsing is the hard part of designing a compiler any more

PaulHoule 5y ago
It's not the hard part of "designing a compiler" but who wants to "design a compiler?" In ordinary software people spend most of their time modifying existing code. I wouldn't want to reinvent "javac" or "cpython" but adding a new feature to a compiler like that should be easy. Alternately there are a number of "compiler-technology based tools" that we don't have. Instead we get problems like https://www.liquibase.org/ https://www.liquibase.org/ which defines a whole new XML-based language to manage database changes. That's like jumping from the frying pan to the fire since you're still going to have to understand your RDBMS, how to write SQL for it, what the non-standard features are in your database, etc. A sane tool like that would use an extensible parser generator to easily support the variant syntaxes of different SQL dialects and would be able to parse and unparse SQL or a variant of SQL to define change sets. (speaking of which... shouldn't a compiler generator automatically make something that can reverse the AST back to source code? how about round-tripping source code comments, formatting and all like some CASE tools did in the 1990s?) A while back I had the problem that arangodb scrambled the order of columns in JSON result sets and wrote something that parsed the AQL, figured out what the order of the columns was, and turned the result sets into nice pandas frames. If "writing compilers" was easier people would write "compilers" that do things other than compiling to machine code.
- phillipcarter 5y ago
  > but who wants to "design a compiler?" Lots of people, actually! Just in recent years, Golang, TypeScript, and Rust have shown there's plenty of demand for new languages (compilers!) as developers and their requirements keep evolving. Given how expensive and time-consuming languages are to build and evolve, I'm honestly flabbergasted that we've seen 3+ languages come of age in the past decade and break into the mainstream.
  - nottorp 5y ago
    Let's see... Go is someone at Google going up in the hierarchy by doing things that sound advanced. TypeScript is an attempt to make Javascript less of a mess. Dunno about Rust.
    - phillipcarter 5y ago
      Not sure I understand your point. One of the major reasons why new languages emerge is to solve problems that the current incumbent language(s) cannot solve easily.
      - nottorp 5y ago
        > have shown there's plenty of demand for new languages Guess I'm disagreeing with "demand" in two of the cases. One because you don't know what office politics at Google led to Go being promoted. I strongly doubt it was to solve a problem, just someone with a lot of clout had a pet project. ... and second because it attempts to patch an existing trainwreck and isn't even a new language. Rust does seem well intentioned though so maybe there was a "demand".
        phillipcarter 5y ago
        > I strongly doubt it was to solve a problem, just someone with a lot of clout had a pet project. I don't know office politics there, sure. But Go most definitely solved problems by having a language optimized for fast(er) builds and generally being a language you can write code without thinking too hard about and getting good runtime performance out of. Many languages start out as pet projects, but can often solve a real problem such that they see internal adoption. That adoption can grow for various reasons, and in some cases, end up creating a sociotechnical system where it's now a thing even though it never started out with the intention of being a thing. It's at this time that demand is a key driver behind adoption, new features, etc. Do you still feel that Go "doesn't count" with respect to demand? > second because it attempts to patch an existing trainwreck and isn't even a new language. Firstly, JavaScript is hardly a trainwreck of a language, considering how successful it has been with respect to adoption and having such an incredible ecosystem for developers to work in. I worked on a typed functional programming language for 5 years, so be aware that my statement isn't one of ignorance of other languages. But nonetheless, since this is really a philosophical question, what exactly makes up a new language to you? In TypeScript, the type system is Turing Complete. It's a novel type system; there is no other language with the same set of features and semantics associated with it. I can write very functional and type-heavy TypeScript today and it's really not reminiscent of JavaScript aside from a few syntactical elements. Do you still feel that it "doesn't count"?

It's not the hard part of "designing a compiler" but who wants to "design a compiler?" In ordinary software people spend most of their time modifying existing code. I wouldn't want to reinvent "javac" or "cpython" but adding a new feature to a compiler like that should be easy. Alternately there are a number of "compiler-technology based tools" that we don't have. Instead we get problems like https://www.liquibase.org/ https://www.liquibase.org/ which defines a whole new XML-based language to manage database changes. That's like jumping from the frying pan to the fire since you're still going to have to understand your RDBMS, how to write SQL for it, what the non-standard features are in your database, etc. A sane tool like that would use an extensible parser generator to easily support the variant syntaxes of different SQL dialects and would be able to parse and unparse SQL or a variant of SQL to define change sets. (speaking of which... shouldn't a compiler generator automatically make something that can reverse the AST back to source code? how about round-tripping source code comments, formatting and all like some CASE tools did in the 1990s?) A while back I had the problem that arangodb scrambled the order of columns in JSON result sets and wrote something that parsed the AQL, figured out what the order of the columns was, and turned the result sets into nice pandas frames. If "writing compilers" was easier people would write "compilers" that do things other than compiling to machine code.

phillipcarter 5y ago
> but who wants to "design a compiler?" Lots of people, actually! Just in recent years, Golang, TypeScript, and Rust have shown there's plenty of demand for new languages (compilers!) as developers and their requirements keep evolving. Given how expensive and time-consuming languages are to build and evolve, I'm honestly flabbergasted that we've seen 3+ languages come of age in the past decade and break into the mainstream.
- nottorp 5y ago
  Let's see... Go is someone at Google going up in the hierarchy by doing things that sound advanced. TypeScript is an attempt to make Javascript less of a mess. Dunno about Rust.
  - phillipcarter 5y ago
    Not sure I understand your point. One of the major reasons why new languages emerge is to solve problems that the current incumbent language(s) cannot solve easily.
    - nottorp 5y ago
      > have shown there's plenty of demand for new languages Guess I'm disagreeing with "demand" in two of the cases. One because you don't know what office politics at Google led to Go being promoted. I strongly doubt it was to solve a problem, just someone with a lot of clout had a pet project. ... and second because it attempts to patch an existing trainwreck and isn't even a new language. Rust does seem well intentioned though so maybe there was a "demand".
      - phillipcarter 5y ago
        > I strongly doubt it was to solve a problem, just someone with a lot of clout had a pet project. I don't know office politics there, sure. But Go most definitely solved problems by having a language optimized for fast(er) builds and generally being a language you can write code without thinking too hard about and getting good runtime performance out of. Many languages start out as pet projects, but can often solve a real problem such that they see internal adoption. That adoption can grow for various reasons, and in some cases, end up creating a sociotechnical system where it's now a thing even though it never started out with the intention of being a thing. It's at this time that demand is a key driver behind adoption, new features, etc. Do you still feel that Go "doesn't count" with respect to demand? > second because it attempts to patch an existing trainwreck and isn't even a new language. Firstly, JavaScript is hardly a trainwreck of a language, considering how successful it has been with respect to adoption and having such an incredible ecosystem for developers to work in. I worked on a typed functional programming language for 5 years, so be aware that my statement isn't one of ignorance of other languages. But nonetheless, since this is really a philosophical question, what exactly makes up a new language to you? In TypeScript, the type system is Turing Complete. It's a novel type system; there is no other language with the same set of features and semantics associated with it. I can write very functional and type-heavy TypeScript today and it's really not reminiscent of JavaScript aside from a few syntactical elements. Do you still feel that it "doesn't count"?