3 ms·

One of the most fundamental experiences I ever had was attempting a graduate level course at the end of a long series on compilers. You really get an eye openin

One of the most fundamental experiences I ever had was attempting a graduate level course at the end of a long series on compilers. You really get an eye opening view of how languages are translated into the language the machine understands. After going through a few toy languages and then finally tackling creating a simple JVM, here is the #1 thing I would go back to myself and scream until I was blue - Make your initial grammar SUPER simple. Like, don't go guns blazing and try to add all these cool features you saw in another language or always thought about. Start stupid, stupid simple, get that working, then build on top of it.

lioeters 2y ago
This is why the Lisp syntax is a great candidate for an exercise in making your own language. For example, Make a Lisp. https://github.com/kanaka/mal https://github.com/kanaka/mal It's simple to lex and parse into an abstract syntax tree, so you can get on with exploring more interesting aspects of programming beyond the mere syntax. (Not to say that there aren't interesting aspects of grammar and innovative syntax, but those can probably be explored later on as macros.) Last time I created a toy language, I implemented a C-like infix syntax but still used a Lisp evaluator at its core from a previous project.
- JohnMakin 2y ago
  > It's simple to lex and parse into an abstract syntax tree, There are some "cheats" for this, tools like ANTLR etc. that are good at generating parsers from a particular grammar. But of course, I think a beginner should try to do this on their own to get a feel for it. Personally, for me, I do find writing parsers a little tedious and not the most fun part of making a language.
whartung 2y ago
For many simple languages, the most complex construct is the expression. Lots of things come to light there. Lots of recursion/fun with stacks, operator precedence, the type system, parameter passing. Pretty much a good solid chunk of language is wrapped up in expressions. Get expressions working, and the rest starts to readily fall into place.
Hunpeter 2y ago
I've been working on a toy compiler on-and-off, which is basically just a "reinvent-the-wheel simulator" since I really haven't looked at much existing literature. A very janky, bug-prone part of it is a sort-of mini parser generator, which you can feed a dictionary of rules (as strings) to. This, while slowing down the compilation speed, has allowed me to expand the grammar incrementally from dead simple to more complex, which has been a nice thing.
hoosieree 2y ago
Building an interpreter for myself, can confirm. Saying "no" to feature requests is especially hard because I'm the one requesting them!
maxbond 2y ago
> Make your initial grammar SUPER simple. I would go further and say don't write a parser at first, unless what's novel and interesting about your language is it's syntax. Use the configuration language of your choice (like TOML or YAML) to write your ASTs directly, so you can focus on writing the runtime/backend and playing with the semantics of your language (where the novelty probably is). When you feel it's the appropriate time, you can circle back to the frontend and implement a parser. But if you're writing a DSL, a config language may well be good enough. An additional bonus is that your config language syntax will make it easier to write certain unit tests. I've had several experiments in writing a new language, and the first few times I got completely bogged down in parsing. I learned a lot about parsing, which is great, but it made it difficult for me to get at the meat of a project.
systemBuilder 2y ago
The difficulty in learning a language is proportional to the SQUARE of the number of BNF rules! Let that sink in. When last I looked, C had 120 rules and C++ had 250. C++ was already out of control and has a bunch of really stupid features that nobody with any intelligence uses for anything other than showing off (and let me tell you - there are A LOT of showoffs at Google!) Anyway, that's why C++ is 4x harder to learn than C ... I call it ... "Don's Law".

lioeters 2y ago

This is why the Lisp syntax is a great candidate for an exercise in making your own language. For example, Make a Lisp. https://github.com/kanaka/mal https://github.com/kanaka/mal It's simple to lex and parse into an abstract syntax tree, so you can get on with exploring more interesting aspects of programming beyond the mere syntax. (Not to say that there aren't interesting aspects of grammar and innovative syntax, but those can probably be explored later on as macros.) Last time I created a toy language, I implemented a C-like infix syntax but still used a Lisp evaluator at its core from a previous project.

JohnMakin 2y ago
> It's simple to lex and parse into an abstract syntax tree, There are some "cheats" for this, tools like ANTLR etc. that are good at generating parsers from a particular grammar. But of course, I think a beginner should try to do this on their own to get a feel for it. Personally, for me, I do find writing parsers a little tedious and not the most fun part of making a language.

whartung 2y ago

For many simple languages, the most complex construct is the expression. Lots of things come to light there. Lots of recursion/fun with stacks, operator precedence, the type system, parameter passing. Pretty much a good solid chunk of language is wrapped up in expressions. Get expressions working, and the rest starts to readily fall into place.

Hunpeter 2y ago

I've been working on a toy compiler on-and-off, which is basically just a "reinvent-the-wheel simulator" since I really haven't looked at much existing literature. A very janky, bug-prone part of it is a sort-of mini parser generator, which you can feed a dictionary of rules (as strings) to. This, while slowing down the compilation speed, has allowed me to expand the grammar incrementally from dead simple to more complex, which has been a nice thing.

hoosieree 2y ago

Building an interpreter for myself, can confirm. Saying "no" to feature requests is especially hard because I'm the one requesting them!

maxbond 2y ago

> Make your initial grammar SUPER simple. I would go further and say don't write a parser at first, unless what's novel and interesting about your language is it's syntax. Use the configuration language of your choice (like TOML or YAML) to write your ASTs directly, so you can focus on writing the runtime/backend and playing with the semantics of your language (where the novelty probably is). When you feel it's the appropriate time, you can circle back to the frontend and implement a parser. But if you're writing a DSL, a config language may well be good enough. An additional bonus is that your config language syntax will make it easier to write certain unit tests. I've had several experiments in writing a new language, and the first few times I got completely bogged down in parsing. I learned a lot about parsing, which is great, but it made it difficult for me to get at the meat of a project.

systemBuilder 2y ago

The difficulty in learning a language is proportional to the SQUARE of the number of BNF rules! Let that sink in. When last I looked, C had 120 rules and C++ had 250. C++ was already out of control and has a bunch of really stupid features that nobody with any intelligence uses for anything other than showing off (and let me tell you - there are A LOT of showoffs at Google!) Anyway, that's why C++ is 4x harder to learn than C ... I call it ... "Don's Law".