15 ms·

So You Want to Write Your Own Language (2014)

183 points by rspivak 9y ago

rzzzwilson 9y ago
Is it just me being in my dotage, or does anyone else feel that websites that override my "open in new tab" action to force their tab to the top of my view plus tout their app should be taken out behind the shed and shot?
- andreareina 9y ago
  I've said the same thing about applications that steal focus.
  - cryptoz 9y ago
    It's remarkably hard to find or configure an OS to not steal focus. Everything I've tried for my whole life has failed, with the exception of not using a GUI and just staying on the linux command line without letting startx run. iOS? Steals focus all the time. Android? Steals focus all the time. Windows? Steals focus all the time. Mac OS X? Steals focus all the time. Ubuntu? Steals focus all the time. It's outrageous and annoying. Does anyone have a simple OS configuration that will prevent all focus-stealing system dialogs from actually interrupting focus? Edit: To continue the rant, Windows used to pop OS update dialogs that interrupted keyboard focus, took keystrokes like spacebar and enter to mean "lose all work and reboot". I remember a long time ago, typing and looking away from the screen, only to find that my computer had turned off. Infuriating.
    - spathi_fwiffo 9y ago
      Should be possible on Ubuntu. I have most of this set on my KDE setup, but harder to configure than back when i used FVWM. You can at least set sloppy focus. I haven't bothered to look if you can still open behind/lower on open.
    - user5994461 9y ago
      A windows application can't steal the focus unless it is the the top window already. This is behavior built deep into the system that cannot be overridden. It is very effective against numerous stupid apps.
      - TeMPOraL 9y ago
        > A windows application can't steal the focus unless it is the the top window already. Or unless it decides to throw up a modal dialog, AFAIR. Unless they changed that at some point. Also, Windows Update "rebooting in 5 minutes, do you want to reboot now instead?" popups in pre-Win8.
    - Too 9y ago
      When was the last time you used windows? ME? Nowadays the worst thing that can happen is that the task bar icon starts blinking or a non focus stealing balloon popup. (not balloons anymore but don't know the new term) *Win32 does expose functions to enumerate applications and to give arbitrary handles focus. So an app could theoretically use this to give itself focus but that is far from the default for requesting dialogs or attention and no serious application would do such a thing. Such sleazy applications shouldn't be installed in the first place. Unless that application is designed to be a window manager.
    - kevin_thibedeau 9y ago
      This is possible with any number of X11 window managers.
    - TeMPOraL 9y ago
      I'm with you on this. I'd love to have such a configuration too. RE your edit: I feel your pain, had that happen to me many times. Ubuntu does this too with "hey, there are updates ready to be installed!", but at least this doesn't cause a reboot.
  - Pica_soO 9y ago
    I ve spend a good part of my lifetime making that program- it deserves to be the star of the desktop. And calling the wizard of OS to find out what happened after it took gratuitous time to load for the user-attention-whoring is simply cowardice.
nailer 9y ago
> Redundancy. Yes, the grammar should be redundant. You've all heard people say that statement terminating ; are not necessary because the compiler can figure it out. That's true — but such non-redundancy makes for incomprehensible error messages. Consider a syntax with no redundancy: Any random sequence of characters would then be a valid program. No error messages are even possible. A good syntax needs redundancy in order to diagnose errors and give good error messages. This makes no sense. A terminator is required, multiple terminators (ie, redundancy) are not. Various modern languages eliminate redundancy: any random sequence of characters is not a valid program in those languages. Disappointed Dr Dobbs would publish something like this. More subjectively: > Tried and true. Absent a very strong reason, it's best to stick with tried and true grammatical forms for familiar constructs. It really cuts the learning curve for the language and will increase adoption rates. More people will program in the future than who do at present. If you aim to approach this audience, things like using '=' to set values, rather than test equality, make no sense to the vast majority of those people.
- CJefferson 9y ago
  The lack of a terminator in python, and the optional terminator in JavaScript, has often produced weird errors for me, when the compiler and me disagree on if something should be interpreted as one line or two. With a semicolon, it's clearer. Of course, missing semicolon errors are still annoyingly bad, I wish GCC and clang could produce better errors messages in this case.
  - ben-schaaf 9y ago
    I've had similarly weird errors in python, but given the same situations its ever occurred to me in ruby. Maybe its just the way the grammar is defined, for example stuff like the following works perfectly fine in ruby while python just says "Invalid Syntax": foo = 1 + 2
    - optimuspaul 9y ago
      yeah, ruby allows some odd stuff for sure.
dronemallone 9y ago
Related :) - http://colinm.org/language_checklist.html http://colinm.org/language_checklist.html
Cacti 9y ago
If you haven't done this before and want to explore a bit, I'd recommend using ANTLR 3. You can get an interpreter up and running pretty easily and it targets the major platforms, so you can typically use the produced grammars and generators in your language of choice. Recursive descent is usually the easiest to implement. This is really only 5 or 10% of the work involved in creating your own language, but it's enough to experiment and give you an idea if you want to proceed with the long and hard work of developing it further.
- zaphar 9y ago
  The author actually recommends against using an lexer/parser generator. If you are just playing around they can be find but he's right that when you are ready to get serious they often end up being a hindrance. Additionally a it's not usually that hard to create your own tokenizer and parser by hand.
mannykannot 9y ago
There are so many languages available these days, so the first thing to ask is "what is unique and beneficial about my language?" (valid answers might be that it is a unique combination of features seen elsewhere, that it addresses a specific domain's problems better than any general-purpose language, or that it is an experiment.) Someone made an interesting point about Kotlin recently: as one might hope for a language coming from an IDE maker, it has excellent tools. I have heard Alan Kay make a similar comment about Smalltalk. There are a couple of semi-popular languages that I feel are significantly disadvantaged by their lack of tools support (primarily in debugging and documentation.)
- psyc 9y ago
  I have long wanted to see programmers experiment with taking this IDE thing much further. Almost every other authoring tool has an opaque file format, manipulated via one or more views. E.g. when a digital artist wants to create and manage a complex 3D scene, they use a tool like Maya. They're not fussed about whether they can read the file format! So why not try letting the program exist as an embellished AST, that you edit in multiple IDE views? You could still have text views that present the program in one or more programming languages. But also views like circuit diagrams, graphical call graphs, other standard programmer diagram types. A run-time view with a time axis and time controls, with graphical representations of the program, ability to rewind, fiddle with state, travel visually down all the code paths. Can we really do no better than text + our imaginations and whiteboards? I'm almost certain we can do better. Or, if you find those ideas a bit eccentric, then start with just having views of source code. For example, my team requires verbose symbols, but I like very short symbols. I wish I could just have a simple map from the long symbol names to short ones, and toggle my view of the source code without affecting the source code. But since no IDE I know of has the concept of a view, I don't know how to do this today with existing tools.
  - jsymolon 9y ago
    AST = Abstract Syntax Tree. " ... manipulate AST ... " Interesting thought, but that's not a property of a language per se but a function of the IDE.
jakub_g 9y ago
> These are false gods: > Minimizing keystrokes. Maybe this mattered when programmers used paper tape, and it matters for small languages like bash or awk. For larger applications, much more programming time is spent reading than writing, so reducing keystrokes shouldn't be a goal in itself. Of course, I'm not suggesting that large amounts of boilerplate is a good idea. Couldn't agree more! No one needs another perl or bash. Languages should be as concise as possible, but not more. Also good points about familiarity and helpful error messages.
- tyingq 9y ago
  > No one needs another perl or bash. Languages should be as concise as possible, but not more I've seen you lump perl and bash together before, but around the idea of sigils like $ and @. This sounds more like a complaint about "default" operations. Like Perl's $_ perhaps? Other than that sort of thing, I don't see where Perl reaches that far in being concise. Maybe regular expressions being a first class thing? Like "if ($foo =~ /bar/)" ? Though that seems straightforward to me.
- kazinator 9y ago
  Reducing useless punctuation is good. Other than that, reducing the number of tokens is good. If we can express a solution in fewer tokens of one language than of another, that's a good indicator that our language having better abstractions for that problem. The longer solution is perhaps spending some tokens to solve some distracting sub-problems which the shorter one doesn't have to. But if the token count stays the same, and we simply rename the tokens to one or two character sequences, we haven't gained much, and it is possibly even detrimental.
Koshkin 9y ago
Trying yourself at inventing a new programming language is both fun and an excellent intellectual exercise - regardless of whether you want to make your language "better" than the one(s) that you are familiar with. It also opens a path to humility and a true appreciation of work of others.
- andreareina 9y ago
  It's fun taking an existing feature in one language, and implementing it in another language that doesn't have it. Often you'll come up against a simplicity/power tradeoff; making that decision (after thinking hard) and then comparing it to how the others did it is usually an interesting revelation. Lisps are the canonical language for doing this in, but sweet.js macros could enable some interesting results as well.
jokoon 9y ago
Honestly if I'd make a language, I'd take most thing that make C and python popular, to make something as simple as possible, and as similar to C as possible. All I could wish for is C with native features of python (dict, list comprehensions, libs like sqlite and xml...) What I think is the most important thing for a language is that it must be easy to learn and read for beginners. There is no need to have advanced features for advanced programmers. Those guys will use other languages and their specific tools to solve their advanced problems. A language is something that is "talked" by many people, so the easier is it to learn, the more people will use it. It is all about a low barrier of entry. Just stop focusing on particular features or what you like about that X or Y language. Just imagine CS students and beginners, and make a language that let them write code and do their assignments.
- doubleunplussed 9y ago
  Cython is a fantastic and underrated language. It basically allows you to write C using Python syntax, and to also write Python in the same source file. Basically if you use a dict or some other nice python functionality, your code will run at Python speed, but if you add a few type annotations and whatnot it runs like C. Not to mention that you can import and use C libraries. Mix and match Python and C as required for the ultimate balance of speed and convenience. It's great.
  - dom0 9y ago
    I disagree (strongly). Cython is nice to save some typing for generating wrappers. It is a very tough language to work with for actually implementing things that you'd want to do in a low-level language, because it's severely underdocumented and underspecified, but extremely complex at the same time. To actually see whether the code as written is correct, one must almost always cythonize it and dig through the generated, verbose and hard to decipher C code. Both the language and the compiler have a ton of things I'd call bugs, induced by the complexities and mismatches in the type system. E.g. it's awkwardly easy to have Cython call some PyObject function on something that isn't a Python object. No thank you. PS: Also avoid using setup.py Extension, including Cython's variant, if you can at all. (Source: I've been maintaining software containing Cython stuff for some while, written and reviewed quite some of it, too.)
jlebrech 9y ago
what of forking an existing language and adding one of two features you want to see it it. i'd like to make a subset of ruby minus blocks and indentation (yes i'm crazy)
- rurban 9y ago
  see tinyrb based on potion then.
KirinDave 9y ago
Alternatively, just write software in a Lisp variant (with good macros, clojure is quite meh here) or Ocaml or Haskell. These languages put you at the level of writing a language for each problem domain immediately and with a comprehensive and useful toolkit. These languages also do this while directly competing with all but the most carefully tuned of C++/C environments. Steel Bank Common Lisp and the Glasgow Haskell Compiler are good references here. These languages are often considered strange because of this, and feel somewhat alien. But once you realize you're building a language to model your problems, suddenly tons of stuff makes more sense. Previously alien concepts like Macros and Monads are outside the typical language's experience precisely because the language authors created these contexts and put you inside them. This article is sort of fundamentally wrong that it's difficult to write small, purpose built languages. It's not, and even outside the functional and homoiconic meta-syntactic world we have seen code generators deployed regularly. Successful products and libraries are build using these techniques all the time, and with a modern toolchain it delivers excellent results. It's just that other more restricted and guided approaches are often introduced earlier in people's learning curve and sets the expectations for them subsequently.
- flavio81 9y ago
  > Alternatively, just write software in a Lisp variant (with good macros, clojure is quite meh here) or Ocaml or Haskell. These languages put you at the level of writing a language for each problem domain immediately and with a comprehensive and useful toolkit. Great comment. And if a domain-specific-language (DSL) of really custom syntax is needed, an alternative is to use Racket (a Lisp-family language), which allows for creating DSLs of all kind. Racket is already used for designing your own programming language and trying it, so I'm surprised that the author did not mention suggest Racket as a test-bed for the new language. > This article is sort of fundamentally wrong that it's difficult to write small, purpose built languages. It's not, and even outside the functional and homoiconic meta-syntactic world we have seen code generators deployed regularly. Correct. For example, most of the common systems used today do use a 'code generator' internally: Every software that uses an ORM library (i.e. Hibernate), is already using a sort of domain-specific-language (i.e. HQL) that, via a code generator, translates to another language (i.e. SQL). So most major platforms (Java, .NET, etc) already have one or many popular libraries that do implement some sort of (limited) DSL and code generator. I think the aim of the article was to create a new compiler from the ground-up. I think the correct choice for syntax is something that can't be boiled down to three pieces of advice or one article. It is a topic that would require very deep discussion.
WalterBright 9y ago
Author here. So Ask Me Anything!
- chubot 9y ago
  A context-free grammar, besides making things a lot simpler, means that IDEs can do syntax highlighting without integrating most of a compiler front end. As a result, third-party tools become much more likely to exist. This statement feels imprecise in a couple ways. It seems to imply that some IDEs actually use context-free grammars for syntax highlighting? Which ones? As far as I can tell, Vim and Textmate bundles (i.e. what Github uses for syntax highlighting) don't use anything close to a context-free grammar for their syntax highlighting models. They are more like ad hoc lexers -- a collection of rules and regular expressions. Certainly an editor doesn't want to parse the entire file to highlight text, because it has to potentially re-highlight at every keystroke. Also, you want to be able to highlight malformed programs (i.e. code with syntax errors). As far as I understand, that's generally why they don't use grammars. I think you might mean that a language should be designed with a concise grammar so that somebody else can reimplement it by hand more easily? That is, you want multiple independent implementations. If that's true, "context-free grammars" is the wrong term to express that notion. Context-free grammars don't handle many real languages, not just C and C++, but also Python/Perl/Ruby, and even JavaScript and Go (semi-colon insertion.) Also, a lot of tools are enabled not by grammars but by producing specific data structures for your front end. (e.g. here is the way I think about it: http://www.oilshell.org/blog/2017/02/11.html http://www.oilshell.org/blog/2017/02/11.html ) The different between Clang and GCC is that Clang is a library that enables a whole ecosystem of fantastic tools, including IDE support. But Clang doesn't do much at all with CFGs. The real difference is that the front end is a library that produces a very rich representation of the code. The C# front end does similar things. More links here: https://github.com/oilshell/oil/wiki/Lossless-Syntax-Tree-Pattern https://github.com/oilshell/oil/wiki/Lossless-Syntax-Tree-Pa... In other words, I would say that integrating the compiler front end into the IDE became the more popular and successful approach. And of course the compiler very much has to be designed with this use case in mind.
  - WalterBright 9y ago
dadvocate 9y ago
Nice Article, although I clicked on the link thinking that finally I got something that I can follow and get started with writing my own language.
- kozhevnikov 9y ago
  https://news.ycombinator.com/item?id=9718472 https://news.ycombinator.com/item?id=9718472
bobsgame 9y ago
For anyone who hasn't yet seen it, check out Jonathan Blow's Jai language, it is fantastic! https://www.youtube.com/watch?v=gWv_vUgbmug https://www.youtube.com/watch?v=gWv_vUgbmug
- papaf 9y ago
  That's cool. Is there somewhere I can download the language to play with?
  - wulfklaue 9y ago
    No ...
tanilama 9y ago
I don't think so. I feel like invent a language is more and more useless in nowadays.
geekpowa 9y ago
Another reason for writing your own: I have a sprawling codebase written in a 80s/90s era closed 4gl. Crossroads. a) keep kicking this can down the road using what I have inspite of its obvious and increasingly imposing limitations, b) discard it and rewrite or code in something else inspite of loss of hard won business rules, logic, behaviours and feel of the code over the years or c) develop a compiler/runtime and evolve it from there. Went with c). In hindsight was incredibly ambitious but glad I did it. Been evolving compiler and runtime steadily. Now at point where I am starting to think about how to bend the language to discourage certain anti-patterns the language by design encourages and move the code base away from these anti-patterns (excessive use of global variables for one). Alot of consequences I have to accept with this approach though, one obvious one is it is hard to find and hire programmers who want to work on the frankenstein's monster I've created here. Programming seems easy when it's all new and shiny and you are minting things for the first time. Old code is challenging, there are no obvious solutions forward, none I've found at least.

183 points by rspivak 9y ago

Is it just me being in my dotage, or does anyone else feel that websites that override my "open in new tab" action to force their tab to the top of my view plus tout their app should be taken out behind the shed and shot?

andreareina 9y ago
I've said the same thing about applications that steal focus.
- cryptoz 9y ago
  It's remarkably hard to find or configure an OS to not steal focus. Everything I've tried for my whole life has failed, with the exception of not using a GUI and just staying on the linux command line without letting startx run. iOS? Steals focus all the time. Android? Steals focus all the time. Windows? Steals focus all the time. Mac OS X? Steals focus all the time. Ubuntu? Steals focus all the time. It's outrageous and annoying. Does anyone have a simple OS configuration that will prevent all focus-stealing system dialogs from actually interrupting focus? Edit: To continue the rant, Windows used to pop OS update dialogs that interrupted keyboard focus, took keystrokes like spacebar and enter to mean "lose all work and reboot". I remember a long time ago, typing and looking away from the screen, only to find that my computer had turned off. Infuriating.
  - spathi_fwiffo 9y ago
    Should be possible on Ubuntu. I have most of this set on my KDE setup, but harder to configure than back when i used FVWM. You can at least set sloppy focus. I haven't bothered to look if you can still open behind/lower on open.
  - user5994461 9y ago
    A windows application can't steal the focus unless it is the the top window already. This is behavior built deep into the system that cannot be overridden. It is very effective against numerous stupid apps.
    - TeMPOraL 9y ago
      > A windows application can't steal the focus unless it is the the top window already. Or unless it decides to throw up a modal dialog, AFAIR. Unless they changed that at some point. Also, Windows Update "rebooting in 5 minutes, do you want to reboot now instead?" popups in pre-Win8.
  - Too 9y ago
    When was the last time you used windows? ME? Nowadays the worst thing that can happen is that the task bar icon starts blinking or a non focus stealing balloon popup. (not balloons anymore but don't know the new term) *Win32 does expose functions to enumerate applications and to give arbitrary handles focus. So an app could theoretically use this to give itself focus but that is far from the default for requesting dialogs or attention and no serious application would do such a thing. Such sleazy applications shouldn't be installed in the first place. Unless that application is designed to be a window manager.
  - kevin_thibedeau 9y ago
    This is possible with any number of X11 window managers.
  - TeMPOraL 9y ago
    I'm with you on this. I'd love to have such a configuration too. RE your edit: I feel your pain, had that happen to me many times. Ubuntu does this too with "hey, there are updates ready to be installed!", but at least this doesn't cause a reboot.
- Pica_soO 9y ago
  I ve spend a good part of my lifetime making that program- it deserves to be the star of the desktop. And calling the wizard of OS to find out what happened after it took gratuitous time to load for the user-attention-whoring is simply cowardice.

nailer 9y ago

> Redundancy. Yes, the grammar should be redundant. You've all heard people say that statement terminating ; are not necessary because the compiler can figure it out. That's true — but such non-redundancy makes for incomprehensible error messages. Consider a syntax with no redundancy: Any random sequence of characters would then be a valid program. No error messages are even possible. A good syntax needs redundancy in order to diagnose errors and give good error messages. This makes no sense. A terminator is required, multiple terminators (ie, redundancy) are not. Various modern languages eliminate redundancy: any random sequence of characters is not a valid program in those languages. Disappointed Dr Dobbs would publish something like this. More subjectively: > Tried and true. Absent a very strong reason, it's best to stick with tried and true grammatical forms for familiar constructs. It really cuts the learning curve for the language and will increase adoption rates. More people will program in the future than who do at present. If you aim to approach this audience, things like using '=' to set values, rather than test equality, make no sense to the vast majority of those people.

CJefferson 9y ago
The lack of a terminator in python, and the optional terminator in JavaScript, has often produced weird errors for me, when the compiler and me disagree on if something should be interpreted as one line or two. With a semicolon, it's clearer. Of course, missing semicolon errors are still annoyingly bad, I wish GCC and clang could produce better errors messages in this case.
- ben-schaaf 9y ago
  I've had similarly weird errors in python, but given the same situations its ever occurred to me in ruby. Maybe its just the way the grammar is defined, for example stuff like the following works perfectly fine in ruby while python just says "Invalid Syntax": foo = 1 + 2
  - optimuspaul 9y ago
    yeah, ruby allows some odd stuff for sure.

dronemallone 9y ago

Related :) - http://colinm.org/language_checklist.html http://colinm.org/language_checklist.html

Cacti 9y ago

If you haven't done this before and want to explore a bit, I'd recommend using ANTLR 3. You can get an interpreter up and running pretty easily and it targets the major platforms, so you can typically use the produced grammars and generators in your language of choice. Recursive descent is usually the easiest to implement. This is really only 5 or 10% of the work involved in creating your own language, but it's enough to experiment and give you an idea if you want to proceed with the long and hard work of developing it further.

zaphar 9y ago
The author actually recommends against using an lexer/parser generator. If you are just playing around they can be find but he's right that when you are ready to get serious they often end up being a hindrance. Additionally a it's not usually that hard to create your own tokenizer and parser by hand.

mannykannot 9y ago

There are so many languages available these days, so the first thing to ask is "what is unique and beneficial about my language?" (valid answers might be that it is a unique combination of features seen elsewhere, that it addresses a specific domain's problems better than any general-purpose language, or that it is an experiment.) Someone made an interesting point about Kotlin recently: as one might hope for a language coming from an IDE maker, it has excellent tools. I have heard Alan Kay make a similar comment about Smalltalk. There are a couple of semi-popular languages that I feel are significantly disadvantaged by their lack of tools support (primarily in debugging and documentation.)

psyc 9y ago
I have long wanted to see programmers experiment with taking this IDE thing much further. Almost every other authoring tool has an opaque file format, manipulated via one or more views. E.g. when a digital artist wants to create and manage a complex 3D scene, they use a tool like Maya. They're not fussed about whether they can read the file format! So why not try letting the program exist as an embellished AST, that you edit in multiple IDE views? You could still have text views that present the program in one or more programming languages. But also views like circuit diagrams, graphical call graphs, other standard programmer diagram types. A run-time view with a time axis and time controls, with graphical representations of the program, ability to rewind, fiddle with state, travel visually down all the code paths. Can we really do no better than text + our imaginations and whiteboards? I'm almost certain we can do better. Or, if you find those ideas a bit eccentric, then start with just having views of source code. For example, my team requires verbose symbols, but I like very short symbols. I wish I could just have a simple map from the long symbol names to short ones, and toggle my view of the source code without affecting the source code. But since no IDE I know of has the concept of a view, I don't know how to do this today with existing tools.
- jsymolon 9y ago
  AST = Abstract Syntax Tree. " ... manipulate AST ... " Interesting thought, but that's not a property of a language per se but a function of the IDE.

jakub_g 9y ago

> These are false gods: > Minimizing keystrokes. Maybe this mattered when programmers used paper tape, and it matters for small languages like bash or awk. For larger applications, much more programming time is spent reading than writing, so reducing keystrokes shouldn't be a goal in itself. Of course, I'm not suggesting that large amounts of boilerplate is a good idea. Couldn't agree more! No one needs another perl or bash. Languages should be as concise as possible, but not more. Also good points about familiarity and helpful error messages.

tyingq 9y ago
> No one needs another perl or bash. Languages should be as concise as possible, but not more I've seen you lump perl and bash together before, but around the idea of sigils like $ and @. This sounds more like a complaint about "default" operations. Like Perl's $_ perhaps? Other than that sort of thing, I don't see where Perl reaches that far in being concise. Maybe regular expressions being a first class thing? Like "if ($foo =~ /bar/)" ? Though that seems straightforward to me.
kazinator 9y ago
Reducing useless punctuation is good. Other than that, reducing the number of tokens is good. If we can express a solution in fewer tokens of one language than of another, that's a good indicator that our language having better abstractions for that problem. The longer solution is perhaps spending some tokens to solve some distracting sub-problems which the shorter one doesn't have to. But if the token count stays the same, and we simply rename the tokens to one or two character sequences, we haven't gained much, and it is possibly even detrimental.

Koshkin 9y ago

Trying yourself at inventing a new programming language is both fun and an excellent intellectual exercise - regardless of whether you want to make your language "better" than the one(s) that you are familiar with. It also opens a path to humility and a true appreciation of work of others.

andreareina 9y ago
It's fun taking an existing feature in one language, and implementing it in another language that doesn't have it. Often you'll come up against a simplicity/power tradeoff; making that decision (after thinking hard) and then comparing it to how the others did it is usually an interesting revelation. Lisps are the canonical language for doing this in, but sweet.js macros could enable some interesting results as well.

jokoon 9y ago

Honestly if I'd make a language, I'd take most thing that make C and python popular, to make something as simple as possible, and as similar to C as possible. All I could wish for is C with native features of python (dict, list comprehensions, libs like sqlite and xml...) What I think is the most important thing for a language is that it must be easy to learn and read for beginners. There is no need to have advanced features for advanced programmers. Those guys will use other languages and their specific tools to solve their advanced problems. A language is something that is "talked" by many people, so the easier is it to learn, the more people will use it. It is all about a low barrier of entry. Just stop focusing on particular features or what you like about that X or Y language. Just imagine CS students and beginners, and make a language that let them write code and do their assignments.

doubleunplussed 9y ago
Cython is a fantastic and underrated language. It basically allows you to write C using Python syntax, and to also write Python in the same source file. Basically if you use a dict or some other nice python functionality, your code will run at Python speed, but if you add a few type annotations and whatnot it runs like C. Not to mention that you can import and use C libraries. Mix and match Python and C as required for the ultimate balance of speed and convenience. It's great.
- dom0 9y ago
  I disagree (strongly). Cython is nice to save some typing for generating wrappers. It is a very tough language to work with for actually implementing things that you'd want to do in a low-level language, because it's severely underdocumented and underspecified, but extremely complex at the same time. To actually see whether the code as written is correct, one must almost always cythonize it and dig through the generated, verbose and hard to decipher C code. Both the language and the compiler have a ton of things I'd call bugs, induced by the complexities and mismatches in the type system. E.g. it's awkwardly easy to have Cython call some PyObject function on something that isn't a Python object. No thank you. PS: Also avoid using setup.py Extension, including Cython's variant, if you can at all. (Source: I've been maintaining software containing Cython stuff for some while, written and reviewed quite some of it, too.)

jlebrech 9y ago

what of forking an existing language and adding one of two features you want to see it it. i'd like to make a subset of ruby minus blocks and indentation (yes i'm crazy)

rurban 9y ago
see tinyrb based on potion then.

KirinDave 9y ago

Alternatively, just write software in a Lisp variant (with good macros, clojure is quite meh here) or Ocaml or Haskell. These languages put you at the level of writing a language for each problem domain immediately and with a comprehensive and useful toolkit. These languages also do this while directly competing with all but the most carefully tuned of C++/C environments. Steel Bank Common Lisp and the Glasgow Haskell Compiler are good references here. These languages are often considered strange because of this, and feel somewhat alien. But once you realize you're building a language to model your problems, suddenly tons of stuff makes more sense. Previously alien concepts like Macros and Monads are outside the typical language's experience precisely because the language authors created these contexts and put you inside them. This article is sort of fundamentally wrong that it's difficult to write small, purpose built languages. It's not, and even outside the functional and homoiconic meta-syntactic world we have seen code generators deployed regularly. Successful products and libraries are build using these techniques all the time, and with a modern toolchain it delivers excellent results. It's just that other more restricted and guided approaches are often introduced earlier in people's learning curve and sets the expectations for them subsequently.

flavio81 9y ago
> Alternatively, just write software in a Lisp variant (with good macros, clojure is quite meh here) or Ocaml or Haskell. These languages put you at the level of writing a language for each problem domain immediately and with a comprehensive and useful toolkit. Great comment. And if a domain-specific-language (DSL) of really custom syntax is needed, an alternative is to use Racket (a Lisp-family language), which allows for creating DSLs of all kind. Racket is already used for designing your own programming language and trying it, so I'm surprised that the author did not mention suggest Racket as a test-bed for the new language. > This article is sort of fundamentally wrong that it's difficult to write small, purpose built languages. It's not, and even outside the functional and homoiconic meta-syntactic world we have seen code generators deployed regularly. Correct. For example, most of the common systems used today do use a 'code generator' internally: Every software that uses an ORM library (i.e. Hibernate), is already using a sort of domain-specific-language (i.e. HQL) that, via a code generator, translates to another language (i.e. SQL). So most major platforms (Java, .NET, etc) already have one or many popular libraries that do implement some sort of (limited) DSL and code generator. I think the aim of the article was to create a new compiler from the ground-up. I think the correct choice for syntax is something that can't be boiled down to three pieces of advice or one article. It is a topic that would require very deep discussion.

WalterBright 9y ago

Author here. So Ask Me Anything!

chubot 9y ago
A context-free grammar, besides making things a lot simpler, means that IDEs can do syntax highlighting without integrating most of a compiler front end. As a result, third-party tools become much more likely to exist. This statement feels imprecise in a couple ways. It seems to imply that some IDEs actually use context-free grammars for syntax highlighting? Which ones? As far as I can tell, Vim and Textmate bundles (i.e. what Github uses for syntax highlighting) don't use anything close to a context-free grammar for their syntax highlighting models. They are more like ad hoc lexers -- a collection of rules and regular expressions. Certainly an editor doesn't want to parse the entire file to highlight text, because it has to potentially re-highlight at every keystroke. Also, you want to be able to highlight malformed programs (i.e. code with syntax errors). As far as I understand, that's generally why they don't use grammars. I think you might mean that a language should be designed with a concise grammar so that somebody else can reimplement it by hand more easily? That is, you want multiple independent implementations. If that's true, "context-free grammars" is the wrong term to express that notion. Context-free grammars don't handle many real languages, not just C and C++, but also Python/Perl/Ruby, and even JavaScript and Go (semi-colon insertion.) Also, a lot of tools are enabled not by grammars but by producing specific data structures for your front end. (e.g. here is the way I think about it: http://www.oilshell.org/blog/2017/02/11.html http://www.oilshell.org/blog/2017/02/11.html ) The different between Clang and GCC is that Clang is a library that enables a whole ecosystem of fantastic tools, including IDE support. But Clang doesn't do much at all with CFGs. The real difference is that the front end is a library that produces a very rich representation of the code. The C# front end does similar things. More links here: https://github.com/oilshell/oil/wiki/Lossless-Syntax-Tree-Pattern https://github.com/oilshell/oil/wiki/Lossless-Syntax-Tree-Pa... In other words, I would say that integrating the compiler front end into the IDE became the more popular and successful approach. And of course the compiler very much has to be designed with this use case in mind.
- WalterBright 9y ago

dadvocate 9y ago

Nice Article, although I clicked on the link thinking that finally I got something that I can follow and get started with writing my own language.

kozhevnikov 9y ago
https://news.ycombinator.com/item?id=9718472 https://news.ycombinator.com/item?id=9718472

bobsgame 9y ago

For anyone who hasn't yet seen it, check out Jonathan Blow's Jai language, it is fantastic! https://www.youtube.com/watch?v=gWv_vUgbmug https://www.youtube.com/watch?v=gWv_vUgbmug

papaf 9y ago
That's cool. Is there somewhere I can download the language to play with?
- wulfklaue 9y ago
  No ...

tanilama 9y ago

I don't think so. I feel like invent a language is more and more useless in nowadays.

geekpowa 9y ago

Another reason for writing your own: I have a sprawling codebase written in a 80s/90s era closed 4gl. Crossroads. a) keep kicking this can down the road using what I have inspite of its obvious and increasingly imposing limitations, b) discard it and rewrite or code in something else inspite of loss of hard won business rules, logic, behaviours and feel of the code over the years or c) develop a compiler/runtime and evolve it from there. Went with c). In hindsight was incredibly ambitious but glad I did it. Been evolving compiler and runtime steadily. Now at point where I am starting to think about how to bend the language to discourage certain anti-patterns the language by design encourages and move the code base away from these anti-patterns (excessive use of global variables for one). Alot of consequences I have to accept with this approach though, one obvious one is it is hard to find and hire programmers who want to work on the frankenstein's monster I've created here. Programming seems easy when it's all new and shiny and you are minting things for the first time. Old code is challenging, there are no obvious solutions forward, none I've found at least.