18 ms·

So you want to write your own language?

deleted 13y ago
[deleted]
programminggeek 13y ago
The last bit I think is the most important - that you will have to take the show on the road and tell people about it to ever have it really take off is interesting and important. Promoting a piece of tech as critical as a language is no small chore. You can't just put something out on the internet and hope.
- carsongross 13y ago
  Having helped put something out on the internet and hoping (http://gosu-lang.org http://gosu-lang.org) I 100% agree, but I think there is something else in there beyond just advocating a language. Let me try to articulate it: Programming languages don't win because they are better qua programming languages. They win because they solve one problem really well that others do not. Ruby (+ Rails) -> building structured web apps. Javascript -> Being there. C -> Being portable assembler. Java -> C with garbage collection. Consider the greatest language ever, Lisp (I hate it, but I recognize it's power.) The reason it's never taken off is that there is no one big problem that it solves that other languages can't solve easily enough. The bad news for a lot of the better, smaller languages out there is that the newer problems are often solved in libraries and momentum (tooling, deployment support, etc.) in the big languages is huge.
  - falcolas 13y ago
    I would say that Lisp never took off because when it had its moment to shine, it failed in the speed department. Since then, many languages have since filled the expressiveness niche that Lisp filled. Of course, that discounts Clojure, which is showing Lisp's power to a new generation, in an environment they can relate to (the JVM).
  - jerf 13y ago
    In short, a language needs advocates that can brag about something, and the brags need to be objectively accurate. Do not claim the adjective "fast" if I can trivially outrun you in Python, or claim "simple" if I can trivially write shorter source code in Java, etc. It also need to be an interesting thing to brag about, but, honestly, just getting the first bit is enough challenge....
    - Sssnake 13y ago
      >and the brags need to be objectively accurate No, it just needs to be convincing. Plenty of convincing lies (or unproven claims) are used to sell programming languages.
      - WalterBright 13y ago
        cough, cough
      - jerf 13y ago
        The problem with inaccurate bragging is that the loop ends up unclosed; the advocate says "This language is so fast!", the earliest of the early adopters try it and it isn't fast, and so they slag your language on their blog. And in this case, there is a such thing as bad publicity. Even when I disagree with the propaganda of a language community (lookin' right at you, Node), there is enough truth that it made it through that gauntlet. In that case I'd argue it isn't that they lie about Node, it's that they lie about all the other languages they putatively compare themselves to. Apparently that strategy does work....
  - masklinn 13y ago
    > They win because they solve one problem really well that others do not. That assertion's got more holes than emmental. Where does Python fit? C#? Javascript and Objective-C don't really "solve one problem really well", they're the only fucking option in their space. Java was not and has never been "C with garbage collection", the syntax is the only thing it got from C. The language being great at solving a specific use case is definitely a boon and increases its chances (witness PHP or Lua), but it's nowhere near sufficient (Tcl lives in the same niche as Lua — lightweight, extensible, multi-paradigm language for embedding in and scripting of native codebases, predates Lua, and is considered by many of its lovers better than Lua, yet it's dying its slow and protracted death) > The reason it's never taken off is that there is no one big problem that it solves that other languages can't solve easily enough. The reasons it never took of is a combination of little advertising/pushing (compare with java), and highly fragmentary communities (the community unit of Lisp, as those of mages, is 1) leading to no corpus of code. > The bad news for a lot of the better, smaller languages And yet there's never been more uptake in new languages.
    - Silhouette 13y ago
      That assertion's got more holes than emmental. Maybe, but your counterexamples don't seem particularly convincing. Where does Python fit? Originally, a readable scripting language. More recently, a language for back-end web development that doesn't ram OO down your throat. Also carving out something of a niche as a general purpose scripting language embedded in other applications instead of a custom macro language, like Lua but many more people have prior experience programming in it. C# Similar strengths to Java, but with a lot of useful extra features, without most of the limitations that should have gone away a decade ago but didn't, and with a runtime environment that is available by default on Windows. The reasons [Lisp] never took of is a combination of little advertising/pushing ... and highly fragmentary communities ... leading to no corpus of code. Also, a highly uniform and generic structure is not always an advantage in a programming language. Sometimes it's better to have things that are different obviously look like they're different.
  - chrisseaton 13y ago
    Have you thought about moving Gosu to using Truffle and Graal for high performance? http://blog.jruby.org/2014/01/truffle_graal_high_performance_backend/ http://blog.jruby.org/2014/01/truffle_graal_high_performance...
  - programminggeek 13y ago
    I agree that languages need to solve a real problem. When I created Obvious Architecture, I actually realized that Ruby is basically the wrong language to do Obvious in, but I did it anyway because it had enough tools to make it kind of work anyway. Since then, I've been looking for a language that really offers all the bits that would make for a truly compelling Obvious experience without having to write it as a "framework" and it's pretty difficult to find something that really hits all the points well. I haven't ventured into building my own language yet, but it's awfully tempting.
    - Sssnake 13y ago
      I think it is worth considering that perhaps obvious architecture is the wrong thing to make, rather than existing languages being wrong for making it.
  - asdasf 13y ago
    Your own examples contradict your hypothesis. Ruby is a generic scripting language, it is not better or worse than python or perl at anything. Rails made ruby popular due to evangelism, not solving a problem others do not (notice how rails was literally just a crappier django?). "Being portable assembler" is not solving one problem at all, it is one of the most general things you could cite. That literally is being a better language. Java took millions of dollars of marketing to make it popular, people didn't just suddenly flock to it because it was C with garbage collection (which was already a solved problem: see C). Perhaps lisp "never took off" (seems like it took off pretty well to me) because it isn't the greatest language ever? Lisp barely is a language, it is more of a toolkit to build a language with.
    - NAFV_P 13y ago
      > Java took millions of dollars of marketing to make it popular, people didn't just suddenly flock to it because it was C with garbage collection (which was already a solved problem: see C). Could you please explain what you mean? C is the only language I know in any depth. Beyond a basic description, "garbage collection" is a fairly alien concept to me.
      - dllthomas 13y ago
        http://en.wikipedia.org/wiki/Boehm_garbage_collector http://en.wikipedia.org/wiki/Boehm_garbage_collector
        NAFV_P 13y ago
        I was thinking along those lines, cheers.
- fhars 13y ago
  If you actually expect your language to be used (and not only write it because it is fun), you should probably also consider this checklist: http://colinm.org/language_checklist.html http://colinm.org/language_checklist.html It isn't quite serious, but it can prepare you for a lot of the justified and unjustified dismissals you will have to face.
- cpncrunch 13y ago
  It depends what your goal is. My own reason for writing my own programming language was to have a virtual machine for a multi-player 3d games framework that I wrote in 1996. This sat on the back-burner for a few years, and then I used the compiler and virtual machine in a commercial project. Probably less than 100 people have developed for the language (mostly just tweaking the provided applet code), but the goal never was to unleash a new language onto the world. The aim was to provide a rock solid platform for our product that allowed people to customize the applets if they wished, and it achieved that. I would recommend people build their own programming language simply for the hell of it - it's challening, fun, and could come in useful in future. Also it won't take "years" to get a working prototype. You can get something working in less than a month. It will obviously take a lot longer to get it stable and bug-free.
  - DougMerritt 13y ago
    > Also it won't take "years" to get a working prototype. You can get something working in less than a month. It all depends on the language. Tiny domain specific languages might only take hours, under the right circumstances (for an experienced language implementer). When someone says "years", I assume they mean an industrial strength general purpose language.
    - cpncrunch 13y ago
      Yes, perhaps a month was optimistic. Looking back through my emails, I see it took me 5 months (in my spare time) to design my language and instruction set and develop the compiler and VM. After that I wrote the function libraries. This was the first (and only) compiler I've written, and I just bought a book on compiler construction to figure out how to do it. This was an object-oriented C-like language, which was general purpose. It took a few years of actually using the language to make it "instustrial strength". Writing a compiler is a lot simpler than people might think. I used "Practice and Principles of Compiler Building With C", which explains things very well. It seems to be out of print now.
- deleted 13y ago
  [deleted]
stormbrew 13y ago
I really like that this article suggests against using compiler compilers. My experiences have always been much worse when using them. I think the biggest risk when not using them (especially if you're writing a recursive descent by hand) is avoiding the temptation to make things more context sensitive (not necessarily in the rigorous language theory sense) than absolutely necessary.
- anaphor 13y ago
  Yes, as Rob Pike said, just write everything by hand. Ken Thompson also recommends writing your lexers by hand as well! I also agree about the temptation to make it too complex just because how powerful writing it by hand is, I've found myself succumbing a bit to the temptation by adding in the ability to change the behaviour of the lexer in the source itself.
  - wilsonfiifi 13y ago
    +1 for mentioning Rob Pike, the guy's gifted!
    - wilsonfiifi 13y ago
      Yikes! I guess that comment didn't go down well with a few folks! I found his explanation on how the lexer/parser for the templating language used in go http://www.youtube.com/watch?v=HxaD_trXwRE http://www.youtube.com/watch?v=HxaD_trXwRE was created much easier to understand than other media/literature available online on the same subject. Hence my appreciation of his work. :)
      - Dewie 13y ago
        > Yikes! I guess that comment didn't go down well with a few folks! I guess because '+1 I agree' or variations on it aren't appreciated here. That's my impression of this community, anyway. A post usually needs more meat, like the way you elaborated on your OP.
  - saurik 13y ago
deleted 13y ago
[deleted]
rch 13y ago
Though is probably isn't as straightforward as other options, I really like how Lepl is put together: http://www.acooke.org/lepl/intro.html http://www.acooke.org/lepl/intro.html A quick example: https://gist.github.com/rch/8791893 https://gist.github.com/rch/8791893 Edit -- I guess not many people agree with me: http://www.acooke.org/lepl/discontinued.html http://www.acooke.org/lepl/discontinued.html
cocoflunchy 13y ago
If you don't know how to start but want to give it a try, I highly recommend the How To Create Your Own Freaking Awesome Programming Language book http://createyourproglang.com/ http://createyourproglang.com/ . One of the best purchases of this year for sure. I've been writing my own toy compile-to-js language after reading the book: https://github.com/cosmith/panda https://github.com/cosmith/panda (I might have to change the name though, I recently discovered http://yesco.org/panda.html http://yesco.org/panda.html ...)
- gordonguthrie 13y ago
  I am writing an OTP-ish dialect of Erlang that compiles to javascript http://luvv.ie http://luvv.ie Just bought that book on your recommendation - better be good or else ;-D
  - biot 13y ago
    Suggestion: you really need "here's what the syntax looks like" example(s) on your home page with some non-trivial code (more than just hello world). Ideally, show the sample and the JS output side by side like on the CoffeeScript home page: http://coffeescript.org/ http://coffeescript.org/
    - gordonguthrie 13y ago
      I suppose I am being a bit of a blushing bride about the syntax as it isn't mine, just being good old-fashioned Erlang. The demo doesn't show much more than hello world, because that's all there is at the moment - the focus has been on building the end-to-end toolchain and I am now filling out the features, but good suggestion going forward.
      - biot 13y ago
        I read "OTP-ish dialect of Erlang" as meaning it's not quite the same, and I was looking for details of those differences. If it's just plain Erlang, then I see your point.
  - 13y ago
lnanek2 13y ago
Hmm, so this is the author of D. I've never seen D actually used anywhere. Honestly, I think he missed the biggest thing - being employed by a large company that will push your platform. From the Wiki page, D sounds like a replacement systems programming language, but I've never heard of it while Google's Go system programming language replacement is all over the place. Similarly Sun, now Oracle, and Microsoft push languages that actually get used.
- asdasf 13y ago
  Well one other difference is that D is a systems language, and Go is nothing even remotely close to a systems language but initially claimed to be one. There is not much demand for a new systems language, most people are happy to use higher level languages for what they do these days. Most go converts are coming from python and ruby, not C++.
- mercurial 13y ago
  So? Java's success can be largely attributed to Sun's marketing push. On the other hand, no corporation stands behind a language like Lua which is enjoying success. Same for a language like Julia (though very young, it has clearly managed to attract attention).
- swah 13y ago
  Yep, actually Steve Yegge has a rant saying that his friend Walter Bright (creator of D) didn't care enough about marketing.
- billforsternz 13y ago
  The first thing that jumped out at me in the (very interesting I thought) article was this; "If you need validation and encouragement from others, it isn't going to happen". In other words, you need a thick skin as well as technical ability. I remember Walter Bright achieved a certain amount of fame in the programming community back in the 80s by writing and productizing outstanding C and C++ compilers for MS-DOS when that was an important market with big competitors including MS and Borland. He seems to have been plugging away with D, conceived as a better successor for C than C++ for a long time now. I suspect that the reason that C++ is ubiquitous while D remains relatively obscure has a lot more to do with inertia and momentum than the objective merits of the two languages. It's a tough field to get traction in with so many brilliant and opinionated critics and competitors. My impression is that Walter Bright is an outstanding programmer who deserves more recognition and kudos.
agumonkey 13y ago
What do you think about the point of view of skipping concrete syntax and just design the language at the construct level using s-exps ? ( as mentioned here http://cs.brown.edu/courses/cs173/2012/book/Everything__We_Will_Say__About_Parsing.html http://cs.brown.edu/courses/cs173/2012/book/Everything__We_W... )
- mamcx 13y ago
  This how could help in build a C-like or Python-like language? Or you say just make a LISP-like language?
  - agumonkey 13y ago
    meaning the syntax is not the semantic, I can try imperative algol things in lisp clothing, or prolog things or both .. but I won't waste time on syntax, more on semantics.
    - marktangotango 13y ago
      I think that's valid, even desirable. Others have said lexing/parsing are a fraction of the language implementation problem. So, when using s-expression like syntax (xml as some have done) you are basically writing code in the AST. Then you can concentrate on interpretating the AST, or translating it to bytecode for some vm (maybe of your own design) or performing passes on the AST to do things like type checking, up casting values in expressions, optimizations, etc. There is really a lot more to language implementation than lexing parsing.
      - agumonkey 13y ago
        That's the point made by the course instructors, focus on semantics/interpretation/compilation rather. Another point I'd like to see is that with a small syntactic surface it may be easier to compare linguistic traits. Instead of having 219 languages we'd have one core and libs (with their macro sugar) giving the desired features. But maybe that would only move the problem further.
agentultra 13y ago
My unsolicited, biased advice: use s-expressions. Don't trigger a parse simply to compute with the sums of products and integers [0]. If you can think of something better than s-expressions use that. Also, read http://this-plt-life.tumblr.com/ http://this-plt-life.tumblr.com/ and don't take yourself too seriously. Stand on the shoulders of giants and continue reaching for the sky. Languages are super fun. [0] http://www.infoq.com/interviews/mccarthy-elephant-2000 http://www.infoq.com/interviews/mccarthy-elephant-2000
- qznc 13y ago
  Well, infix math is very nice and easy on the other hand. Also, range and index syntax like Python is nice.
  - sillysaurus2 13y ago
    Infix math is only nice because you've been trained to believe it's nice.
    - dllthomas 13y ago
      While I am pretty sure this is entirely true, it's crazy not to recognize that you're producing a language where basically everyone has been thusly trained since kindergarten.
      - stormbrew 13y ago
        I question its truthiness. It's not as if there's some shadowy cabal conspiring to force people to put operators in the middle, we developed the infix notation over a long period of time and it seems likely there are good, though possibly not entirely logical, reasons for it. We humans have funny brains, after all. And if you really want to dive down that rabbit hole I think you have to establish that prefix is better than suffix as well. I'm somewhat dubious of even this claim, since I find suffix easier to reason about. But maybe that's just me.
asdasf 13y ago
I think this shows that Walter has lots of compiler experience, not so much lots of language design experience. You can't get good error messages without semicolons? How can anyone say something that absurd in a serious article? Have you tried a language without them? It works just fine. Minimizing keywords is important not because of a word shortage, but because of an overlap with words the programmer wants to use. It is obvious that if you make "i" a keyword, someone will likely murder you. But there's tons of grey area where you might be stepping on toes and getting in the way, so minimizing the amount of toe stepping is good. Especially since there is absolutely no reason to have lots of keywords.
- Goosey 13y ago
  Full context: "Redundancy. Yes, the grammar should be redundant. You've all heard people say that statement terminating ; are not necessary because the compiler can figure it out. That's true — but such non-redundancy makes for incomprehensible error messages. Consider a syntax with no redundancy: Any random sequence of characters would then be a valid program. No error messages are even possible. A good syntax needs redundancy in order to diagnose errors and give good error messages." Given the context I think it isn't so much that Walter is saying you can't have good error messages without semicolons. He's saying you can't have good error messages without redundancy (in this example a statement terminator).
  - asdasf 13y ago
    >Walter is saying you can't have good error messages without semicolons That is what he said though. I understand it is one example for a larger point, but it is an example that doesn't support that point at all. It is pretty hard to judge the overall point when the example is nonsense.
    - ronaldx 13y ago
      I think you're really missing the point. Your goal is to write a whole programming language. Calculating where each statement starts and ends in order to serve a good error message is misdirected effort. Just use a terminating character, and use the standard one: a semi-colon.
deleted 13y ago
[deleted]
lmm 13y ago
I found http://www.hokstad.com/compiler/ http://www.hokstad.com/compiler/ really educational - implementing a language in the same way I'm used to writing code, by solving one problem at a time. It makes sense in a way that compiler theory lectures never did for me.
webreac 13y ago
I have ideas about a new language since I finished school in 1993. I had many ideas, some of them got implemented in java. Recently, I discovered scala. All my vague ideas are in scala. The only complaint I have against scala is the slowness of the compiler. I have stopped learning about compilers.
- username42 13y ago
  At coursera.org, there is an excellent resource for learning about compilers: "Stanford University - Compilers"
Fasebook 13y ago
Here's a more technical version of a very similar article by the same author that was previously posted on HN: http://www.drdobbs.com/tools/creating-your-own-domain-specific-langua/231002656 http://www.drdobbs.com/tools/creating-your-own-domain-specif...
rurban 13y ago
Surprisingly shadow article for someone like Walter Bright. Just some basic recommendations, but nothing practical? No links to books, parsers, implementations? The easiest start would be "Compiler Construction using Flex and Bison" by Anthony A. Aaby, available as free pdf, and for adding LLVM to the mix the link would be: http://gnuu.org/2009/09/18/writing-your-own-toy-compiler/ http://gnuu.org/2009/09/18/writing-your-own-toy-compiler/ There needs much more to be said, e.g. links to such languages which were written in a day and are not necessarily lisp or scheme. That would be too easy.
- joe_the_user 13y ago
  This is a fabulous article. This article seems like the perfect antidote to the usual "here's the dragon book" article. Articles that point you to the dragon book, give a few ideas of what parser and lexer are, and then turn you loose are just terrible. They are look articles on "how to build a house" that focus on "here's where to buy a backhoe, here's where you can concrete, etc..." Essentially, designing your own programming language is a difficult and in many ways, bad, idea. But what you to have a ghost of a chance is "what choice should you make, what will it feel like" narrative (there should also be "what are the milestones" but given programming, that will sadly be too variable to predict).
- WalterBright 13y ago
  I tried to avoid all the usual stuff that is well covered elsewhere, and touch on things I thought were poorly covered or not even mentioned. After all, it's just an article, not a comprehensive tome.
- acqq 13y ago
  The author writes his own experiences, and they are directly opposite of the "Compiler Construction using Flex and Bison." And he explains why. I also remember Stroustrup's story where once he wanted to develop C++ he got well meant advices to use lex and yacc, and he claims that he much later realized that much better approach for him would have been to code both things by hand. This matches Walter's experiences, and not only his.
jhallenworld 13y ago
Well my pet language Ivy http://sourceforge.net/projects/ivy-lang/ http://sourceforge.net/projects/ivy-lang/ allows you to define new statements via delayed evaluation tricks (but without actual quoting): fn myfor(&init, &test, &incr, &body) *init while *test *body *incr var a myfor a=0, a!=10, ++a print a & creates a zero-argument lambda function out of the argument. * is sugar to call a zero-argument function. Also: fn set(&x y) *x = y set a 7 # Set variable a to 7 It works because when variables are evaluated, the result includes both the value and the address from which it originated. Anyway, creating a language is one thing. Creating a fast compiler for it is quite another. It's impressive how well v8 works for the dynamic language javascript, but the amount of work embedded in it is daunting. On my list of future projects is to try to make a simple SSA-based optimizer.
- mattgreenrocks 13y ago
  That's really awesome!
kriro 13y ago
I'm building a toy-DSL and working through the ANTLR4 book right now mostly because it was the first thing that came up when searching. Since I don't have a formal CS background I'm lacking a bit in the "language development" department. Is there a good overview comparison of strength/weaknesses of what's available (blog post, academic journal don't really care)? I mostly know stuff exists because I've heard it mentioned somewhere (Flex/Bison, yacc, using Prolog :P) and it's not the easiest thing in the world to make an educated decision. My current plan is to stick with ANTR and muddle through and build a prototype then spend a day or two researching alternatives and seeing what comes up but if someone could cut down that research time I'd be thankful :)
- andrewflnr 13y ago
  Recursive descent parsers are easy, at least if your grammar is LL(1). I seem to recall there being a lot of resources on how to make your grammar be LL(1), but I don't recall which one I read.
andrewflnr 13y ago
I have to speak up in defense of minimizing typing. Sure, programming is mostly long stretches of thinking with short stretches of typing, but when you do start typing, it's usually the result of some piece of inspiration. It's especially at those times that typing lots of useless characters is a distraction. Lots of noise in the code is also distracting while reading, which we all know is what we spend most of our time doing, though you don't want it to be too short either.
- Scramblejams 13y ago
  As someone who had a bout with RSI, I agree. Minimizing keystrokes, if it can be done without impacting readability too much, is excellent.
  - WalterBright 13y ago
    I don't have RSI, so don't know from experience, but wouldn't a syntax aware IDE with autocomplete be very useful for minimizing keystrokes without necessarily minimizing the characters?
    - Scramblejams 13y ago
      Probably. I tend to avoid IDEs because to me they feel like straight jackets and their editors tend to be inferior to standalone editors. I'll have to look around at autocomplete options for my usual languages (Erlang, Python) though -- I figured if I couldn't fit what I was using into my head, I needed to work on my head, not on my autocomplete options. But maybe I'm wrong. :-)
- bad_user 13y ago
  In my opinion it is not the light syntax that really minimizes typing, but rather the availability of features in the language that can afford higher level abstractions. Light syntax is useful for readability. I like it when I don't have to end each line with a semicolon, because that's just noise meant to spoon-feed the compiler. However, I hate it when the grammar of the language leads to awful gotchas. And sometimes visual clues are useful, again for readability. Since we are on the subject, I hate CoffeeScript.
je_bailey 13y ago
Every so often, there's an article on HN promoting the concept that everyone should learn to code. I feel the same way about programming languages, that every developer should write a language. It doesn't need to be a great language, or overly functional. For me, it has provided incredible insight into what is sugar, what is required, and why languages are syntaxed the way they are. The years I've spent writing languages has made me a far better developer.
e12e 13y ago
If you want to play with languages, you could do worse than taking a peek at OMeta: http://tinlizzie.org/ometa/ http://tinlizzie.org/ometa/

deleted 13y ago

[deleted]

The last bit I think is the most important - that you will have to take the show on the road and tell people about it to ever have it really take off is interesting and important. Promoting a piece of tech as critical as a language is no small chore. You can't just put something out on the internet and hope.

carsongross 13y ago
Having helped put something out on the internet and hoping (http://gosu-lang.org http://gosu-lang.org) I 100% agree, but I think there is something else in there beyond just advocating a language. Let me try to articulate it: Programming languages don't win because they are better qua programming languages. They win because they solve one problem really well that others do not. Ruby (+ Rails) -> building structured web apps. Javascript -> Being there. C -> Being portable assembler. Java -> C with garbage collection. Consider the greatest language ever, Lisp (I hate it, but I recognize it's power.) The reason it's never taken off is that there is no one big problem that it solves that other languages can't solve easily enough. The bad news for a lot of the better, smaller languages out there is that the newer problems are often solved in libraries and momentum (tooling, deployment support, etc.) in the big languages is huge.
- falcolas 13y ago
  I would say that Lisp never took off because when it had its moment to shine, it failed in the speed department. Since then, many languages have since filled the expressiveness niche that Lisp filled. Of course, that discounts Clojure, which is showing Lisp's power to a new generation, in an environment they can relate to (the JVM).
- jerf 13y ago
  In short, a language needs advocates that can brag about something, and the brags need to be objectively accurate. Do not claim the adjective "fast" if I can trivially outrun you in Python, or claim "simple" if I can trivially write shorter source code in Java, etc. It also need to be an interesting thing to brag about, but, honestly, just getting the first bit is enough challenge....
  - Sssnake 13y ago
    >and the brags need to be objectively accurate No, it just needs to be convincing. Plenty of convincing lies (or unproven claims) are used to sell programming languages.
    - WalterBright 13y ago
      cough, cough
    - jerf 13y ago
      The problem with inaccurate bragging is that the loop ends up unclosed; the advocate says "This language is so fast!", the earliest of the early adopters try it and it isn't fast, and so they slag your language on their blog. And in this case, there is a such thing as bad publicity. Even when I disagree with the propaganda of a language community (lookin' right at you, Node), there is enough truth that it made it through that gauntlet. In that case I'd argue it isn't that they lie about Node, it's that they lie about all the other languages they putatively compare themselves to. Apparently that strategy does work....
- masklinn 13y ago
  > They win because they solve one problem really well that others do not. That assertion's got more holes than emmental. Where does Python fit? C#? Javascript and Objective-C don't really "solve one problem really well", they're the only fucking option in their space. Java was not and has never been "C with garbage collection", the syntax is the only thing it got from C. The language being great at solving a specific use case is definitely a boon and increases its chances (witness PHP or Lua), but it's nowhere near sufficient (Tcl lives in the same niche as Lua — lightweight, extensible, multi-paradigm language for embedding in and scripting of native codebases, predates Lua, and is considered by many of its lovers better than Lua, yet it's dying its slow and protracted death) > The reason it's never taken off is that there is no one big problem that it solves that other languages can't solve easily enough. The reasons it never took of is a combination of little advertising/pushing (compare with java), and highly fragmentary communities (the community unit of Lisp, as those of mages, is 1) leading to no corpus of code. > The bad news for a lot of the better, smaller languages And yet there's never been more uptake in new languages.
  - Silhouette 13y ago
    That assertion's got more holes than emmental. Maybe, but your counterexamples don't seem particularly convincing. Where does Python fit? Originally, a readable scripting language. More recently, a language for back-end web development that doesn't ram OO down your throat. Also carving out something of a niche as a general purpose scripting language embedded in other applications instead of a custom macro language, like Lua but many more people have prior experience programming in it. C# Similar strengths to Java, but with a lot of useful extra features, without most of the limitations that should have gone away a decade ago but didn't, and with a runtime environment that is available by default on Windows. The reasons [Lisp] never took of is a combination of little advertising/pushing ... and highly fragmentary communities ... leading to no corpus of code. Also, a highly uniform and generic structure is not always an advantage in a programming language. Sometimes it's better to have things that are different obviously look like they're different.
- chrisseaton 13y ago
  Have you thought about moving Gosu to using Truffle and Graal for high performance? http://blog.jruby.org/2014/01/truffle_graal_high_performance_backend/ http://blog.jruby.org/2014/01/truffle_graal_high_performance...
- programminggeek 13y ago
  I agree that languages need to solve a real problem. When I created Obvious Architecture, I actually realized that Ruby is basically the wrong language to do Obvious in, but I did it anyway because it had enough tools to make it kind of work anyway. Since then, I've been looking for a language that really offers all the bits that would make for a truly compelling Obvious experience without having to write it as a "framework" and it's pretty difficult to find something that really hits all the points well. I haven't ventured into building my own language yet, but it's awfully tempting.
  - Sssnake 13y ago
    I think it is worth considering that perhaps obvious architecture is the wrong thing to make, rather than existing languages being wrong for making it.
- asdasf 13y ago
  Your own examples contradict your hypothesis. Ruby is a generic scripting language, it is not better or worse than python or perl at anything. Rails made ruby popular due to evangelism, not solving a problem others do not (notice how rails was literally just a crappier django?). "Being portable assembler" is not solving one problem at all, it is one of the most general things you could cite. That literally is being a better language. Java took millions of dollars of marketing to make it popular, people didn't just suddenly flock to it because it was C with garbage collection (which was already a solved problem: see C). Perhaps lisp "never took off" (seems like it took off pretty well to me) because it isn't the greatest language ever? Lisp barely is a language, it is more of a toolkit to build a language with.
  - NAFV_P 13y ago
    > Java took millions of dollars of marketing to make it popular, people didn't just suddenly flock to it because it was C with garbage collection (which was already a solved problem: see C). Could you please explain what you mean? C is the only language I know in any depth. Beyond a basic description, "garbage collection" is a fairly alien concept to me.
    - dllthomas 13y ago
      http://en.wikipedia.org/wiki/Boehm_garbage_collector http://en.wikipedia.org/wiki/Boehm_garbage_collector
      - NAFV_P 13y ago
        I was thinking along those lines, cheers.
fhars 13y ago
If you actually expect your language to be used (and not only write it because it is fun), you should probably also consider this checklist: http://colinm.org/language_checklist.html http://colinm.org/language_checklist.html It isn't quite serious, but it can prepare you for a lot of the justified and unjustified dismissals you will have to face.
cpncrunch 13y ago
It depends what your goal is. My own reason for writing my own programming language was to have a virtual machine for a multi-player 3d games framework that I wrote in 1996. This sat on the back-burner for a few years, and then I used the compiler and virtual machine in a commercial project. Probably less than 100 people have developed for the language (mostly just tweaking the provided applet code), but the goal never was to unleash a new language onto the world. The aim was to provide a rock solid platform for our product that allowed people to customize the applets if they wished, and it achieved that. I would recommend people build their own programming language simply for the hell of it - it's challening, fun, and could come in useful in future. Also it won't take "years" to get a working prototype. You can get something working in less than a month. It will obviously take a lot longer to get it stable and bug-free.
- DougMerritt 13y ago
  > Also it won't take "years" to get a working prototype. You can get something working in less than a month. It all depends on the language. Tiny domain specific languages might only take hours, under the right circumstances (for an experienced language implementer). When someone says "years", I assume they mean an industrial strength general purpose language.
  - cpncrunch 13y ago
    Yes, perhaps a month was optimistic. Looking back through my emails, I see it took me 5 months (in my spare time) to design my language and instruction set and develop the compiler and VM. After that I wrote the function libraries. This was the first (and only) compiler I've written, and I just bought a book on compiler construction to figure out how to do it. This was an object-oriented C-like language, which was general purpose. It took a few years of actually using the language to make it "instustrial strength". Writing a compiler is a lot simpler than people might think. I used "Practice and Principles of Compiler Building With C", which explains things very well. It seems to be out of print now.
deleted 13y ago
[deleted]

stormbrew 13y ago

I really like that this article suggests against using compiler compilers. My experiences have always been much worse when using them. I think the biggest risk when not using them (especially if you're writing a recursive descent by hand) is avoiding the temptation to make things more context sensitive (not necessarily in the rigorous language theory sense) than absolutely necessary.

anaphor 13y ago
Yes, as Rob Pike said, just write everything by hand. Ken Thompson also recommends writing your lexers by hand as well! I also agree about the temptation to make it too complex just because how powerful writing it by hand is, I've found myself succumbing a bit to the temptation by adding in the ability to change the behaviour of the lexer in the source itself.
- wilsonfiifi 13y ago
  +1 for mentioning Rob Pike, the guy's gifted!
  - wilsonfiifi 13y ago
    Yikes! I guess that comment didn't go down well with a few folks! I found his explanation on how the lexer/parser for the templating language used in go http://www.youtube.com/watch?v=HxaD_trXwRE http://www.youtube.com/watch?v=HxaD_trXwRE was created much easier to understand than other media/literature available online on the same subject. Hence my appreciation of his work. :)
    - Dewie 13y ago
      > Yikes! I guess that comment didn't go down well with a few folks! I guess because '+1 I agree' or variations on it aren't appreciated here. That's my impression of this community, anyway. A post usually needs more meat, like the way you elaborated on your OP.
- saurik 13y ago

deleted 13y ago

[deleted]

rch 13y ago

Though is probably isn't as straightforward as other options, I really like how Lepl is put together: http://www.acooke.org/lepl/intro.html http://www.acooke.org/lepl/intro.html A quick example: https://gist.github.com/rch/8791893 https://gist.github.com/rch/8791893 Edit -- I guess not many people agree with me: http://www.acooke.org/lepl/discontinued.html http://www.acooke.org/lepl/discontinued.html

cocoflunchy 13y ago

If you don't know how to start but want to give it a try, I highly recommend the How To Create Your Own Freaking Awesome Programming Language book http://createyourproglang.com/ http://createyourproglang.com/ . One of the best purchases of this year for sure. I've been writing my own toy compile-to-js language after reading the book: https://github.com/cosmith/panda https://github.com/cosmith/panda (I might have to change the name though, I recently discovered http://yesco.org/panda.html http://yesco.org/panda.html ...)

gordonguthrie 13y ago
I am writing an OTP-ish dialect of Erlang that compiles to javascript http://luvv.ie http://luvv.ie Just bought that book on your recommendation - better be good or else ;-D
- biot 13y ago
  Suggestion: you really need "here's what the syntax looks like" example(s) on your home page with some non-trivial code (more than just hello world). Ideally, show the sample and the JS output side by side like on the CoffeeScript home page: http://coffeescript.org/ http://coffeescript.org/
  - gordonguthrie 13y ago
    I suppose I am being a bit of a blushing bride about the syntax as it isn't mine, just being good old-fashioned Erlang. The demo doesn't show much more than hello world, because that's all there is at the moment - the focus has been on building the end-to-end toolchain and I am now filling out the features, but good suggestion going forward.
    - biot 13y ago
      I read "OTP-ish dialect of Erlang" as meaning it's not quite the same, and I was looking for details of those differences. If it's just plain Erlang, then I see your point.
- 13y ago

lnanek2 13y ago

Hmm, so this is the author of D. I've never seen D actually used anywhere. Honestly, I think he missed the biggest thing - being employed by a large company that will push your platform. From the Wiki page, D sounds like a replacement systems programming language, but I've never heard of it while Google's Go system programming language replacement is all over the place. Similarly Sun, now Oracle, and Microsoft push languages that actually get used.

asdasf 13y ago
Well one other difference is that D is a systems language, and Go is nothing even remotely close to a systems language but initially claimed to be one. There is not much demand for a new systems language, most people are happy to use higher level languages for what they do these days. Most go converts are coming from python and ruby, not C++.
mercurial 13y ago
So? Java's success can be largely attributed to Sun's marketing push. On the other hand, no corporation stands behind a language like Lua which is enjoying success. Same for a language like Julia (though very young, it has clearly managed to attract attention).
swah 13y ago
Yep, actually Steve Yegge has a rant saying that his friend Walter Bright (creator of D) didn't care enough about marketing.
billforsternz 13y ago
The first thing that jumped out at me in the (very interesting I thought) article was this; "If you need validation and encouragement from others, it isn't going to happen". In other words, you need a thick skin as well as technical ability. I remember Walter Bright achieved a certain amount of fame in the programming community back in the 80s by writing and productizing outstanding C and C++ compilers for MS-DOS when that was an important market with big competitors including MS and Borland. He seems to have been plugging away with D, conceived as a better successor for C than C++ for a long time now. I suspect that the reason that C++ is ubiquitous while D remains relatively obscure has a lot more to do with inertia and momentum than the objective merits of the two languages. It's a tough field to get traction in with so many brilliant and opinionated critics and competitors. My impression is that Walter Bright is an outstanding programmer who deserves more recognition and kudos.

agumonkey 13y ago

What do you think about the point of view of skipping concrete syntax and just design the language at the construct level using s-exps ? ( as mentioned here http://cs.brown.edu/courses/cs173/2012/book/Everything__We_Will_Say__About_Parsing.html http://cs.brown.edu/courses/cs173/2012/book/Everything__We_W... )

mamcx 13y ago
This how could help in build a C-like or Python-like language? Or you say just make a LISP-like language?
- agumonkey 13y ago
  meaning the syntax is not the semantic, I can try imperative algol things in lisp clothing, or prolog things or both .. but I won't waste time on syntax, more on semantics.
  - marktangotango 13y ago
    I think that's valid, even desirable. Others have said lexing/parsing are a fraction of the language implementation problem. So, when using s-expression like syntax (xml as some have done) you are basically writing code in the AST. Then you can concentrate on interpretating the AST, or translating it to bytecode for some vm (maybe of your own design) or performing passes on the AST to do things like type checking, up casting values in expressions, optimizations, etc. There is really a lot more to language implementation than lexing parsing.
    - agumonkey 13y ago
      That's the point made by the course instructors, focus on semantics/interpretation/compilation rather. Another point I'd like to see is that with a small syntactic surface it may be easier to compare linguistic traits. Instead of having 219 languages we'd have one core and libs (with their macro sugar) giving the desired features. But maybe that would only move the problem further.

agentultra 13y ago

My unsolicited, biased advice: use s-expressions. Don't trigger a parse simply to compute with the sums of products and integers [0]. If you can think of something better than s-expressions use that. Also, read http://this-plt-life.tumblr.com/ http://this-plt-life.tumblr.com/ and don't take yourself too seriously. Stand on the shoulders of giants and continue reaching for the sky. Languages are super fun. [0] http://www.infoq.com/interviews/mccarthy-elephant-2000 http://www.infoq.com/interviews/mccarthy-elephant-2000

qznc 13y ago
Well, infix math is very nice and easy on the other hand. Also, range and index syntax like Python is nice.
- sillysaurus2 13y ago
  Infix math is only nice because you've been trained to believe it's nice.
  - dllthomas 13y ago
    While I am pretty sure this is entirely true, it's crazy not to recognize that you're producing a language where basically everyone has been thusly trained since kindergarten.
    - stormbrew 13y ago
      I question its truthiness. It's not as if there's some shadowy cabal conspiring to force people to put operators in the middle, we developed the infix notation over a long period of time and it seems likely there are good, though possibly not entirely logical, reasons for it. We humans have funny brains, after all. And if you really want to dive down that rabbit hole I think you have to establish that prefix is better than suffix as well. I'm somewhat dubious of even this claim, since I find suffix easier to reason about. But maybe that's just me.

asdasf 13y ago

I think this shows that Walter has lots of compiler experience, not so much lots of language design experience. You can't get good error messages without semicolons? How can anyone say something that absurd in a serious article? Have you tried a language without them? It works just fine. Minimizing keywords is important not because of a word shortage, but because of an overlap with words the programmer wants to use. It is obvious that if you make "i" a keyword, someone will likely murder you. But there's tons of grey area where you might be stepping on toes and getting in the way, so minimizing the amount of toe stepping is good. Especially since there is absolutely no reason to have lots of keywords.

Goosey 13y ago
Full context: "Redundancy. Yes, the grammar should be redundant. You've all heard people say that statement terminating ; are not necessary because the compiler can figure it out. That's true — but such non-redundancy makes for incomprehensible error messages. Consider a syntax with no redundancy: Any random sequence of characters would then be a valid program. No error messages are even possible. A good syntax needs redundancy in order to diagnose errors and give good error messages." Given the context I think it isn't so much that Walter is saying you can't have good error messages without semicolons. He's saying you can't have good error messages without redundancy (in this example a statement terminator).
- asdasf 13y ago
  >Walter is saying you can't have good error messages without semicolons That is what he said though. I understand it is one example for a larger point, but it is an example that doesn't support that point at all. It is pretty hard to judge the overall point when the example is nonsense.
  - ronaldx 13y ago
    I think you're really missing the point. Your goal is to write a whole programming language. Calculating where each statement starts and ends in order to serve a good error message is misdirected effort. Just use a terminating character, and use the standard one: a semi-colon.

deleted 13y ago

[deleted]

lmm 13y ago

I found http://www.hokstad.com/compiler/ http://www.hokstad.com/compiler/ really educational - implementing a language in the same way I'm used to writing code, by solving one problem at a time. It makes sense in a way that compiler theory lectures never did for me.

webreac 13y ago

I have ideas about a new language since I finished school in 1993. I had many ideas, some of them got implemented in java. Recently, I discovered scala. All my vague ideas are in scala. The only complaint I have against scala is the slowness of the compiler. I have stopped learning about compilers.

username42 13y ago
At coursera.org, there is an excellent resource for learning about compilers: "Stanford University - Compilers"

Fasebook 13y ago

Here's a more technical version of a very similar article by the same author that was previously posted on HN: http://www.drdobbs.com/tools/creating-your-own-domain-specific-langua/231002656 http://www.drdobbs.com/tools/creating-your-own-domain-specif...

rurban 13y ago

Surprisingly shadow article for someone like Walter Bright. Just some basic recommendations, but nothing practical? No links to books, parsers, implementations? The easiest start would be "Compiler Construction using Flex and Bison" by Anthony A. Aaby, available as free pdf, and for adding LLVM to the mix the link would be: http://gnuu.org/2009/09/18/writing-your-own-toy-compiler/ http://gnuu.org/2009/09/18/writing-your-own-toy-compiler/ There needs much more to be said, e.g. links to such languages which were written in a day and are not necessarily lisp or scheme. That would be too easy.

joe_the_user 13y ago
This is a fabulous article. This article seems like the perfect antidote to the usual "here's the dragon book" article. Articles that point you to the dragon book, give a few ideas of what parser and lexer are, and then turn you loose are just terrible. They are look articles on "how to build a house" that focus on "here's where to buy a backhoe, here's where you can concrete, etc..." Essentially, designing your own programming language is a difficult and in many ways, bad, idea. But what you to have a ghost of a chance is "what choice should you make, what will it feel like" narrative (there should also be "what are the milestones" but given programming, that will sadly be too variable to predict).
WalterBright 13y ago
I tried to avoid all the usual stuff that is well covered elsewhere, and touch on things I thought were poorly covered or not even mentioned. After all, it's just an article, not a comprehensive tome.
acqq 13y ago
The author writes his own experiences, and they are directly opposite of the "Compiler Construction using Flex and Bison." And he explains why. I also remember Stroustrup's story where once he wanted to develop C++ he got well meant advices to use lex and yacc, and he claims that he much later realized that much better approach for him would have been to code both things by hand. This matches Walter's experiences, and not only his.

jhallenworld 13y ago

Well my pet language Ivy http://sourceforge.net/projects/ivy-lang/ http://sourceforge.net/projects/ivy-lang/ allows you to define new statements via delayed evaluation tricks (but without actual quoting): fn myfor(&init, &test, &incr, &body) *init while *test *body *incr var a myfor a=0, a!=10, ++a print a & creates a zero-argument lambda function out of the argument. * is sugar to call a zero-argument function. Also: fn set(&x y) *x = y set a 7 # Set variable a to 7 It works because when variables are evaluated, the result includes both the value and the address from which it originated. Anyway, creating a language is one thing. Creating a fast compiler for it is quite another. It's impressive how well v8 works for the dynamic language javascript, but the amount of work embedded in it is daunting. On my list of future projects is to try to make a simple SSA-based optimizer.

mattgreenrocks 13y ago
That's really awesome!

kriro 13y ago

I'm building a toy-DSL and working through the ANTLR4 book right now mostly because it was the first thing that came up when searching. Since I don't have a formal CS background I'm lacking a bit in the "language development" department. Is there a good overview comparison of strength/weaknesses of what's available (blog post, academic journal don't really care)? I mostly know stuff exists because I've heard it mentioned somewhere (Flex/Bison, yacc, using Prolog :P) and it's not the easiest thing in the world to make an educated decision. My current plan is to stick with ANTR and muddle through and build a prototype then spend a day or two researching alternatives and seeing what comes up but if someone could cut down that research time I'd be thankful :)

andrewflnr 13y ago
Recursive descent parsers are easy, at least if your grammar is LL(1). I seem to recall there being a lot of resources on how to make your grammar be LL(1), but I don't recall which one I read.

andrewflnr 13y ago

I have to speak up in defense of minimizing typing. Sure, programming is mostly long stretches of thinking with short stretches of typing, but when you do start typing, it's usually the result of some piece of inspiration. It's especially at those times that typing lots of useless characters is a distraction. Lots of noise in the code is also distracting while reading, which we all know is what we spend most of our time doing, though you don't want it to be too short either.

Scramblejams 13y ago
As someone who had a bout with RSI, I agree. Minimizing keystrokes, if it can be done without impacting readability too much, is excellent.
- WalterBright 13y ago
  I don't have RSI, so don't know from experience, but wouldn't a syntax aware IDE with autocomplete be very useful for minimizing keystrokes without necessarily minimizing the characters?
  - Scramblejams 13y ago
    Probably. I tend to avoid IDEs because to me they feel like straight jackets and their editors tend to be inferior to standalone editors. I'll have to look around at autocomplete options for my usual languages (Erlang, Python) though -- I figured if I couldn't fit what I was using into my head, I needed to work on my head, not on my autocomplete options. But maybe I'm wrong. :-)
bad_user 13y ago
In my opinion it is not the light syntax that really minimizes typing, but rather the availability of features in the language that can afford higher level abstractions. Light syntax is useful for readability. I like it when I don't have to end each line with a semicolon, because that's just noise meant to spoon-feed the compiler. However, I hate it when the grammar of the language leads to awful gotchas. And sometimes visual clues are useful, again for readability. Since we are on the subject, I hate CoffeeScript.

je_bailey 13y ago

Every so often, there's an article on HN promoting the concept that everyone should learn to code. I feel the same way about programming languages, that every developer should write a language. It doesn't need to be a great language, or overly functional. For me, it has provided incredible insight into what is sugar, what is required, and why languages are syntaxed the way they are. The years I've spent writing languages has made me a far better developer.

e12e 13y ago

If you want to play with languages, you could do worse than taking a peek at OMeta: http://tinlizzie.org/ometa/ http://tinlizzie.org/ometa/