11 ms·

A Programmable Programming Language

283 points by samth 9y ago

montrose 9y ago
"Lisp is a programmable programming language." — John Foderaro, CACM, September 1991.
- matthias_f 9y ago
  Yes our title intentionally alludes to CL.
nickpsecurity 9y ago
Language-oriented programming sounds like metaprogramming with DSL’s just with a new toolkit. Language-oriented programming might be a more approachable term for that, though. If I heard it, the first things I’d think of were tools such as Rascal and Ometa that let one arbitrarily define, operate on, or use languages. That covers language part. Far as integration, a number of techs supporting DSL’s… probably a lot of LISP’s and REBOL… had a powerful language underneath one could drop into. So, this seems like a new instance of old ideas they’re giving a new name. I do like how they’ve integrated the concept into a general-purpose, GUI-based, programming environment instead of it being it’s own thing like the others. You get best of both worlds. An old idea I had was researchers should do more experiments in building things like Rascal or Ometa alternatives in Racket leveraging what they already have to see how far one can stretch and improve it. http://www.rascal-mpl.org/ http://www.rascal-mpl.org/ https://en.wikipedia.org/wiki/OMeta https://en.wikipedia.org/wiki/OMeta
- stevekemp 9y ago
  Indeed, though I admit my first thought when reading things like this is of FORTH rather than Lisp. I think there's a lot of value in DSL and implementing them in some languages is a lot easier than others.
- gfodor 9y ago
  Language oriented programming is far from a new term. See the history of jetbrains MPS and intentional software. There was a strong meme in the early 2000's this was the future, it never seemed to take off.
  - username223 9y ago
    I even bought into that meme a bit in the early 2000s. It turns out that most "language" designers (including myself) aren't that good at it, and that designing a language that isn't just yet another leaky abstraction takes a lot of work. Your "interface" is the entire syntax and semantics of your "language."
    - stevedonovan 9y ago
      Yes - there is definitely a hierarchy of difficulty: user code, library, macros, full syntax extension. The trouble is that designing new languages takes a lot of _taste_ and experience. We don't know the full domain, so DSLs tend to have bits added to them as knowledge of the domain grows - and that's the scary point in language evolution. So, despite these things being tremendous fun, I tend more towards correctness and clarity at the expense of sheer expressiveness. (An example of ad-hoc language design is CMake, which makes my eyes bleed. Kitware should have used an existing language - even tcl would have been better.)
samdg 9y ago
Title should be edited: Lanugage -> Language.
- sctb 9y ago
  Thanks! Updated.
hyperion2010 9y ago
I have been building a new not-exactly-a-programming-language using Racket and it has been an absolute pleasure, and the community has been extremely friendly and helpful. One existing stumbling block that I encountered on the way is that I could not find documentation for the full process for the setup for creating a new packaged language (I have been known to fail hard at finding the right documents so this may just be my problem). https://beautifulracket.com/ https://beautifulracket.com/ has excellent coverage of the general process, but the differences between #lang br and #lang racket/base are just large enough that I couldn't fill in the gaps without going and looking through multiple repos to see how others had implemented their repo layout and then could intuit which setup commands I needed to run. If I find time I intend to write up my findings and submit them to the documentation, the short version is here in case someone finds it useful. repo-base ├── .git ├── my-lang ; run `raco pkg install` here │ └── info.rkt ; meta package depends on my-lang-lib └── my-lang-lib ; run `raco pkg install` here ├── info.rkt ; include actual dependencies └── my-lang ; implementation goes here └── main.rkt ; in theory you can put your whole implementation here Once that set up is complete you should be able to use `#lang my-lang`.
- araes 9y ago
  Thank you for the link. Reading the example on the creation of stacker.rkt (a toy reverse polish language) was dramatically more enlightening than the main article. As I understand, Racket = source-to-source interpreter with a lot of helpful shorthand for defining syntax of {insert language here} or arbitrary language you make on the spot. Do they already have libraries for major languages like c++, JS, python? Would be nice to write in a syntax I like (perl) and turn it into c++ that compiled.
  - hyperion2010 9y ago
    Racket isn't really a source to source compiler/interpreter (you could try to write a compiler that converted racket into your language of choice, but that can often be quite difficult). Everything ultimately has to pass through Racket's language semantics, so matching implementation details requires a lot of work if the underlying language spec is large. The #lang C example down below sidesteps these issues because it acts as a pass through to the system c compiler and then handles returning those results into racket, essentially it delegates the language semantics to the c compiler and only deals with how to map the results of calling c code back onto racket objects. Similar issue with the FFI if you want to translate from one language to another, you still have to go through the process of mapping language semantics. Two examples where languages/parsers have been implemented in racket: Algol60 is the best and most complete example https://docs.racket-lang.org/algol60/index.html https://docs.racket-lang.org/algol60/index.html. A python parser has been implemented https://github.com/dyoo/ragg/blob/master/ragg/examples/python-grammar.rkt https://github.com/dyoo/ragg/blob/master/ragg/examples/pytho...
yogthos 9y ago
Instead of having to think about every possible way a language might be used and account for it up front, languages with good metaprogramming facilities let the users them to fit their needs. This way the core language can stay small and focused while remaining flexible. While you don’t want everybody building their own language, userland extensibility can play a huge impact on the overall cleanliness of the language because new ideas can be tried out as libraries. If these ideas don’t pan out, the community moves on. Projects using these libraries will keep working, but the core language doesn’t end up accumulating cruft. I wrote about that in detail here: https://yogthos.net/posts/2017-10-03-MovingForwardByLettingGo.html https://yogthos.net/posts/2017-10-03-MovingForwardByLettingG...
- mcguire 9y ago
  ...which is why Scheme has, like, 37 different, incompatible object systems. Everything has costs.
  - eadmund 9y ago
    That's Scheme. Lisp, meanwhile, has one: CLOS. One is excellent for learning about things like hand-rolled object systems, and one is excellent for building large systems.
    - iak8god 9y ago
      Common Lisp has CLOS. I'm aware some purists don't think Scheme is a Lisp but, regardless of one's stance on this, there are most definitely other Lisps than Common Lisp.
      - YouAreGreat 9y ago
        Having called the result of Lisp standardization ANSI Common Lisp instead of just ANSI Lisp doesn't change the fact that the whole effort was all about standardizing Lisp. Moreover, why not just say Scheme or Clojure when talking about Scheme or Clojure? If ANSI C had been called ANSI Common C, would Java fans never cease to insist on calling Java a dialect of C?
MaxBarraclough 9y ago
> Racket eliminates the hard boundary between library and language, overcoming a seemingly intractable conflict I don't buy it. A language extension means the syntax wouldn't ever be valid in the base language. The fact that Racket defines its 'for' construct using macros/functions/whatever is cute, but I see no reason to pretend it's a language extension.
- soegaard 9y ago
  > A language extension means the syntax wouldn't ever be > valid in the base language. Well, this works: #lang base-without-for (require the-for-library) ...use for here... Of course if the author of `for` has made a `base-with-for` language, then I can just use: #lang base-with-for ...use for here...
- shakna 9y ago
  > I don't buy it. > A language extension means the syntax wouldn't ever be valid in the base language. Sure. (struct pt ([x : Real] [y : Real])) That's not valid Racket, but is valid Typed Racket. A more extreme example: int main(int argc, char** argv) { return 0; } That's C! Oh wait, it's the C #lang [0]... Which means you can things like: base.rkt: #lang C float f( float x1, float x2 ) { return x1 / x2; } main.rkt: #lang racket (require base) f(10.2, 1.8) Those syntaxes aren't compatible, by any stretch. The magic is, Racket gives you access to powerful pre-processing, in the form of some amazing reader macros. What syntax you use, doesn't really matter, so long as you provide that bridge. They're incompatible syntax, with a custom made parser allowing data to flow both ways. [0] https://github.com/jeapostrophe/c https://github.com/jeapostrophe/c
  - mcguire 9y ago
    Shouldn't that second example be (f 10.2 1.8)
    - shakna 9y ago
      Yep. My bad.
zitterbewegung 9y ago
Racket is awesome. I originally used scheme48 but was impressed with rackets tooling (when it was known as plt scheme) . After it branched out as racket and added a large standard library with batteries included it became much more practical to use. I am experimenting with using it with smart contracts and having a library where you can make your own language and work with program synthesis to web development it’s really awesome!
apeace 9y ago
I recall going to a talk when I was at Northeastern, and at one point the speaker said something like, "But why would you create a new programming language just to solve one problem?" Matthias Felleisen (the guy in the video) interrupted him from the audience to respond, "I know several people who have done exactly that, and it worked out quite well for them!" As he says in the video, Northeastern teaches students how to program in a systematic way, rather than by copy-paste-modify from other examples. They certainly teach a powerful way of thinking, and it has served me well over the years. If anyone out there is reading this and is thinking about going to college for computer science, definitely check out Northeastern. I could not be more thankful that I went there. You can read the freshman text book online: "How to Design Programs"[1]. [1] http://htdp.org/ http://htdp.org/
- relyio 9y ago
  I had a similar experience at Waterloo. The first semester of class we used Racket, HTDP (linked above) and even Haskell. I had already some experience with programming but it was like starting all over again to build solid foundations. If you are in the business of teaching introductory computer science material, I suggest you consider "How to design programs". Fun, good times. https://www.student.cs.uwaterloo.ca/~cs145/experienced.shtml https://www.student.cs.uwaterloo.ca/~cs145/experienced.shtml
- humanrebar 9y ago
  > "I know several people who have done exactly that, and it worked out quite well for them!" I've seen it done several times. It tends to work better in the short run than the long run compared to alternative approaches.
- discreteevent 9y ago
  The problem is that the several people he knows are not the average architecture astronaut. It takes a lot of taste and experience to design a usable language.
  - humanrebar 9y ago
    And it takes a lot of organizational dedication and work to support the language properly: training, documentation, packaging, regression testing (1), editor support, debugger support, libraries, and the list goes on. I doubt a single individual could do all that for more than a short while, so you need an organization that wants to do all that right for the sake of the language. Most companies won't really care that your language isn't in fedora upstream. (1) Lots of popular languages get the community to do their regression testing for them, but if you are the sole user of your language, you need to do all the testing yourself somehow.
mark_l_watson 9y ago
Great article. I have in the past (when I wanted to use Scheme) used either Gambit (great support from the author!) or Chez Scheme (fast!), and only have played around with Racket a bit (perhaps 50 hours in the last 5 years). I have been thinking of switching to Racket, and this article 'pushed me over the edge' on that decision.
trakam 9y ago
A side note: React and jQuery are not really complementary libraries dealing with separate concerns..as suggested by the article,unless i have fundamentally misunderstood both.
tambourine_man 9y ago
Typo in the headline: lanugage
skywhopper 9y ago
I’m curious how this compares with the similar built in tools for building grammars, etc in Perl 6, and also if the syntax flexibility possible is worth the trade off vs using more native (if obscured or downplayed) syntax eg the sorts of DSLs you can build with Ruby.
cornholio 9y ago
I wonder how this approach scales to large code bases maintained by global teams. Instead of the usual discovery that each contractor or "rockstar" programmer created modules using their preferred language or in vogue technology, you might now find they derived their own obscure language flavors that read right to left and use "=" for function calls. In large projects, readability and maintainability trump expressibility.
- cat199 9y ago
  > In large projects, readability and maintainability trump expressibility. and neither is a substitute for proper governance...
- yomly 9y ago
  Naughty Dog seem like a relatively large team. If you're thinking a megacorp tech team, there seems to be few examples of truly large successful Lisp projects, though it will be interesting to see how some of the growing Clojure companies get on in the next 5-10 years.
QuadrupleA 9y ago
I don't think I've seen any convincing examples yet of how writing a new language or DSL is superior to the already-very-flexible idea of user-defined functions, supported by almost all languages. Isn't a DSL just syntactic sugar in most cases? Functions are verbs, usually named in problem domain terms, that can operate on a huge variety of nouns (data) also namable in the problem domain. So I don't buy that existing languages can't represent the problem domain, or force programmers to only do things in machine or traditional-language terms. Don't want to be a spoilsport either, glad people are researching and experimenting with new approaches - just not sure I'm convinced yet the "make a DSL for everything" idea seems all that promising or different from what we already do as programmers.
- Volt 9y ago
  >Isn't a DSL just syntactic sugar in most cases? Maybe. But you probably have a programming language that you like to use when building Web apps, and another that you like to use when building command-line utilities. The differences aren't usually just the libraries available, but how easy it is to express certain solutions.
  - 8ytecoder 9y ago
    Nailed it. As you work longer on a project you'll realise patterns emerging. It's the same behavior repeated again and again. Take Rails for example, it is where it is because someone saw the patterns for a webapp and made a framework that conceptualizes it. It's something that's not appreciated very well. The concepts serve a vital purpose in organising your thinking in the problem domain.
    - hyperion2010 9y ago
      In a sense a DSL lets burried implicit knowledge become enshrined more formally (even if it is still implicit in a way) so that people don't keep making the same mistakes over and over because they are presented with tools where all the footguns haven't been safely dealt with. Consider how many people make the same mistakes over and over and over again because they go back to html, css, an js which give ZERO constraints on what one ought to do with them to make a useable website. Now imagine that we had been growing the language for writing websites so that new users literally could not write code that would footgun themselves into oblivion.
gwbas1c 9y ago
I can just imagine the learning curve when encountering legacy code in a language that's only used for the project, product, or company. I can't begin to explain how many times I have just Googled a question in a particular programming language and very quickly found an answer. When I work in new or unpopular languages, I often can't find an easy answer. The difference in time is staggering. The learning curve for a popular language means that learning something new takes me 5 minutes, but in an unknown language learning something you could take me all day if there are not a lot of things like stack Overflow posts. If a project specific custom language deviates too far from the norms, then it means that incoming programmers will have a harder time maintaining it.
- branchless 9y ago
  There is a third circle of hell: kdb. Unpopular and what is there is unsearchable as the syntax consists of items like ,:/. I like kdb and the docs are okay, but my word...
  - tluyben2 9y ago
    Not easy to get into indeed but I find it a pleasure with some experience. Personally I find k/q nicer to work with than say Ruby as used in Rails.
- Too 9y ago
  Having seen all kinds of abuse of C macros or python metaprogramming I can't even imagine the mess this could create. Some people just seem to love the idea of creating a undebuggable Frankenstein macro system just so they can save 3 characters of code when writing foo.bar instead of foo["bar"]. Python libraries depending on @ annotations everywhere is another favourite. You save 1 measly line of code by not having to call or register the function somewhere but you loose so much more as you can't control initialization order and your program usually is forced to store it's data as global state. I do see the powers and benefits this can give but with great power comes great responsibility. It must be truly isolated to places where you absolutely need it instead of just for fun.
bsenftner 9y ago
Wasn't "Script X" Apple's failed attempt at this back in the early 90's? People found creating custom for every problem was unmaintainable. Each developer was encouraged to customize their own personal language, and work with syntax translators to work with group written code. Nightmare.
- erikj 9y ago
  I presume you are talking about ScriptX created by an Apple spin-off Kaleida Labs? I'm reading about it right now, and I don't see any specific features useful for language-oriented programming, not even Lisp-like macros. From what I can see it doesn't seem to encourage this approach to software development at all, it looks like some kind of Smalltalk derivative with multiple inheritance bolted on.
antirez 9y ago
Tcl can do wonders in this area. uplevel is in some way more powerful than Lisp macros, since it happens at runtime.
- ken 9y ago
  Isn't "uplevel" essentially Tcl's way to choose dynamic scope? Lisp has ways to do that, too.
  - antirez 9y ago
    It's more like "eval" in the stack frame you select. This added to the fact Tcl, like Lisp, has the same form for all the commands, allow to do anything you could conceive and it will look like if it's part of the language. For instance you can do a looping construct that works in a different way depending on the calling function name or state or any other crazy stuff like that.
- kazinator 9y ago
  Uplevel and its ilk is related to the Lisp FEXPR and NLAMBDA mechanisms. Mainstream Lisp discarded these things decades ago; they didn't surive into ANSI standardization.
criddell 9y ago
This is offtopic, but seeing ACM made me think about my long-lapsed membership. Are you a member of the ACM? What do you get out of your membership?
dwarman 9y ago
Forgive me, but I recall, with some decades of practice, a major point in FORTH is that one constructs a vocabulary and syntax specifically for solving the problem at hand. Compiler primitives are just as first class as regular word (aka function) definitions, including how to compile itself as well as what to do when executed (DOES> keyword, which can itself be redefined). FORTH may have a clunky and primitive reputation, but it seems to me it qualifies as a programmable programming language.
- skrishnamurthi 9y ago
  Sure. But the degree of clunkiness matters. Forth doesn't have the ability to build linguistic _abstractions_ the way Racket does. But if you want to add Forth to the evidence of the paper's thesis, that's fine, more the better.

283 points by samth 9y ago

montrose 9y ago

"Lisp is a programmable programming language." — John Foderaro, CACM, September 1991.

matthias_f 9y ago
Yes our title intentionally alludes to CL.

nickpsecurity 9y ago

Language-oriented programming sounds like metaprogramming with DSL’s just with a new toolkit. Language-oriented programming might be a more approachable term for that, though. If I heard it, the first things I’d think of were tools such as Rascal and Ometa that let one arbitrarily define, operate on, or use languages. That covers language part. Far as integration, a number of techs supporting DSL’s… probably a lot of LISP’s and REBOL… had a powerful language underneath one could drop into. So, this seems like a new instance of old ideas they’re giving a new name. I do like how they’ve integrated the concept into a general-purpose, GUI-based, programming environment instead of it being it’s own thing like the others. You get best of both worlds. An old idea I had was researchers should do more experiments in building things like Rascal or Ometa alternatives in Racket leveraging what they already have to see how far one can stretch and improve it. http://www.rascal-mpl.org/ http://www.rascal-mpl.org/ https://en.wikipedia.org/wiki/OMeta https://en.wikipedia.org/wiki/OMeta

stevekemp 9y ago
Indeed, though I admit my first thought when reading things like this is of FORTH rather than Lisp. I think there's a lot of value in DSL and implementing them in some languages is a lot easier than others.
gfodor 9y ago
Language oriented programming is far from a new term. See the history of jetbrains MPS and intentional software. There was a strong meme in the early 2000's this was the future, it never seemed to take off.
- username223 9y ago
  I even bought into that meme a bit in the early 2000s. It turns out that most "language" designers (including myself) aren't that good at it, and that designing a language that isn't just yet another leaky abstraction takes a lot of work. Your "interface" is the entire syntax and semantics of your "language."
  - stevedonovan 9y ago
    Yes - there is definitely a hierarchy of difficulty: user code, library, macros, full syntax extension. The trouble is that designing new languages takes a lot of _taste_ and experience. We don't know the full domain, so DSLs tend to have bits added to them as knowledge of the domain grows - and that's the scary point in language evolution. So, despite these things being tremendous fun, I tend more towards correctness and clarity at the expense of sheer expressiveness. (An example of ad-hoc language design is CMake, which makes my eyes bleed. Kitware should have used an existing language - even tcl would have been better.)

samdg 9y ago

Title should be edited: Lanugage -> Language.

sctb 9y ago
Thanks! Updated.

hyperion2010 9y ago

I have been building a new not-exactly-a-programming-language using Racket and it has been an absolute pleasure, and the community has been extremely friendly and helpful. One existing stumbling block that I encountered on the way is that I could not find documentation for the full process for the setup for creating a new packaged language (I have been known to fail hard at finding the right documents so this may just be my problem). https://beautifulracket.com/ https://beautifulracket.com/ has excellent coverage of the general process, but the differences between #lang br and #lang racket/base are just large enough that I couldn't fill in the gaps without going and looking through multiple repos to see how others had implemented their repo layout and then could intuit which setup commands I needed to run. If I find time I intend to write up my findings and submit them to the documentation, the short version is here in case someone finds it useful. repo-base ├── .git ├── my-lang ; run `raco pkg install` here │ └── info.rkt ; meta package depends on my-lang-lib └── my-lang-lib ; run `raco pkg install` here ├── info.rkt ; include actual dependencies └── my-lang ; implementation goes here └── main.rkt ; in theory you can put your whole implementation here Once that set up is complete you should be able to use `#lang my-lang`.

araes 9y ago
Thank you for the link. Reading the example on the creation of stacker.rkt (a toy reverse polish language) was dramatically more enlightening than the main article. As I understand, Racket = source-to-source interpreter with a lot of helpful shorthand for defining syntax of {insert language here} or arbitrary language you make on the spot. Do they already have libraries for major languages like c++, JS, python? Would be nice to write in a syntax I like (perl) and turn it into c++ that compiled.
- hyperion2010 9y ago
  Racket isn't really a source to source compiler/interpreter (you could try to write a compiler that converted racket into your language of choice, but that can often be quite difficult). Everything ultimately has to pass through Racket's language semantics, so matching implementation details requires a lot of work if the underlying language spec is large. The #lang C example down below sidesteps these issues because it acts as a pass through to the system c compiler and then handles returning those results into racket, essentially it delegates the language semantics to the c compiler and only deals with how to map the results of calling c code back onto racket objects. Similar issue with the FFI if you want to translate from one language to another, you still have to go through the process of mapping language semantics. Two examples where languages/parsers have been implemented in racket: Algol60 is the best and most complete example https://docs.racket-lang.org/algol60/index.html https://docs.racket-lang.org/algol60/index.html. A python parser has been implemented https://github.com/dyoo/ragg/blob/master/ragg/examples/python-grammar.rkt https://github.com/dyoo/ragg/blob/master/ragg/examples/pytho...

yogthos 9y ago

Instead of having to think about every possible way a language might be used and account for it up front, languages with good metaprogramming facilities let the users them to fit their needs. This way the core language can stay small and focused while remaining flexible. While you don’t want everybody building their own language, userland extensibility can play a huge impact on the overall cleanliness of the language because new ideas can be tried out as libraries. If these ideas don’t pan out, the community moves on. Projects using these libraries will keep working, but the core language doesn’t end up accumulating cruft. I wrote about that in detail here: https://yogthos.net/posts/2017-10-03-MovingForwardByLettingGo.html https://yogthos.net/posts/2017-10-03-MovingForwardByLettingG...

mcguire 9y ago
...which is why Scheme has, like, 37 different, incompatible object systems. Everything has costs.
- eadmund 9y ago
  That's Scheme. Lisp, meanwhile, has one: CLOS. One is excellent for learning about things like hand-rolled object systems, and one is excellent for building large systems.
  - iak8god 9y ago
    Common Lisp has CLOS. I'm aware some purists don't think Scheme is a Lisp but, regardless of one's stance on this, there are most definitely other Lisps than Common Lisp.
    - YouAreGreat 9y ago
      Having called the result of Lisp standardization ANSI Common Lisp instead of just ANSI Lisp doesn't change the fact that the whole effort was all about standardizing Lisp. Moreover, why not just say Scheme or Clojure when talking about Scheme or Clojure? If ANSI C had been called ANSI Common C, would Java fans never cease to insist on calling Java a dialect of C?

MaxBarraclough 9y ago

> Racket eliminates the hard boundary between library and language, overcoming a seemingly intractable conflict I don't buy it. A language extension means the syntax wouldn't ever be valid in the base language. The fact that Racket defines its 'for' construct using macros/functions/whatever is cute, but I see no reason to pretend it's a language extension.

soegaard 9y ago
> A language extension means the syntax wouldn't ever be > valid in the base language. Well, this works: #lang base-without-for (require the-for-library) ...use for here... Of course if the author of `for` has made a `base-with-for` language, then I can just use: #lang base-with-for ...use for here...
shakna 9y ago
> I don't buy it. > A language extension means the syntax wouldn't ever be valid in the base language. Sure. (struct pt ([x : Real] [y : Real])) That's not valid Racket, but is valid Typed Racket. A more extreme example: int main(int argc, char** argv) { return 0; } That's C! Oh wait, it's the C #lang [0]... Which means you can things like: base.rkt: #lang C float f( float x1, float x2 ) { return x1 / x2; } main.rkt: #lang racket (require base) f(10.2, 1.8) Those syntaxes aren't compatible, by any stretch. The magic is, Racket gives you access to powerful pre-processing, in the form of some amazing reader macros. What syntax you use, doesn't really matter, so long as you provide that bridge. They're incompatible syntax, with a custom made parser allowing data to flow both ways. [0] https://github.com/jeapostrophe/c https://github.com/jeapostrophe/c
- mcguire 9y ago
  Shouldn't that second example be (f 10.2 1.8)
  - shakna 9y ago
    Yep. My bad.

zitterbewegung 9y ago

Racket is awesome. I originally used scheme48 but was impressed with rackets tooling (when it was known as plt scheme) . After it branched out as racket and added a large standard library with batteries included it became much more practical to use. I am experimenting with using it with smart contracts and having a library where you can make your own language and work with program synthesis to web development it’s really awesome!

apeace 9y ago

I recall going to a talk when I was at Northeastern, and at one point the speaker said something like, "But why would you create a new programming language just to solve one problem?" Matthias Felleisen (the guy in the video) interrupted him from the audience to respond, "I know several people who have done exactly that, and it worked out quite well for them!" As he says in the video, Northeastern teaches students how to program in a systematic way, rather than by copy-paste-modify from other examples. They certainly teach a powerful way of thinking, and it has served me well over the years. If anyone out there is reading this and is thinking about going to college for computer science, definitely check out Northeastern. I could not be more thankful that I went there. You can read the freshman text book online: "How to Design Programs"[1]. [1] http://htdp.org/ http://htdp.org/

relyio 9y ago
I had a similar experience at Waterloo. The first semester of class we used Racket, HTDP (linked above) and even Haskell. I had already some experience with programming but it was like starting all over again to build solid foundations. If you are in the business of teaching introductory computer science material, I suggest you consider "How to design programs". Fun, good times. https://www.student.cs.uwaterloo.ca/~cs145/experienced.shtml https://www.student.cs.uwaterloo.ca/~cs145/experienced.shtml
humanrebar 9y ago
> "I know several people who have done exactly that, and it worked out quite well for them!" I've seen it done several times. It tends to work better in the short run than the long run compared to alternative approaches.
discreteevent 9y ago
The problem is that the several people he knows are not the average architecture astronaut. It takes a lot of taste and experience to design a usable language.
- humanrebar 9y ago
  And it takes a lot of organizational dedication and work to support the language properly: training, documentation, packaging, regression testing (1), editor support, debugger support, libraries, and the list goes on. I doubt a single individual could do all that for more than a short while, so you need an organization that wants to do all that right for the sake of the language. Most companies won't really care that your language isn't in fedora upstream. (1) Lots of popular languages get the community to do their regression testing for them, but if you are the sole user of your language, you need to do all the testing yourself somehow.

mark_l_watson 9y ago

Great article. I have in the past (when I wanted to use Scheme) used either Gambit (great support from the author!) or Chez Scheme (fast!), and only have played around with Racket a bit (perhaps 50 hours in the last 5 years). I have been thinking of switching to Racket, and this article 'pushed me over the edge' on that decision.

trakam 9y ago

A side note: React and jQuery are not really complementary libraries dealing with separate concerns..as suggested by the article,unless i have fundamentally misunderstood both.

tambourine_man 9y ago

Typo in the headline: lanugage

skywhopper 9y ago

I’m curious how this compares with the similar built in tools for building grammars, etc in Perl 6, and also if the syntax flexibility possible is worth the trade off vs using more native (if obscured or downplayed) syntax eg the sorts of DSLs you can build with Ruby.

cornholio 9y ago

I wonder how this approach scales to large code bases maintained by global teams. Instead of the usual discovery that each contractor or "rockstar" programmer created modules using their preferred language or in vogue technology, you might now find they derived their own obscure language flavors that read right to left and use "=" for function calls. In large projects, readability and maintainability trump expressibility.

cat199 9y ago
> In large projects, readability and maintainability trump expressibility. and neither is a substitute for proper governance...
yomly 9y ago
Naughty Dog seem like a relatively large team. If you're thinking a megacorp tech team, there seems to be few examples of truly large successful Lisp projects, though it will be interesting to see how some of the growing Clojure companies get on in the next 5-10 years.

QuadrupleA 9y ago

I don't think I've seen any convincing examples yet of how writing a new language or DSL is superior to the already-very-flexible idea of user-defined functions, supported by almost all languages. Isn't a DSL just syntactic sugar in most cases? Functions are verbs, usually named in problem domain terms, that can operate on a huge variety of nouns (data) also namable in the problem domain. So I don't buy that existing languages can't represent the problem domain, or force programmers to only do things in machine or traditional-language terms. Don't want to be a spoilsport either, glad people are researching and experimenting with new approaches - just not sure I'm convinced yet the "make a DSL for everything" idea seems all that promising or different from what we already do as programmers.

Volt 9y ago
>Isn't a DSL just syntactic sugar in most cases? Maybe. But you probably have a programming language that you like to use when building Web apps, and another that you like to use when building command-line utilities. The differences aren't usually just the libraries available, but how easy it is to express certain solutions.
- 8ytecoder 9y ago
  Nailed it. As you work longer on a project you'll realise patterns emerging. It's the same behavior repeated again and again. Take Rails for example, it is where it is because someone saw the patterns for a webapp and made a framework that conceptualizes it. It's something that's not appreciated very well. The concepts serve a vital purpose in organising your thinking in the problem domain.
  - hyperion2010 9y ago
    In a sense a DSL lets burried implicit knowledge become enshrined more formally (even if it is still implicit in a way) so that people don't keep making the same mistakes over and over because they are presented with tools where all the footguns haven't been safely dealt with. Consider how many people make the same mistakes over and over and over again because they go back to html, css, an js which give ZERO constraints on what one ought to do with them to make a useable website. Now imagine that we had been growing the language for writing websites so that new users literally could not write code that would footgun themselves into oblivion.

gwbas1c 9y ago

I can just imagine the learning curve when encountering legacy code in a language that's only used for the project, product, or company. I can't begin to explain how many times I have just Googled a question in a particular programming language and very quickly found an answer. When I work in new or unpopular languages, I often can't find an easy answer. The difference in time is staggering. The learning curve for a popular language means that learning something new takes me 5 minutes, but in an unknown language learning something you could take me all day if there are not a lot of things like stack Overflow posts. If a project specific custom language deviates too far from the norms, then it means that incoming programmers will have a harder time maintaining it.

branchless 9y ago
There is a third circle of hell: kdb. Unpopular and what is there is unsearchable as the syntax consists of items like ,:/. I like kdb and the docs are okay, but my word...
- tluyben2 9y ago
  Not easy to get into indeed but I find it a pleasure with some experience. Personally I find k/q nicer to work with than say Ruby as used in Rails.
Too 9y ago
Having seen all kinds of abuse of C macros or python metaprogramming I can't even imagine the mess this could create. Some people just seem to love the idea of creating a undebuggable Frankenstein macro system just so they can save 3 characters of code when writing foo.bar instead of foo["bar"]. Python libraries depending on @ annotations everywhere is another favourite. You save 1 measly line of code by not having to call or register the function somewhere but you loose so much more as you can't control initialization order and your program usually is forced to store it's data as global state. I do see the powers and benefits this can give but with great power comes great responsibility. It must be truly isolated to places where you absolutely need it instead of just for fun.

bsenftner 9y ago

Wasn't "Script X" Apple's failed attempt at this back in the early 90's? People found creating custom for every problem was unmaintainable. Each developer was encouraged to customize their own personal language, and work with syntax translators to work with group written code. Nightmare.

erikj 9y ago
I presume you are talking about ScriptX created by an Apple spin-off Kaleida Labs? I'm reading about it right now, and I don't see any specific features useful for language-oriented programming, not even Lisp-like macros. From what I can see it doesn't seem to encourage this approach to software development at all, it looks like some kind of Smalltalk derivative with multiple inheritance bolted on.

antirez 9y ago

Tcl can do wonders in this area. uplevel is in some way more powerful than Lisp macros, since it happens at runtime.

ken 9y ago
Isn't "uplevel" essentially Tcl's way to choose dynamic scope? Lisp has ways to do that, too.
- antirez 9y ago
  It's more like "eval" in the stack frame you select. This added to the fact Tcl, like Lisp, has the same form for all the commands, allow to do anything you could conceive and it will look like if it's part of the language. For instance you can do a looping construct that works in a different way depending on the calling function name or state or any other crazy stuff like that.
kazinator 9y ago
Uplevel and its ilk is related to the Lisp FEXPR and NLAMBDA mechanisms. Mainstream Lisp discarded these things decades ago; they didn't surive into ANSI standardization.

criddell 9y ago

This is offtopic, but seeing ACM made me think about my long-lapsed membership. Are you a member of the ACM? What do you get out of your membership?

dwarman 9y ago

Forgive me, but I recall, with some decades of practice, a major point in FORTH is that one constructs a vocabulary and syntax specifically for solving the problem at hand. Compiler primitives are just as first class as regular word (aka function) definitions, including how to compile itself as well as what to do when executed (DOES> keyword, which can itself be redefined). FORTH may have a clunky and primitive reputation, but it seems to me it qualifies as a programmable programming language.

skrishnamurthi 9y ago
Sure. But the degree of clunkiness matters. Forth doesn't have the ability to build linguistic _abstractions_ the way Racket does. But if you want to add Forth to the evidence of the paper's thesis, that's fine, more the better.