7 ms·

Shen: A Sufficiently Advanced Lisp [video] (2014)

112 points by sea6ear 12y ago

codygman 12y ago
Put this in my "watch later" queue. I'm very interested in a Lisp whose type system is on par with that of Haskell/Ocaml. EDIT: Especially since Shen is now BSD licensed.
- AnimalMuppet 12y ago
  To me, it seems impossible to have both macros and a type system on par with Haskell. Either your macros can no longer do arbitrary textural transformations, or your type system can't reason about macro cases. (Or, I suppose, your type system has to run after the macro transformations, and you could get a type error then just like you could get a syntax error then. But that means your IDE/development environment can't give you any type assistance on macro calls - you have to compile it to find out if it works. (Or, I suppose, your IDE has to run the macro for you and then do the type checking on the resulting post-macro code.)) I am open to being proven wrong, though...
  - doublec 12y ago
    ATS had macros: http://ats-lang.sourceforge.net/htdocs-old/TUTORIAL/contents/macros.html http://ats-lang.sourceforge.net/htdocs-old/TUTORIAL/contents... I'm not sure if these made it into ATS2 though.
  - necubi 12y ago
    Scala has macros [0], and as you reasoned the type checker has to run after the macro expansion. That's not a huge downside; to the user it still appears to be a single pipeline (parsing -> macro expansion -> type checking -> compilation). Haskell also has very powerful metaprogramming facilities, like template haskell [1]. [0] http://scalamacros.org/ http://scalamacros.org/ [1] https://wiki.haskell.org/Template_Haskell https://wiki.haskell.org/Template_Haskell
  - sordina 12y ago
    > it seems impossible to have both macros and a type system on par with Haskell You should check out the work being done on typed template Haskell then.
  - andrewchambers 12y ago
    You can have a language which has to be parsed top down, then you can have both complete type info, and arbitrary macros.
  - deech 12y ago
    The second half of the talk (http://www.youtube.com/watch?v=lMcRBdSdO_U&t=17m30s http://www.youtube.com/watch?v=lMcRBdSdO_U&t=17m30s) is about turning a JSON object into a type using macros and meta-programming.
  - gmfawcett 12y ago
    MetaOcaml's type-safe staged metaprogramming is not strictly a macro system, but it is morally equivalent: http://okmij.org/ftp/ML/MetaOCaml.html http://okmij.org/ftp/ML/MetaOCaml.html
- 616c 12y ago
  Now, I understand that is BSD licensed, but I am bothered by their proud "we support OSI, but never GPL" attitude. I wonder what happened to the original developer, but does anyone know why he has GPL issues? Yes, I know he is not unique in the industry at large, but I rarely see project download pages with GPL with a red line through it. http://www.shenlanguage.org/download_form.html http://www.shenlanguage.org/download_form.html
  - doublec 12y ago
    The original developers thoughts on GPL have been discussed at length on the Shen mailing list during the BSD licensing donation period. If you look through the archives you'll be able to read their thoughts. This link may be a start: https://groups.google.com/d/msg/qilang/mVSJIyp-OhM/FjcAOAWUio0J https://groups.google.com/d/msg/qilang/mVSJIyp-OhM/FjcAOAWUi... It's unfortunate that the original authors views on licensing takes so much discussion away from Shen itself as it's an interesting language.
    - 616c 12y ago
      So true. I am watching this video and I am very impressed. I am a novice programmer, but from what I read on HN the Shen system hits all cylinders a lot of the more powerful advanced programming paradigms. I am a Shen outsider and there seems to be some much overhead that is not technical when I read about it and it upsets me bc, well, it is a work of art. I just started studying Java, and found the yet to be certified Java implementation. I am afraid to look at it, lest brains come of my nose. https://github.com/hraberg/Shen.java https://github.com/hraberg/Shen.java
      - tluyben2 12y ago
        One of the great advantages of Shen is that it is built on top of a very simple Lisp called KL of which you can (at least basically working) implement an interpreter or compiler in any language in a day. And when you did that, Shen will run on it. That's why there are so many targets and that's why most targets are not mature. I find it a great advantage though; you can make a compiler or runtime embedded in whatever you are doing really fast and then optimise it as you go. Edit: I would say, if you want to practice on implementing a compiler & runtime (or VM with JIT or LLVM frontend etc) KL is a good place to start. It's so easy (again, to get something working; to optimise etc is obviously just as painful as with any other language implementation) that it is a lot of fun and you learn a lot if you never did it before.
alvatar 12y ago
Once again, types are way overrated. However, Shen is an interesting new Lisp to watch. Needs a more open community, though (like a free book and/or deep documentation).
- codygman 12y ago
  > Once again, types are way overrated. Care to expand on this? I'm curious, what is your opinion and preferred platform/languages?
  - tormeh 12y ago
    I'm not the parent, but in noncritical applications an accommodating compiler/interpreter will speed up development. Few advocate such an attitude for critical applications, but whether all compile-time checking should be done in terms of types or not is an open question.
    - nothrabannosir 12y ago
      Types speed up my development: they obviate a certain kind of unit tests. If you weren't going to write those unit tests without the types, anyway, then it helps because you now catch type mismatches at compile time instead of runtime.
      - tormeh 12y ago
        The question is whether types are the most productive area to focus on when developing better compile-time checks. Maybe it would be better to design languages that solve problems by other means.
        tluyben2 12y ago
        With Shen you can use them when you want. So you can create things fast and add types / tests / proofs when you want and when it seems prudent.
    - 12y ago
mordocai 12y ago
Unfortunately I found shen a pain to get into due to it still being mostly a teaching tool. However, if the community builds some nice tooling around it I will definitely give it another look.
- tluyben2 12y ago
  Working on it!
616c 12y ago
Also interesting and worthy of note is that this project transitioned to BSD recently, and they are doing it as part of a donation drive. http://www.shenlanguage.org/shenbsd.htm http://www.shenlanguage.org/shenbsd.htm I am an amateur Lisper, and like others I whined (perhaps irrationally) that their custom non-OSI only one implementation of the language or else license was scaring people off. Not that the BSD license assuages my concerns about community weirdness (hard to describe, but this ia a very personal opinion and you can dismiss it), but I would like to see if the demand will drive open sourcing the code. Also very cool is that you can find Shen implementations in more than one flavor: Common Lisp, Clojure, Ruby, and more. Definitely check out the site if you have not yet.
- lomnakkus 12y ago
  That was a really weird "episode" in the history of Shen, IMO. I mean, the outcome was good and all, but it was really weird to ask for donations to change the license. I don't think there was any real explanation for that other that "we want money"?
  - doublec 12y ago
    I believe it came from a discussion where to draft a new license would require the original developer to pay a lawyer. Someone suggested raising money to pay the developer the money to transition to BSD, a known license, instead of to a lawyer for yet another unknown license.
    - lomnakkus 12y ago
      Wow, that's even weirder than I imagined. Regardless, I'm glad it's been freed due to the generosity of some individuals :)
juliangamble 12y ago
Shen stands above other languages (apart from Idris and Coq) in the area of dependent types.
- doublec 12y ago
  I haven't looked at the ability to have dependent types in Shen - it'd be great to see some examples. A quick search on the mailing list found: https://groups.google.com/d/msg/qilang/lokvvf0xLJc/20y8ZkzoNH4J https://groups.google.com/d/msg/qilang/lokvvf0xLJc/20y8ZkzoN... https://groups.google.com/d/msg/qilang/D3tmhQvfnOE/qcsXhRomFb0J https://groups.google.com/d/msg/qilang/D3tmhQvfnOE/qcsXhRomF...
- nullc 12y ago
  and apart from ATS.
dgreensp 12y ago
It's pretty strange to see a "Lisp" with so much syntax. Lots of infix notation, and even semicolons! Usually Lisps have a very simple reader that reads S-expression and a few other things, but here we have this: (define map { (A --> B) --> (list A) --> (list B) } F [] -> [] F [ X | XS ] -> [(F X) | (map F XS)]) And this is one of the cases without any semicolons. Shen also uses the atrocious "curried form" of function types, where the function that adds two numbers has type number->number->number, and the reader must do the mental gymnastics to convert it to (number,number)->number (and good luck when it's more complicated than that). I wonder if you took a language like Haskell and changed some of the syntax to look Lispy if it could masquerade as a Lisp? I'm not saying that's what's happening, it's just hard to wrap my head around a syntax like this that claims to be Lispy.
- deech 12y ago
  You don't have to use any of the syntax if you don't want to. You can write directly in KLambda. Here's how `map` is defined in KLambda: (defun map (f l) (shen.map-h f l ())) (defun shen.map-h (f l accum) (cond ((= () l) (reverse accum)) ((cons? l) (shen.map-h f (tl l) (cons (f (hd l)) accum))) (true (shen.f_error shen.map-h))))
- typedweb 12y ago
  http://readwrite.com/2010/11/27/haskell-with-lisp-syntax?_escaped_fragment_=# http://readwrite.com/2010/11/27/haskell-with-lisp-syntax?_es...!

112 points by sea6ear 12y ago

Put this in my "watch later" queue. I'm very interested in a Lisp whose type system is on par with that of Haskell/Ocaml. EDIT: Especially since Shen is now BSD licensed.

AnimalMuppet 12y ago
To me, it seems impossible to have both macros and a type system on par with Haskell. Either your macros can no longer do arbitrary textural transformations, or your type system can't reason about macro cases. (Or, I suppose, your type system has to run after the macro transformations, and you could get a type error then just like you could get a syntax error then. But that means your IDE/development environment can't give you any type assistance on macro calls - you have to compile it to find out if it works. (Or, I suppose, your IDE has to run the macro for you and then do the type checking on the resulting post-macro code.)) I am open to being proven wrong, though...
- doublec 12y ago
  ATS had macros: http://ats-lang.sourceforge.net/htdocs-old/TUTORIAL/contents/macros.html http://ats-lang.sourceforge.net/htdocs-old/TUTORIAL/contents... I'm not sure if these made it into ATS2 though.
- necubi 12y ago
  Scala has macros [0], and as you reasoned the type checker has to run after the macro expansion. That's not a huge downside; to the user it still appears to be a single pipeline (parsing -> macro expansion -> type checking -> compilation). Haskell also has very powerful metaprogramming facilities, like template haskell [1]. [0] http://scalamacros.org/ http://scalamacros.org/ [1] https://wiki.haskell.org/Template_Haskell https://wiki.haskell.org/Template_Haskell
- sordina 12y ago
  > it seems impossible to have both macros and a type system on par with Haskell You should check out the work being done on typed template Haskell then.
- andrewchambers 12y ago
  You can have a language which has to be parsed top down, then you can have both complete type info, and arbitrary macros.
- deech 12y ago
  The second half of the talk (http://www.youtube.com/watch?v=lMcRBdSdO_U&t=17m30s http://www.youtube.com/watch?v=lMcRBdSdO_U&t=17m30s) is about turning a JSON object into a type using macros and meta-programming.
- gmfawcett 12y ago
  MetaOcaml's type-safe staged metaprogramming is not strictly a macro system, but it is morally equivalent: http://okmij.org/ftp/ML/MetaOCaml.html http://okmij.org/ftp/ML/MetaOCaml.html
616c 12y ago
Now, I understand that is BSD licensed, but I am bothered by their proud "we support OSI, but never GPL" attitude. I wonder what happened to the original developer, but does anyone know why he has GPL issues? Yes, I know he is not unique in the industry at large, but I rarely see project download pages with GPL with a red line through it. http://www.shenlanguage.org/download_form.html http://www.shenlanguage.org/download_form.html
- doublec 12y ago
  The original developers thoughts on GPL have been discussed at length on the Shen mailing list during the BSD licensing donation period. If you look through the archives you'll be able to read their thoughts. This link may be a start: https://groups.google.com/d/msg/qilang/mVSJIyp-OhM/FjcAOAWUio0J https://groups.google.com/d/msg/qilang/mVSJIyp-OhM/FjcAOAWUi... It's unfortunate that the original authors views on licensing takes so much discussion away from Shen itself as it's an interesting language.
  - 616c 12y ago
    So true. I am watching this video and I am very impressed. I am a novice programmer, but from what I read on HN the Shen system hits all cylinders a lot of the more powerful advanced programming paradigms. I am a Shen outsider and there seems to be some much overhead that is not technical when I read about it and it upsets me bc, well, it is a work of art. I just started studying Java, and found the yet to be certified Java implementation. I am afraid to look at it, lest brains come of my nose. https://github.com/hraberg/Shen.java https://github.com/hraberg/Shen.java
    - tluyben2 12y ago
      One of the great advantages of Shen is that it is built on top of a very simple Lisp called KL of which you can (at least basically working) implement an interpreter or compiler in any language in a day. And when you did that, Shen will run on it. That's why there are so many targets and that's why most targets are not mature. I find it a great advantage though; you can make a compiler or runtime embedded in whatever you are doing really fast and then optimise it as you go. Edit: I would say, if you want to practice on implementing a compiler & runtime (or VM with JIT or LLVM frontend etc) KL is a good place to start. It's so easy (again, to get something working; to optimise etc is obviously just as painful as with any other language implementation) that it is a lot of fun and you learn a lot if you never did it before.

alvatar 12y ago

Once again, types are way overrated. However, Shen is an interesting new Lisp to watch. Needs a more open community, though (like a free book and/or deep documentation).

codygman 12y ago
> Once again, types are way overrated. Care to expand on this? I'm curious, what is your opinion and preferred platform/languages?
- tormeh 12y ago
  I'm not the parent, but in noncritical applications an accommodating compiler/interpreter will speed up development. Few advocate such an attitude for critical applications, but whether all compile-time checking should be done in terms of types or not is an open question.
  - nothrabannosir 12y ago
    Types speed up my development: they obviate a certain kind of unit tests. If you weren't going to write those unit tests without the types, anyway, then it helps because you now catch type mismatches at compile time instead of runtime.
    - tormeh 12y ago
      The question is whether types are the most productive area to focus on when developing better compile-time checks. Maybe it would be better to design languages that solve problems by other means.
      - tluyben2 12y ago
        With Shen you can use them when you want. So you can create things fast and add types / tests / proofs when you want and when it seems prudent.
  - 12y ago

mordocai 12y ago

Unfortunately I found shen a pain to get into due to it still being mostly a teaching tool. However, if the community builds some nice tooling around it I will definitely give it another look.

tluyben2 12y ago
Working on it!

616c 12y ago

Also interesting and worthy of note is that this project transitioned to BSD recently, and they are doing it as part of a donation drive. http://www.shenlanguage.org/shenbsd.htm http://www.shenlanguage.org/shenbsd.htm I am an amateur Lisper, and like others I whined (perhaps irrationally) that their custom non-OSI only one implementation of the language or else license was scaring people off. Not that the BSD license assuages my concerns about community weirdness (hard to describe, but this ia a very personal opinion and you can dismiss it), but I would like to see if the demand will drive open sourcing the code. Also very cool is that you can find Shen implementations in more than one flavor: Common Lisp, Clojure, Ruby, and more. Definitely check out the site if you have not yet.

lomnakkus 12y ago
That was a really weird "episode" in the history of Shen, IMO. I mean, the outcome was good and all, but it was really weird to ask for donations to change the license. I don't think there was any real explanation for that other that "we want money"?
- doublec 12y ago
  I believe it came from a discussion where to draft a new license would require the original developer to pay a lawyer. Someone suggested raising money to pay the developer the money to transition to BSD, a known license, instead of to a lawyer for yet another unknown license.
  - lomnakkus 12y ago
    Wow, that's even weirder than I imagined. Regardless, I'm glad it's been freed due to the generosity of some individuals :)

juliangamble 12y ago

Shen stands above other languages (apart from Idris and Coq) in the area of dependent types.

doublec 12y ago
I haven't looked at the ability to have dependent types in Shen - it'd be great to see some examples. A quick search on the mailing list found: https://groups.google.com/d/msg/qilang/lokvvf0xLJc/20y8ZkzoNH4J https://groups.google.com/d/msg/qilang/lokvvf0xLJc/20y8ZkzoN... https://groups.google.com/d/msg/qilang/D3tmhQvfnOE/qcsXhRomFb0J https://groups.google.com/d/msg/qilang/D3tmhQvfnOE/qcsXhRomF...
nullc 12y ago
and apart from ATS.

dgreensp 12y ago

It's pretty strange to see a "Lisp" with so much syntax. Lots of infix notation, and even semicolons! Usually Lisps have a very simple reader that reads S-expression and a few other things, but here we have this: (define map { (A --> B) --> (list A) --> (list B) } F [] -> [] F [ X | XS ] -> [(F X) | (map F XS)]) And this is one of the cases without any semicolons. Shen also uses the atrocious "curried form" of function types, where the function that adds two numbers has type number->number->number, and the reader must do the mental gymnastics to convert it to (number,number)->number (and good luck when it's more complicated than that). I wonder if you took a language like Haskell and changed some of the syntax to look Lispy if it could masquerade as a Lisp? I'm not saying that's what's happening, it's just hard to wrap my head around a syntax like this that claims to be Lispy.

deech 12y ago
You don't have to use any of the syntax if you don't want to. You can write directly in KLambda. Here's how `map` is defined in KLambda: (defun map (f l) (shen.map-h f l ())) (defun shen.map-h (f l accum) (cond ((= () l) (reverse accum)) ((cons? l) (shen.map-h f (tl l) (cons (f (hd l)) accum))) (true (shen.f_error shen.map-h))))
typedweb 12y ago
http://readwrite.com/2010/11/27/haskell-with-lisp-syntax?_escaped_fragment_=# http://readwrite.com/2010/11/27/haskell-with-lisp-syntax?_es...!