8 ms·

Lispsyntax.jl: A Clojure-like Lisp syntax for julia

162 points by phonebucket 6y ago

degraafc 6y ago
Great example of building something fun not because we should but because we can :) My take on "Lisp in Julia" using deprecated-but-still-parseable syntax instead of a string macro: https://github.com/christopher-dG/jlisp https://github.com/christopher-dG/jlisp I should probably put some examples in the readme.
- canadianfella 6y ago
  Examples would help.
- cbkeller 6y ago
  That hidden Julia s-expression syntax helped me start to understand metaprogramming in Julia. Along the way I found that it takes surprisingly few lines of code to turn the normal Julia repl into a more-or-less working Julia-lisp repl: https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc765466f749ba https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc76...
ampdepolymerase 6y ago
The irony here is particularly rich considering Julia's parser frontend is written in Scheme.
- yrimaxi 6y ago
  Why is that ironic?
  - DannyB2 6y ago
    At the very least it is amusing.
  - eigenspace 6y ago
    I'm not sure, but maybe they're unhappy that it uses a more Clojure style syntax?
- sritchie 6y ago
  A couple years into fatherhood and I see puns everywhere, and suspect others of dropping them too. Particularly Rich... Hickey?
  - asimjalis 6y ago
    I juxt ignore the puns.
    - deleted 6y ago
      [deleted]
  - canadianfella 6y ago
    Where did you steele that joke from?
- mumblemumble 6y ago
  It seems more fitting than ironic to me. Julia's never been shy about its lisp heritage, and the reason why this project is only a couple hundred lines of code is that Julia remains very close to those roots.
  - Blikkentrekker 6y ago
    Certainly any alternative syntax parser for any language would be? It would also only take a couple of hundred lines to write a parser for a C compiler that would allow that.
    - mumblemumble 6y ago
      Not any to any. You could't write a thin lispy alternative syntax for C because of some semantic differences. "If" is an expression in Julia and Lisp, and it's a statement in C. Dealing with things like goto statements and preprocessor macros would also require some extra work. Add all that up, and you're no longer looking at alternative syntax, you're looking at a transpiler.
      - Blikkentrekker 6y ago
        But this is also simply an alternative Syntax that mirrors Clojure, not actual Clojure semantics.
        eigenspace 6y ago
        While julia does not have full lisp semantics, it is very close. Granted, it's not especially similar to Clojure itself. It's closer to a lisp though than almost any other relatively popular modern language that doesn't use S-expressions.
        Blikkentrekker 6y ago
        I'm very sceptical of such claims because I do not believe there is such a thing as “Lisp semantics” and that it is primarily a language of syntax. In particular, Common Lisp has many semantic features such as dynamic variables and the function–value dichotomy that do not exist in, say Scheme, or really any other commonly used programming language. And Scheme, yet again, has many rather esoteric operations, such as continuation capturing, that exists nowhere else as well. The three most popular Lisps today are C.L., Clojure, and Scheme, and I don't see these three having unified semantics at all and they are often further from each other than they are to many other languages.
fulafel 6y ago
My current favourite of these is this Lua version: https://github.com/bakpakin/Fennel https://github.com/bakpakin/Fennel
phonebucket 6y ago
A fun Julia easter egg I recently discovered. Running 'julia --lisp' launches a femtolisp (https://github.com/JeffBezanson/femtolisp https://github.com/JeffBezanson/femtolisp) interpreter.
- retzkek 6y ago
  The Julia parser is actually written in femtolisp: https://docs.julialang.org/en/v1/devdocs/eval/#dev-parsing https://docs.julialang.org/en/v1/devdocs/eval/#dev-parsing https://github.com/JuliaLang/julia/blob/master/src/julia-parser.scm https://github.com/JuliaLang/julia/blob/master/src/julia-par...
ddragon 6y ago
While not nearly as mature as the main platforms (like the JVM, .NET, BEAM), the Julia compiler does have some unique features (an extremely aggressive optimizer, a sophisticated type inference mechanism, possibly the most exceptional multiple dispatch support, Go-like lightweight processes), including a macro system that supports reader macros for writing parsers (like this library) and extensible multi-stage compilation (like the IR manipulation that is used in auto-differentiation of source code), which makes me wonder if once all those features (and possibly new ones) become robust enough it will become one of the targets for people creating new programming languages.
- stabbles 6y ago
  What you see most is domain-specific languages implemented through macro's. For instance this package, https://jump.dev/JuMP.jl/dev/examples/basic/ https://jump.dev/JuMP.jl/dev/examples/basic/, https://github.com/SciML/ModelingToolkit.jl https://github.com/SciML/ModelingToolkit.jl or https://turing.ml/dev/docs/using-turing/get-started https://turing.ml/dev/docs/using-turing/get-started
  - ddragon 6y ago
    Yes, I was thinking about someone going many step furthers, like Elixir is for Erlang and the BEAM, someone who disagrees with design decisions of the Julia language but appreciates the compiler's capabilities and the ecosystem enough to use them to implement their own vision for a programming language. Making it a Lisp is just one of the more obvious scenarios, as someone may like Julia in general but want the full power of s-expressions.
    - rlander 6y ago
      How is Elixir’s design different from Erlang’s? Semantically they’re pretty much the same language. A better example would be Java/Clojure.
      - ddragon 6y ago
eyelidlessness 6y ago
> Optional typing - Currently not implemented Jesus H Christ could the whole FP world stop this? You're either going to prioritize static types or you're going to blow them off. Either front-and-center your type system, or stop mentioning how you blew it off. Those of us who want Clojure's syntax and semantics and a decent type system would like to stop being taunted.
- mumblemumble 6y ago
  I think you may have misunderstood what you're looking at. Julia is very much a dynamic language. It also has the ability to add type hints, but, unlike for most optionally typed languages, these aren't just type hints for human readers or an external type checker. They're actually one of Julia's more interesting features. Julia's type declarations provide information that the compiler uses to generate faster code, and to support its multiple dispatch functionality. When the notes say "optional typing - currently not implemented", all they're saying is that this syntax doesn't allow you to tap into that feature, because a syntax for type declarations has not been provided.
  - Sukera 6y ago
    Just a minor point - type hints only really help if you put them in struct fields. In function signatures they only limit what can be dispatched to that specific method, there's (almost always) no difference to leaving it untyped since functions are compiled for specialized types anyway.
- StefanKarpinski 6y ago
  I’m kind of confused by your comment. What is it that you’re unhappy about?
  - eyelidlessness 6y ago
    Basically every new lisp that comes out has a big bullet point of features and included in the list of features is a link in the README about static type checking. They all invariably raise the hope of a statically typed lisp for the given use case, then clicking through to the linked heading say “this project isn’t focusing on static types”. It’s Lucy and Charlie with the football. If you’re developing a lisp and you’re not interested in a static type system, not mentioning it is good enough. Making it a big headline only to yank it away in the rest of the text is just a disappointing setup.
blackrock 6y ago
What is the magic behind Lisp anyways? Is it that the Lisp language exposes the Abstract Syntax Tree (AST) of the code, so that you can reformulate it into your domain-specific macros, thereby helping you encapsulate complex logic into more succinct constructs? Is this why they say that Lisp is fantastic for small groups of programmers, that share similar groupthink, and uses all the same macros, but it becomes a horrendous mess when multiple teams are involved.
- keymone 6y ago
  lisp program is the program, not a representation of one like in all other kinds of languages (except machine code, which isn’t really a language).
- kazinator 6y ago
  > Is this why they say No, why they say is because they have zero experience with Lisp, in any size project. And there exists no tool using which some people have not made big, huge mess. We can easily lob this criticism at anything, quite randomly. Imagine a tool which has the property that a team of doofuses, no matter how large (in head count and doofiness), cannot make any sort of "horrendous mess". What sort of limitations does that entail, and could any of us here live with them?
  - blackrock 6y ago
    So what has your industry experience with Lisp been like?
- vindarel 6y ago
  > magic I'd say yes, this, though you don't actually manipulate a proper AST, but a list (tree) of program instructions. An AST would be given by a "code walker". Plus, the syntax is small and coherent, the language is stable, the syntax makes it straightforward to add new language constructs that would need a language release for another classical language. With most implementations of Common Lisp, you code against a live image, so you get instant feedback: compile a function with a keybinding, see compiler warnings or errors instantly, try it right away in the REPL (no process had to restart), get an interactive debugger on an error, fix it and resume the execution from a chosen frame (no stack unbinding), inspect objects, change a class definition and have existing objects being (lazely) updated, given rules in the standard that you can control… when ready, build a binary, and deploy. Today, SBCL's compile-time type-inference warnings are very helpful. Of course, some companies use CL in a million-sized codebase (if that helps as a counter example): Google (ITA software), SISCOG (underground and rail transport optimisation), ACL2 (industry-strength theorem prover)… https://lisp-lang.org/success/ https://lisp-lang.org/success/ https://github.com/azzamsa/awesome-lisp-companies https://github.com/azzamsa/awesome-lisp-companies (disclaimer: these resources are not complete)
warmfuzzykitten 6y ago
How about a Julia-like syntax for Clojure?

162 points by phonebucket 6y ago

Great example of building something fun not because we should but because we can :) My take on "Lisp in Julia" using deprecated-but-still-parseable syntax instead of a string macro: https://github.com/christopher-dG/jlisp https://github.com/christopher-dG/jlisp I should probably put some examples in the readme.

canadianfella 6y ago
Examples would help.
cbkeller 6y ago
That hidden Julia s-expression syntax helped me start to understand metaprogramming in Julia. Along the way I found that it takes surprisingly few lines of code to turn the normal Julia repl into a more-or-less working Julia-lisp repl: https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc765466f749ba https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc76...

ampdepolymerase 6y ago

The irony here is particularly rich considering Julia's parser frontend is written in Scheme.

yrimaxi 6y ago
Why is that ironic?
- DannyB2 6y ago
  At the very least it is amusing.
- eigenspace 6y ago
  I'm not sure, but maybe they're unhappy that it uses a more Clojure style syntax?
sritchie 6y ago
A couple years into fatherhood and I see puns everywhere, and suspect others of dropping them too. Particularly Rich... Hickey?
- asimjalis 6y ago
  I juxt ignore the puns.
  - deleted 6y ago
    [deleted]
- canadianfella 6y ago
  Where did you steele that joke from?
mumblemumble 6y ago
It seems more fitting than ironic to me. Julia's never been shy about its lisp heritage, and the reason why this project is only a couple hundred lines of code is that Julia remains very close to those roots.
- Blikkentrekker 6y ago
  Certainly any alternative syntax parser for any language would be? It would also only take a couple of hundred lines to write a parser for a C compiler that would allow that.
  - mumblemumble 6y ago
    Not any to any. You could't write a thin lispy alternative syntax for C because of some semantic differences. "If" is an expression in Julia and Lisp, and it's a statement in C. Dealing with things like goto statements and preprocessor macros would also require some extra work. Add all that up, and you're no longer looking at alternative syntax, you're looking at a transpiler.
    - Blikkentrekker 6y ago
      But this is also simply an alternative Syntax that mirrors Clojure, not actual Clojure semantics.
      - eigenspace 6y ago
        While julia does not have full lisp semantics, it is very close. Granted, it's not especially similar to Clojure itself. It's closer to a lisp though than almost any other relatively popular modern language that doesn't use S-expressions.
        Blikkentrekker 6y ago
        I'm very sceptical of such claims because I do not believe there is such a thing as “Lisp semantics” and that it is primarily a language of syntax. In particular, Common Lisp has many semantic features such as dynamic variables and the function–value dichotomy that do not exist in, say Scheme, or really any other commonly used programming language. And Scheme, yet again, has many rather esoteric operations, such as continuation capturing, that exists nowhere else as well. The three most popular Lisps today are C.L., Clojure, and Scheme, and I don't see these three having unified semantics at all and they are often further from each other than they are to many other languages.

fulafel 6y ago

My current favourite of these is this Lua version: https://github.com/bakpakin/Fennel https://github.com/bakpakin/Fennel

phonebucket 6y ago

A fun Julia easter egg I recently discovered. Running 'julia --lisp' launches a femtolisp (https://github.com/JeffBezanson/femtolisp https://github.com/JeffBezanson/femtolisp) interpreter.

retzkek 6y ago
The Julia parser is actually written in femtolisp: https://docs.julialang.org/en/v1/devdocs/eval/#dev-parsing https://docs.julialang.org/en/v1/devdocs/eval/#dev-parsing https://github.com/JuliaLang/julia/blob/master/src/julia-parser.scm https://github.com/JuliaLang/julia/blob/master/src/julia-par...

ddragon 6y ago

While not nearly as mature as the main platforms (like the JVM, .NET, BEAM), the Julia compiler does have some unique features (an extremely aggressive optimizer, a sophisticated type inference mechanism, possibly the most exceptional multiple dispatch support, Go-like lightweight processes), including a macro system that supports reader macros for writing parsers (like this library) and extensible multi-stage compilation (like the IR manipulation that is used in auto-differentiation of source code), which makes me wonder if once all those features (and possibly new ones) become robust enough it will become one of the targets for people creating new programming languages.

stabbles 6y ago
What you see most is domain-specific languages implemented through macro's. For instance this package, https://jump.dev/JuMP.jl/dev/examples/basic/ https://jump.dev/JuMP.jl/dev/examples/basic/, https://github.com/SciML/ModelingToolkit.jl https://github.com/SciML/ModelingToolkit.jl or https://turing.ml/dev/docs/using-turing/get-started https://turing.ml/dev/docs/using-turing/get-started
- ddragon 6y ago
  Yes, I was thinking about someone going many step furthers, like Elixir is for Erlang and the BEAM, someone who disagrees with design decisions of the Julia language but appreciates the compiler's capabilities and the ecosystem enough to use them to implement their own vision for a programming language. Making it a Lisp is just one of the more obvious scenarios, as someone may like Julia in general but want the full power of s-expressions.
  - rlander 6y ago
    How is Elixir’s design different from Erlang’s? Semantically they’re pretty much the same language. A better example would be Java/Clojure.
    - ddragon 6y ago

eyelidlessness 6y ago

> Optional typing - Currently not implemented Jesus H Christ could the whole FP world stop this? You're either going to prioritize static types or you're going to blow them off. Either front-and-center your type system, or stop mentioning how you blew it off. Those of us who want Clojure's syntax and semantics and a decent type system would like to stop being taunted.

mumblemumble 6y ago
I think you may have misunderstood what you're looking at. Julia is very much a dynamic language. It also has the ability to add type hints, but, unlike for most optionally typed languages, these aren't just type hints for human readers or an external type checker. They're actually one of Julia's more interesting features. Julia's type declarations provide information that the compiler uses to generate faster code, and to support its multiple dispatch functionality. When the notes say "optional typing - currently not implemented", all they're saying is that this syntax doesn't allow you to tap into that feature, because a syntax for type declarations has not been provided.
- Sukera 6y ago
  Just a minor point - type hints only really help if you put them in struct fields. In function signatures they only limit what can be dispatched to that specific method, there's (almost always) no difference to leaving it untyped since functions are compiled for specialized types anyway.
StefanKarpinski 6y ago
I’m kind of confused by your comment. What is it that you’re unhappy about?
- eyelidlessness 6y ago
  Basically every new lisp that comes out has a big bullet point of features and included in the list of features is a link in the README about static type checking. They all invariably raise the hope of a statically typed lisp for the given use case, then clicking through to the linked heading say “this project isn’t focusing on static types”. It’s Lucy and Charlie with the football. If you’re developing a lisp and you’re not interested in a static type system, not mentioning it is good enough. Making it a big headline only to yank it away in the rest of the text is just a disappointing setup.

blackrock 6y ago

What is the magic behind Lisp anyways? Is it that the Lisp language exposes the Abstract Syntax Tree (AST) of the code, so that you can reformulate it into your domain-specific macros, thereby helping you encapsulate complex logic into more succinct constructs? Is this why they say that Lisp is fantastic for small groups of programmers, that share similar groupthink, and uses all the same macros, but it becomes a horrendous mess when multiple teams are involved.

keymone 6y ago
lisp program is the program, not a representation of one like in all other kinds of languages (except machine code, which isn’t really a language).
kazinator 6y ago
> Is this why they say No, why they say is because they have zero experience with Lisp, in any size project. And there exists no tool using which some people have not made big, huge mess. We can easily lob this criticism at anything, quite randomly. Imagine a tool which has the property that a team of doofuses, no matter how large (in head count and doofiness), cannot make any sort of "horrendous mess". What sort of limitations does that entail, and could any of us here live with them?
- blackrock 6y ago
  So what has your industry experience with Lisp been like?
vindarel 6y ago
> magic I'd say yes, this, though you don't actually manipulate a proper AST, but a list (tree) of program instructions. An AST would be given by a "code walker". Plus, the syntax is small and coherent, the language is stable, the syntax makes it straightforward to add new language constructs that would need a language release for another classical language. With most implementations of Common Lisp, you code against a live image, so you get instant feedback: compile a function with a keybinding, see compiler warnings or errors instantly, try it right away in the REPL (no process had to restart), get an interactive debugger on an error, fix it and resume the execution from a chosen frame (no stack unbinding), inspect objects, change a class definition and have existing objects being (lazely) updated, given rules in the standard that you can control… when ready, build a binary, and deploy. Today, SBCL's compile-time type-inference warnings are very helpful. Of course, some companies use CL in a million-sized codebase (if that helps as a counter example): Google (ITA software), SISCOG (underground and rail transport optimisation), ACL2 (industry-strength theorem prover)… https://lisp-lang.org/success/ https://lisp-lang.org/success/ https://github.com/azzamsa/awesome-lisp-companies https://github.com/azzamsa/awesome-lisp-companies (disclaimer: these resources are not complete)

warmfuzzykitten 6y ago

How about a Julia-like syntax for Clojure?