7 ms·

Metaprogramming – Julia Language

127 points by susi22 13y ago

breckinloggins 13y ago
Elixir has similar mechanisms. It's great to see language designers explore new ways of making a language homoiconic without pure s-expressions.
- draegtun 13y ago
  Rebol, Io, Smalltalk & Perl6 are also examples of non s-expression homoiconic languages.
  - joe_the_user 13y ago
    c++ too ... just kidding But more seriously, it seems like once one starts expanding the definition, homoiconicity gets to be in the eye of the beholder. "In a homoiconic language the primary representation of programs is also a data structure in a primitive type of the language itself." So you can take a monster language (at least any one with an eval command - Perl5 say), expand it's syntactical definition and get a "homoiconic language". http://en.wikipedia.org/wiki/Homoiconicity http://en.wikipedia.org/wiki/Homoiconicity
    - StefanKarpinski 13y ago
      Fully agree – homoiconicity ends up being almost more of a standard library feature rather than an essential part of the core language. For example, how does the existence of Python's ast module [1] not make Python homoiconic? The main difference, it seems, is how much the language lets you leverage having a representation of code as data to generate code, e.g. with macros. There are projects like macropy [2], but the core Python language doesn't provide macros – although I can't really fathom why at this point since most of the features are there. [1] http://docs.python.org/3/library/ast.html http://docs.python.org/3/library/ast.html [2] https://github.com/lihaoyi/macropy https://github.com/lihaoyi/macropy
    - draegtun 13y ago
      I know what you mean. In my examples I think only Perl6 is stretching the definition of homoiconicity, NB. are Perl6 Grammars == data structure/primitive ?? However Rebol (blocks & words) & Io/Smalltalk (objects and messages) do fully meet the definition you give above for being an homoiconic language.
- joe_the_user 13y ago
  The serious question I would have about non-s-expressions language is - while they might be homoiconic by whatever definition, what makes them easy to parse? The thing that's appealing about s-expression languages is that code-as-data is easy to do because parse, creating and modifying code is dead-easy. Is there some still-very-easy algorithm for parsing things like infix operators or does code-as-data depend on using the facilities of the language. Edit: I can see Julia does this by reflection and macros. But I'm still curious about the general issues and algorithms involved here.
  - krichman 13y ago
    > still-very-easy algorithm for parsing things like infix operators You would be interested in the shunting-yard algorithm[1], to start with. [1] https://en.wikipedia.org/wiki/Shunting_yard_algorithm https://en.wikipedia.org/wiki/Shunting_yard_algorithm
  - draegtun 13y ago
    >The serious question I would have about non-s-expressions language is - while they might be homoiconic by whatever definition, what makes them easy to parse? Rebol is very easy to parse. Here is a Rebol parser in 10 lines of Rebol: parse text blk-rule: [ some [ str: newline | #";" [thru newline | to end] new: (probe copy/part str new) | [#"[" | #"("] blk-rule | [#"]" | #")"] break | skip (set [value new] load/next str probe :value) :new ] ] ref: http://www.rebol.net/cookbook/recipes/0042.html http://www.rebol.net/cookbook/recipes/0042.html
    - joe_the_user 13y ago
      That is interesting and I'm getting back late on this. But after mentally parsing that, it seems all this is doing separating blocks. What one would want to also do is to also scan operators, if one was evaluating code or transforming it into an abstract syntax tree.
      - draegtun 13y ago
andrewcooke 13y ago
is the environment also available? are environments first class? it talks only about expressions - the environment in which they are evaluated seems to be implicit. what could you do with first class environments? could you implement logic variables and backtracking? something like oz? also, i think julia is optionally typed? what's the type system like for expressions? i saw an 'Any' in the link, which looked worryingly vague. [having said all that, as a "scientific programmer", i am worrying more and more that this is a language i really should be using] [edit: thanks for the informative replies]
- StefanKarpinski 13y ago
  Environments are intentionally not first class: that particular dynamic feature really sabotages optimization and can generally be worked around (e.g. instead of letting the full environment escape, expose a closure that mutates only what must be mutated). Julia has optional type annotations, but this doesn't mean that code without types is dynamic while code with types is static – all code is dynamic, the type annotations just limit the set of types that values can have. It's a subtle but important distinction.
- KenoFischer 13y ago
  All toplevel expressions are evaluated within a module which is a first class object. Other than that you always need to place the expression into its environment (to construct top-level expressions). If you just place macros the environment is not accessible (i.e. macros operate solely on their arguments), however, you can distinguish whether a symbol will be evaluated within the defining scope (of the macro) or the scope in which it is used. Regarding types, all expressions are of type Expr or Symbol (there's a few other things that can be in the AST, but that's mostly it). What kind of expression it is is determined by the head field. There as some discussion about making the Expr type parametric on its head field so you can dispatch on that, but nothing in that direction recently.
pekk 13y ago
so many sigils, is Julia channeling Perl?
- StefanKarpinski 13y ago
  The $ is only for splicing expressions into quoted code. Normal code that isn't doing metaprogramming doesn't have any sigils in it. So, no, not really channeling Perl.
  - sigzero 13y ago
    Only Perl can channel Perl.
    - antitrust 13y ago
      Once, deep in a meditative trance, I channeled Perl. My friends took me to the hospital after I refused to stop speaking in regular expressions and hashes of hashes.
  - susi22 13y ago
    Stephan, sorry for using this thread to ask you a question: In the last ~5-10 year everything has moved more an more to the web. HTML5+JS have pretty much replaced Flash,Java and other proprietary stuff. ~10 years ago it was almost all PHP, ASP. Then came Python etc. If you look at the speed that libraries were developed for node.js (and now Go), it seems like the web-dev community can really push forward a technology if it can be used to serve web content. Q: Is there anything done in Julia, to actively attract web-devs for it? I think that if Julia were "promoted" to also be perfect use as a web-server it would attract much more developer and speed up library development. Don't get me wrong, I'm not a web-dev, but would use Julia to replace MATLAB, but I think it might be a good strategy for Julia to speed up the development (especially for libraries).
    - zhemao 13y ago
      Not every language has to have webdev functionality shoehorned into it. I hardly think that having more web developers involved would speed up the development of the high quality scientific packages that Julia contributors are already currently working on. The people the Julia community should focus on attracting are data scientists and researchers in quantitative fields. That being said, the guys at Hackerschool are working on a web framework for Julia https://github.com/hackerschool/Morsel.jl https://github.com/hackerschool/Morsel.jl. Oh and major kudos to the Julia team. This is a seriously awesome piece of software. The decision to write most of the runtime in Julia itself and rely on a solid JIT compiler and simple FFI to get the necessary performance was very well thought out.
terhechte 13y ago
This is a copy from a comment I did in an earlier thread a couple of weeks ago, but I think it still applies: I really like Julia. I'm currently playing around with it whenever I find some spare time. I don't see it as a competitor to R, I think it could well be something along the lines of Python or Go. It is LLVM based, and already really fast even though it is still a 0.2 and the JIT seems to have a lot of room for optimisation. Whats more, it seems to offer just the right blend of language features: - Easily include C libraries via a simple ffi [1] - It is homoiconic like Lisp and thus allows for fantastic macro facilities [2] - It has solid parallel programming support via a Coroutines implementation (Tasks) (similar to Goroutines as far as I can tell) - It is a non-pure functional language - In contrast to Go it has generics, so functional constructs like map, apply, drop, reduce, fold, partition & friends are already in there (or can easily be implemented) [3] - It has optional types, so that you can assign types and the compiler will check for it and mark errors and will be able to create optimised code, but you don't have to [4] - Running external programs is a joy [5] (Example: a=readall(`echo hello`)) The community seems to be very alive. There's a simple web framework called "Morsel" and I've recently set it up against a couple of contenders from the web framework benchmark (cpoll-cppsp, phreeze, and some others), and even though it is still a version 0.2, the performance for the json serialization benchmark would be pretty close to Scalatra (I yet have to publish these numbers, will do so soon). I really hope that Julia will grow, as I love the choices that went into the design of the language, and it would be a shame if it would be only a replacement for R instead of something much bigger, as it is such a nice language. (original: https://news.ycombinator.com/item?id=6172751 https://news.ycombinator.com/item?id=6172751) [1] http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code/ http://docs.julialang.org/en/latest/manual/calling-c-and-for... [2] http://docs.julialang.org/en/latest/manual/metaprogramming/ http://docs.julialang.org/en/latest/manual/metaprogramming/ [3] http://docs.julialang.org/en/latest/stdlib/base/#general-collections http://docs.julialang.org/en/latest/stdlib/base/#general-col... [4] http://docs.julialang.org/en/latest/manual/types/ http://docs.julialang.org/en/latest/manual/types/ [5] http://docs.julialang.org/en/latest/manual/running-external-programs/ http://docs.julialang.org/en/latest/manual/running-external-...
- deleted 13y ago
  [deleted]
enupten 13y ago
I love that Julia does so many things in so Lispy a way. I wish that other languages like Lush received similar amounts of attention.
- zem 13y ago
  i looked at lush several times, but somehow never got into it. some of the things that put me off: the fact that sdl and opengl were not supported under windows (my primary interest was in developing desktop applications and small games), the dynamic scoping (i'm a scheme fan; not having lexical scoping throughout just feels offputting) and the (real or perceived) lack of a strong community. it did look like a great language, though, and i was rooting for it despite never finding a good use for it myself.
  - enupten 13y ago
    I agree with the lexical scoping part (explains more or less why I don't use Lush). Its a pity that there aren't any good numerical computing tools in Common Lisp/Scheme. Oh, well.
    - klibertp 13y ago
      What about http://docs.racket-lang.org/math/ http://docs.racket-lang.org/math/ ?
      - enupten 13y ago
        I haven't looked into Racket-math, but this post: http://lists.racket-lang.org/users/archive/2013-January/056047.html http://lists.racket-lang.org/users/archive/2013-January/0560... makes me wonder if Racket is really up for non-toy numerics. In contrast, SBCL uses tagless float arrays, so that array operations tend to be cache friendly. With adequate declarations, numerical code in SBCL tends to very fast (~1-3x C speed). I assume one can do the same with Bigloo and Chicken scheme as well.
        13y ago
alokv28 13y ago
Nice to see! I choose Mathematica over python for a lot of scientific/numerical computing due to how "lisp-y" Mathematica is. I'll be keeping a keen eye on Julia as well!

127 points by susi22 13y ago

Elixir has similar mechanisms. It's great to see language designers explore new ways of making a language homoiconic without pure s-expressions.

draegtun 13y ago
Rebol, Io, Smalltalk & Perl6 are also examples of non s-expression homoiconic languages.
- joe_the_user 13y ago
  c++ too ... just kidding But more seriously, it seems like once one starts expanding the definition, homoiconicity gets to be in the eye of the beholder. "In a homoiconic language the primary representation of programs is also a data structure in a primitive type of the language itself." So you can take a monster language (at least any one with an eval command - Perl5 say), expand it's syntactical definition and get a "homoiconic language". http://en.wikipedia.org/wiki/Homoiconicity http://en.wikipedia.org/wiki/Homoiconicity
  - StefanKarpinski 13y ago
    Fully agree – homoiconicity ends up being almost more of a standard library feature rather than an essential part of the core language. For example, how does the existence of Python's ast module [1] not make Python homoiconic? The main difference, it seems, is how much the language lets you leverage having a representation of code as data to generate code, e.g. with macros. There are projects like macropy [2], but the core Python language doesn't provide macros – although I can't really fathom why at this point since most of the features are there. [1] http://docs.python.org/3/library/ast.html http://docs.python.org/3/library/ast.html [2] https://github.com/lihaoyi/macropy https://github.com/lihaoyi/macropy
  - draegtun 13y ago
    I know what you mean. In my examples I think only Perl6 is stretching the definition of homoiconicity, NB. are Perl6 Grammars == data structure/primitive ?? However Rebol (blocks & words) & Io/Smalltalk (objects and messages) do fully meet the definition you give above for being an homoiconic language.
joe_the_user 13y ago
The serious question I would have about non-s-expressions language is - while they might be homoiconic by whatever definition, what makes them easy to parse? The thing that's appealing about s-expression languages is that code-as-data is easy to do because parse, creating and modifying code is dead-easy. Is there some still-very-easy algorithm for parsing things like infix operators or does code-as-data depend on using the facilities of the language. Edit: I can see Julia does this by reflection and macros. But I'm still curious about the general issues and algorithms involved here.
- krichman 13y ago
  > still-very-easy algorithm for parsing things like infix operators You would be interested in the shunting-yard algorithm[1], to start with. [1] https://en.wikipedia.org/wiki/Shunting_yard_algorithm https://en.wikipedia.org/wiki/Shunting_yard_algorithm
- draegtun 13y ago
  >The serious question I would have about non-s-expressions language is - while they might be homoiconic by whatever definition, what makes them easy to parse? Rebol is very easy to parse. Here is a Rebol parser in 10 lines of Rebol: parse text blk-rule: [ some [ str: newline | #";" [thru newline | to end] new: (probe copy/part str new) | [#"[" | #"("] blk-rule | [#"]" | #")"] break | skip (set [value new] load/next str probe :value) :new ] ] ref: http://www.rebol.net/cookbook/recipes/0042.html http://www.rebol.net/cookbook/recipes/0042.html
  - joe_the_user 13y ago
    That is interesting and I'm getting back late on this. But after mentally parsing that, it seems all this is doing separating blocks. What one would want to also do is to also scan operators, if one was evaluating code or transforming it into an abstract syntax tree.
    - draegtun 13y ago

andrewcooke 13y ago

is the environment also available? are environments first class? it talks only about expressions - the environment in which they are evaluated seems to be implicit. what could you do with first class environments? could you implement logic variables and backtracking? something like oz? also, i think julia is optionally typed? what's the type system like for expressions? i saw an 'Any' in the link, which looked worryingly vague. [having said all that, as a "scientific programmer", i am worrying more and more that this is a language i really should be using] [edit: thanks for the informative replies]

StefanKarpinski 13y ago
Environments are intentionally not first class: that particular dynamic feature really sabotages optimization and can generally be worked around (e.g. instead of letting the full environment escape, expose a closure that mutates only what must be mutated). Julia has optional type annotations, but this doesn't mean that code without types is dynamic while code with types is static – all code is dynamic, the type annotations just limit the set of types that values can have. It's a subtle but important distinction.
KenoFischer 13y ago
All toplevel expressions are evaluated within a module which is a first class object. Other than that you always need to place the expression into its environment (to construct top-level expressions). If you just place macros the environment is not accessible (i.e. macros operate solely on their arguments), however, you can distinguish whether a symbol will be evaluated within the defining scope (of the macro) or the scope in which it is used. Regarding types, all expressions are of type Expr or Symbol (there's a few other things that can be in the AST, but that's mostly it). What kind of expression it is is determined by the head field. There as some discussion about making the Expr type parametric on its head field so you can dispatch on that, but nothing in that direction recently.

pekk 13y ago

so many sigils, is Julia channeling Perl?

StefanKarpinski 13y ago
The $ is only for splicing expressions into quoted code. Normal code that isn't doing metaprogramming doesn't have any sigils in it. So, no, not really channeling Perl.
- sigzero 13y ago
  Only Perl can channel Perl.
  - antitrust 13y ago
    Once, deep in a meditative trance, I channeled Perl. My friends took me to the hospital after I refused to stop speaking in regular expressions and hashes of hashes.
- susi22 13y ago
  Stephan, sorry for using this thread to ask you a question: In the last ~5-10 year everything has moved more an more to the web. HTML5+JS have pretty much replaced Flash,Java and other proprietary stuff. ~10 years ago it was almost all PHP, ASP. Then came Python etc. If you look at the speed that libraries were developed for node.js (and now Go), it seems like the web-dev community can really push forward a technology if it can be used to serve web content. Q: Is there anything done in Julia, to actively attract web-devs for it? I think that if Julia were "promoted" to also be perfect use as a web-server it would attract much more developer and speed up library development. Don't get me wrong, I'm not a web-dev, but would use Julia to replace MATLAB, but I think it might be a good strategy for Julia to speed up the development (especially for libraries).
  - zhemao 13y ago
    Not every language has to have webdev functionality shoehorned into it. I hardly think that having more web developers involved would speed up the development of the high quality scientific packages that Julia contributors are already currently working on. The people the Julia community should focus on attracting are data scientists and researchers in quantitative fields. That being said, the guys at Hackerschool are working on a web framework for Julia https://github.com/hackerschool/Morsel.jl https://github.com/hackerschool/Morsel.jl. Oh and major kudos to the Julia team. This is a seriously awesome piece of software. The decision to write most of the runtime in Julia itself and rely on a solid JIT compiler and simple FFI to get the necessary performance was very well thought out.

terhechte 13y ago

This is a copy from a comment I did in an earlier thread a couple of weeks ago, but I think it still applies: I really like Julia. I'm currently playing around with it whenever I find some spare time. I don't see it as a competitor to R, I think it could well be something along the lines of Python or Go. It is LLVM based, and already really fast even though it is still a 0.2 and the JIT seems to have a lot of room for optimisation. Whats more, it seems to offer just the right blend of language features: - Easily include C libraries via a simple ffi [1] - It is homoiconic like Lisp and thus allows for fantastic macro facilities [2] - It has solid parallel programming support via a Coroutines implementation (Tasks) (similar to Goroutines as far as I can tell) - It is a non-pure functional language - In contrast to Go it has generics, so functional constructs like map, apply, drop, reduce, fold, partition & friends are already in there (or can easily be implemented) [3] - It has optional types, so that you can assign types and the compiler will check for it and mark errors and will be able to create optimised code, but you don't have to [4] - Running external programs is a joy [5] (Example: a=readall(`echo hello`)) The community seems to be very alive. There's a simple web framework called "Morsel" and I've recently set it up against a couple of contenders from the web framework benchmark (cpoll-cppsp, phreeze, and some others), and even though it is still a version 0.2, the performance for the json serialization benchmark would be pretty close to Scalatra (I yet have to publish these numbers, will do so soon). I really hope that Julia will grow, as I love the choices that went into the design of the language, and it would be a shame if it would be only a replacement for R instead of something much bigger, as it is such a nice language. (original: https://news.ycombinator.com/item?id=6172751 https://news.ycombinator.com/item?id=6172751) [1] http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code/ http://docs.julialang.org/en/latest/manual/calling-c-and-for... [2] http://docs.julialang.org/en/latest/manual/metaprogramming/ http://docs.julialang.org/en/latest/manual/metaprogramming/ [3] http://docs.julialang.org/en/latest/stdlib/base/#general-collections http://docs.julialang.org/en/latest/stdlib/base/#general-col... [4] http://docs.julialang.org/en/latest/manual/types/ http://docs.julialang.org/en/latest/manual/types/ [5] http://docs.julialang.org/en/latest/manual/running-external-programs/ http://docs.julialang.org/en/latest/manual/running-external-...

deleted 13y ago
[deleted]

enupten 13y ago

I love that Julia does so many things in so Lispy a way. I wish that other languages like Lush received similar amounts of attention.

zem 13y ago
i looked at lush several times, but somehow never got into it. some of the things that put me off: the fact that sdl and opengl were not supported under windows (my primary interest was in developing desktop applications and small games), the dynamic scoping (i'm a scheme fan; not having lexical scoping throughout just feels offputting) and the (real or perceived) lack of a strong community. it did look like a great language, though, and i was rooting for it despite never finding a good use for it myself.
- enupten 13y ago
  I agree with the lexical scoping part (explains more or less why I don't use Lush). Its a pity that there aren't any good numerical computing tools in Common Lisp/Scheme. Oh, well.
  - klibertp 13y ago
    What about http://docs.racket-lang.org/math/ http://docs.racket-lang.org/math/ ?
    - enupten 13y ago
      I haven't looked into Racket-math, but this post: http://lists.racket-lang.org/users/archive/2013-January/056047.html http://lists.racket-lang.org/users/archive/2013-January/0560... makes me wonder if Racket is really up for non-toy numerics. In contrast, SBCL uses tagless float arrays, so that array operations tend to be cache friendly. With adequate declarations, numerical code in SBCL tends to very fast (~1-3x C speed). I assume one can do the same with Bigloo and Chicken scheme as well.
      - 13y ago

alokv28 13y ago

Nice to see! I choose Mathematica over python for a lot of scientific/numerical computing due to how "lisp-y" Mathematica is. I'll be keeping a keen eye on Julia as well!