4 ms·

A large advantage of lisp like languages is the data-as-code paradigm. To give up eval is to give up this. It is definitely impossible to efficiently compile c

by seles 15y ago

A large advantage of lisp like languages is the data-as-code paradigm. To give up eval is to give up this. It is definitely impossible to efficiently compile code with evals for all possible programs, but lots of code using the data-as-code paradigm generates data that is completely independent of input sources (macro like). This could be computed at compile time and thus efficiently compiled. A similar technique has been used for making Haskell run faster, called super compilation, and it shouldn't be too hard to implement in any functional language. For evals that can't be compiled, well, it would be nice to still be able to do the dynamic inefficient eval, without botching the efficiency of the entire program. Do the normal dead code analysis, the eval might not work right, so provide a warning. The programmer can always do things to ensure any code they need isn't dead code eliminated by using it elsewhere. Yeah this is messy, but it leaves the choice up to the programmer and has no less power.

takeoutweight 15y ago
Supercompilation is the precisely the tool that should not be used for compiling a dynamic language--it relies heavily on static analysis that are simply unfeasible in a dynamic language. eg: anything that touches a dynamically bindable var cannot be specialized--this dynamicity will essentially infect the rest of the functions in the call graph. This is especially true when you're staging compilation via eval. Supercompilation is very slow and memory intensive, and it would be insane to track variable specialization information throughout runtime.
- seles 15y ago
  Static analysis is already being done, since it is being compiled. Supercompilation taken to the extreme is very slow, but don't take it to the extreme, doing just enough to evaluate the expressions which are passed into eval would be fine (and these are often simple, macro like substitutions).
  - takeoutweight 15y ago
    I'm not sure if we share the same notion of supercompilation. The whole point of supercompilation is to speculatively evaluate terms to supply branch-level constraints that propagate through the call graph, enabling program transformations such as inlining, dead code elimination, partial evaluation and deforestation. Whole-program transformations such as these are done during compilation (i.e. in the dev environment and not in production.) How would you retro-actively undo an existing transformed piece of code when a new form is observed by eval? How would you maintain a list of already-applied transformations to allow new code to be specialized? Supercompilation is not an incremental procedure.
    - seles 15y ago
      My notion of supercompilation is just the evaluating of expressions at compile time that do not depend on things at run time. A trivial case that most compilers do is converting print 1+1 to print 2. Since 1+1 does not depend on anything that could happen at run time and does not produce any other side effects. (You need to analyze that + hasn't been overloaded to have side effects/etc.). The idea is that many of the expressions being passed into eval could be calculated in entirety at compile time, so there is no need to have eval in the compiled result at all. Just evaluate at compile time, remove the eval and the quotes. I am not talking about trying to optimize eval, if you have a variable string going in, you can't possibly do that. I guess I shouldn't have used the term supercompiling, it may mean something more powerful than in the cases I'd seen it and I was probably trying too hard to use a buzzword...
      - takeoutweight 15y ago
        The term that describes what you are referring to is 'partial evaluation'
richhickey 15y ago
> A large advantage of lisp like languages is the data-as-code paradigm. To give up eval is to give up this. Code as data is about (among other things) programs writing programs, the majority of which (compilers and macros) don't need runtime eval, i.e. the program that writes the program isn't the program that's running it. Neither I, nor anyone I know, needs runtime eval of ClojureScript code in the browser. As the post states, it's certainly not needed for REPL interaction with a browser, something we intend to support. Why should we spend time writing something with no demonstrated need, other than to satisfy peoples' abstract notions of what constitutes a Lisp? Whole program optimization: "Optimizer, here is my entire program, do whatever you can in pursuit of size and speed: renaming, moving and eliminating as you see fit." Runtime eval: "Oh, wait, here's more program!" You can't do the former well and support the latter. If you think having the choice is important, write the code. Here's some of what's required: A ClojureScript port of the compiler (when multimethods are finished, not too bad) Syntax quote Runtime representation of namespaces, vars etc What you'll end up with is something that generates huge, slow JavaScript programs. Make sure to emphasize to your users "But it has eval, and is written in a real Lisp!". Seriously, I'm sure someone somewhere might actually require this, but they'll have to satisfy their own needs.
- seles 15y ago
  "As the post states, it's certainly not needed for REPL interaction with a browser" Ok, it is not needed per se, but how can it be done without also writing your own parser and evalulator? I didn't see mention of the how in the article. Also it's not Runtime eval: "Oh, wait, here's more program!" It's: "Oh, you forgot this part of the program, that I already gave you an exact description of". I'm not advocating any sort of complicated change, just evaluating simple independent expressions that generate the input to evals. I hypothosize that in most cases this is possible. Maybe eval is more often used in ways that does depend on input (REPL is such a case).
  - JoelMcCracken 15y ago
    I'm not sure exactly how this would be implemented, but the repl action would be to send your browser clojurescript to a server, which then compiles it, which then replies to you, which you then append to the dom in a script() (or, eval it). This seems tricky, though, because it seems as though you'd need a to evaluate it within the context of the previously compiled environment, and ideally you would want to return /only/ the newly required code + libs. You wouldnt want to return code that re-initializes the dom, for example. https://github.com/ibdknox/brepl https://github.com/ibdknox/brepl has that repl implementation.
lmkg 15y ago
The code-as-data paradigm in Lisp means that you can write your own eval. Eval was an important historical discovery, because of the fact that it could be written in Lisp itself. In fact, a minimal eval is rather easy: Recurse down nested linked lists, switching on the first element to handle special forms, and recursively evaluating functions and arguments. Eval is basically a DSL, which happens to be the same as the host language. Part of the awesomeness of Lisp is that you can write your own DSLs. Sometimes that involves macros, sometimes that involves a home-brew eval function. But either way, the full-blown eval has very few use cases.

by seles 15y ago

takeoutweight 15y ago

Supercompilation is the precisely the tool that should not be used for compiling a dynamic language--it relies heavily on static analysis that are simply unfeasible in a dynamic language. eg: anything that touches a dynamically bindable var cannot be specialized--this dynamicity will essentially infect the rest of the functions in the call graph. This is especially true when you're staging compilation via eval. Supercompilation is very slow and memory intensive, and it would be insane to track variable specialization information throughout runtime.

seles 15y ago
Static analysis is already being done, since it is being compiled. Supercompilation taken to the extreme is very slow, but don't take it to the extreme, doing just enough to evaluate the expressions which are passed into eval would be fine (and these are often simple, macro like substitutions).
- takeoutweight 15y ago
  I'm not sure if we share the same notion of supercompilation. The whole point of supercompilation is to speculatively evaluate terms to supply branch-level constraints that propagate through the call graph, enabling program transformations such as inlining, dead code elimination, partial evaluation and deforestation. Whole-program transformations such as these are done during compilation (i.e. in the dev environment and not in production.) How would you retro-actively undo an existing transformed piece of code when a new form is observed by eval? How would you maintain a list of already-applied transformations to allow new code to be specialized? Supercompilation is not an incremental procedure.
  - seles 15y ago
    My notion of supercompilation is just the evaluating of expressions at compile time that do not depend on things at run time. A trivial case that most compilers do is converting print 1+1 to print 2. Since 1+1 does not depend on anything that could happen at run time and does not produce any other side effects. (You need to analyze that + hasn't been overloaded to have side effects/etc.). The idea is that many of the expressions being passed into eval could be calculated in entirety at compile time, so there is no need to have eval in the compiled result at all. Just evaluate at compile time, remove the eval and the quotes. I am not talking about trying to optimize eval, if you have a variable string going in, you can't possibly do that. I guess I shouldn't have used the term supercompiling, it may mean something more powerful than in the cases I'd seen it and I was probably trying too hard to use a buzzword...
    - takeoutweight 15y ago
      The term that describes what you are referring to is 'partial evaluation'

richhickey 15y ago

> A large advantage of lisp like languages is the data-as-code paradigm. To give up eval is to give up this. Code as data is about (among other things) programs writing programs, the majority of which (compilers and macros) don't need runtime eval, i.e. the program that writes the program isn't the program that's running it. Neither I, nor anyone I know, needs runtime eval of ClojureScript code in the browser. As the post states, it's certainly not needed for REPL interaction with a browser, something we intend to support. Why should we spend time writing something with no demonstrated need, other than to satisfy peoples' abstract notions of what constitutes a Lisp? Whole program optimization: "Optimizer, here is my entire program, do whatever you can in pursuit of size and speed: renaming, moving and eliminating as you see fit." Runtime eval: "Oh, wait, here's more program!" You can't do the former well and support the latter. If you think having the choice is important, write the code. Here's some of what's required: A ClojureScript port of the compiler (when multimethods are finished, not too bad) Syntax quote Runtime representation of namespaces, vars etc What you'll end up with is something that generates huge, slow JavaScript programs. Make sure to emphasize to your users "But it has eval, and is written in a real Lisp!". Seriously, I'm sure someone somewhere might actually require this, but they'll have to satisfy their own needs.

seles 15y ago
"As the post states, it's certainly not needed for REPL interaction with a browser" Ok, it is not needed per se, but how can it be done without also writing your own parser and evalulator? I didn't see mention of the how in the article. Also it's not Runtime eval: "Oh, wait, here's more program!" It's: "Oh, you forgot this part of the program, that I already gave you an exact description of". I'm not advocating any sort of complicated change, just evaluating simple independent expressions that generate the input to evals. I hypothosize that in most cases this is possible. Maybe eval is more often used in ways that does depend on input (REPL is such a case).
- JoelMcCracken 15y ago
  I'm not sure exactly how this would be implemented, but the repl action would be to send your browser clojurescript to a server, which then compiles it, which then replies to you, which you then append to the dom in a script() (or, eval it). This seems tricky, though, because it seems as though you'd need a to evaluate it within the context of the previously compiled environment, and ideally you would want to return /only/ the newly required code + libs. You wouldnt want to return code that re-initializes the dom, for example. https://github.com/ibdknox/brepl https://github.com/ibdknox/brepl has that repl implementation.

lmkg 15y ago

The code-as-data paradigm in Lisp means that you can write your own eval. Eval was an important historical discovery, because of the fact that it could be written in Lisp itself. In fact, a minimal eval is rather easy: Recurse down nested linked lists, switching on the first element to handle special forms, and recursively evaluating functions and arguments. Eval is basically a DSL, which happens to be the same as the host language. Part of the awesomeness of Lisp is that you can write your own DSLs. Sometimes that involves macros, sometimes that involves a home-brew eval function. But either way, the full-blown eval has very few use cases.