11 ms·

Direct Clojure to C Compiler - Based on ClojureScript

89 points by espeed 14y ago

DanWaterworth 14y ago
I'm asking because I'm in the process of writing my own lisp->c compiler: Are you forced to use as environment like: typedef struct environment { struct environment *up; int num_bindings; value_t *bindings [0]; } environment_t; Or would it be possible to have a dynamic vector as your environment (so that you get O(1) lookup) at the cost of making constructing closures more costly (as you'd have to copy the bindings)?
- jules 14y ago
  Sure, you can do that. Besides the closure allocation cost you have to be careful if your Lisp supports setting variables directly: (define x 2) (define y (lambda (z) (+ x z)) (set! x 3) Now you want the `x` referenced inside the lambda to refer to the new value of `x`. So if closure creation copies bindings in the parent scope then you have to introduce an extra indirection for mutable variables. This is called "assignment conversion". (define x (make-ref 2)) (define y (lambda (z) (+ (read-ref x) z)) (set-ref! x 3) Note also that it is unnecessary to store the number of bindings in the environment, because this number is implied by the lambda associated with the closure.
  - DanWaterworth 14y ago
    Awesome, that's exactly the answer I was hoping for. Is this a common way of compiling lisp or does it actually make things slower in the general case?
    - jules 14y ago
      Both methods have pros and cons, and the most advanced implementations use a hybrid method. The copying method is probably the best to start with. It may make things slower in special circumstances, but in the common case it is faster because lambdas tend to have a small number of free variables. It is also way simpler, because linked environments have a tendency to retain garbage if you are not careful. For example: (let ((x 1) (y 2)) (lambda (z) (+ x z)) If you use linked closures naively then the closure will keep `y` in memory even though it's garbage. However, keep in mind that what you usually not think of as free variables are free variables. For example if you have this: (let ((y 4)) (map (lambda (x) (factorial (+ x y))) xs)) Then not only `y` but also `factorial` and `+` are free variables. With some work you can avoid storing those references in the closure. I believe that the state of the art is described in this paper: ftp://ftp.ecn.purdue.edu/qobi/99-105.pdf
      - DanWaterworth 14y ago
        All good points. I will surely read that paper, thanks for all your help.
- exDM69 14y ago
  Depending on what type of Lisp you're compiling, you may be able to resolve some locations at compile time. See section "lexical addressing" in SICP chapter 5, here: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-35.html#%_sec_5.5.6 http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-35.html...
  - deleted 14y ago
    [deleted]
  - DanWaterworth 14y ago
    Thanks, I appreciate the link.
- mark-probst 14y ago
  A dynamic vector will not work in the general case because environments can "fork". You can, however, have shallow environments, where making a new environment based on an old one involves copying some or all of the old environment's fields. That can become a problem, however, if your language supports set!ting local variables - the local variable might be represented in more than one environment, so you'd have to keep track of that and do more than one store. See Andrew Appel's "Compiling with Continuations", for example.
  - DanWaterworth 14y ago
    Could you give an example of where an environment can fork. Do you mean when you create a closure?
    - mark-probst 14y ago
      This example is completely contrived, of course: (let [x 1] (defn make-adder [y] (fn [] (+ x y))) (defn make-multiplier [z] (fn [] (* x z))) (defn mutate! [new-x] (set! x new-x))) Now for (make-adder 2) we get a function whose environment includes x and y, whereas the environment for (make-multiplier 2) includes x and z. Assuming you have shallow environments, you'll have one environment [x' y] and another one [x'' z], whereas with deep environments (like in ClojureC) you'd have [[x] y] and [[x] z] where the [x] is shared between them. Now, if you call (mutate! 3) with deep environments you only need to store the 3 once (to x), whereas with shallow environments you'll need to store to x' and x''.
      - DanWaterworth 14y ago
        Sure, you can come up with pathological cases for either option. For example, with deep environments, the following is inefficient: (lambda (a) (lambda (b) (lambda (c) (lambda (d) (lambda (e) a)))))
- rayiner 14y ago
  The technique used in most Lisp compilers (SBCL, etc) is as follows: Say you have an outer lambda and an inner lambda. All variables (either arguments or local variables introduced by "let") defined in the outer lambda and used in the inner lambda are called the "free variables" of the inner lambda and the "closed over variables" of the outer lambda. Of the closed over variables of the outer lambda, all the ones that are assigned-to in the inner lambda are "assigned to variables." You can perform this classification in one pass before code generation. Once you have annotated each lexical variable with its "closed over" and "assigned to" status, you can generate flat C functions in one code generation pass. The idea is to generate each inner lambda as a C function that accepts a hidden "environment vector" that contains the closed-over variables in the outer lambda. This environment vector is indexed simply by an integer index. Just handle the following AST constructs as follows: LET: if the lexical variable introduced by the LET form is closed-over and assigned-to, it must be demoted to a box because the inner lambda can't modify the outer lambda's stack frame. Whereas you can generate a regular lexical variable directly as a C local variable, you have to generate a closed-over + assigned-to variable as a C variable that points to a single heap-allocated box that references a Lisp value. VARIABLE-REF: for every reference to a variable, you have to look at it's status. If the variable is defined in the current lambda and not assigned-to, you can just emit a use of the C variable directly. If it is defined in the current lambda and assigned-to, you have to emit a load of the value from its box, a pointer to which is in a C variable in the current function. If the variable is defined in an outer lambda (it's a free variable of the current lambda), but not assigned-to, you have to emit the reference as a load (with a known constant index) from the environment vector. If it's a free variable and assigned-to, you have to load a reference to a box from the environment vector, then load the value from the box. LAMBDA: assign each free variable of the lambda an index starting at 0 (which you will use when emitting references to free variables in the body of the lambda). Generate the LAMBDA expression as the creation of an object with two slots: a pointer to the C function of the inner lambda, and a pointer to an environment vector. Emit code to copy each free variable referenced in the inner lambda into the environment vector. Remember you might have multiple levels of nested lambdas, so follow the rules for VARIABLE-REF above when emitting each copy. Then, just structure your calling convention so you can FUNCALL lambda objects and pas the environment vector as a hidden first argument. Note that this requires a bit more environment book-keeping in your compiler than you might otherwise have. You really want each inner lambda to have a complete environment, including free variables. This is so you can keep track of which variable has which index in the environment vector. Say you have: (defun foo() (let ((x 0) (y 1) (z 2)) (labels ((bar () (+ x y)) (baz () (- z x))) ...))) In "bar" your free variable indices probably look like {x: 0, y: 1} while in "baz" your free variable indices probably looks like {z: 0, x: 1}. You need to be able to keep track of these mappings separately, which you can't do if you attach these mappings in the environment structure you use for the LET form. Instead, you need to have a separate environment structure with entries in BAR and BAZ for the free variables of each lambda. In practice this generates very good code, because most lambdas simply don't have very many free variables (at least so long as you don't naively generate LET forms as nested lambdas).
exDM69 14y ago
Clojure is one of the cooler languages out there. There has been one major blocker for me, though: the JVM. I don't want to have anything to do with the JVM. Not because it's bad. It's just that native code is better, especially for the stuff I'm doing. I hope these developments eventually turn into a proper Lisp that you can interface with native C code efficiently. Racket is pretty cool too.
- alberich 14y ago
  I wonder what would be the advantages of using Clojure without the JVM over, say, Common Lisp. To me, it seems the great advantage of Clojure is its ease of using the large number of java libraries out there. However, if you take those libraries out, wouldn't a mature Lisp implementation like SBCL be much better?
  - amouat 14y ago
    There are still some nice things such as destructuring and vectors & maps as base types. The syntax of Clojure is generally a bit cleaner and more concise, though I'm sure some lispers will disagree.
  - pjmlp 14y ago
    There a few differences that might be worth still. Clojure is a Lisp-1, and provides better data structures as Common Lisp as far as I am aware. But the JVM benefit is a big one indeed.
    - alberich 14y ago
      I thought that, besides the JVM itself, the advantages of Clojure would be its concurrency model - as CL standard doesn't define anything related to this - and its stronger functional approach. I don't fully grasp the advantages of being a Lisp-1 or Lisp-2 :)
      - pjmlp 14y ago
        > I don't fully grasp the advantages of being a Lisp-1 or Lisp-2 :) It has to do with variable scoping. Lisp-1 only accepts lexical scoping, while Lisp-2 also allows for dynamic scoping. http://www.nhplace.com/kent/Papers/Technical-Issues.html http://www.nhplace.com/kent/Papers/Technical-Issues.html
andrewcooke 14y ago
obvious question here is: comparative benchmarks, anyone?
- mark-probst 14y ago
  It's too early to do benchmarks. Nothing in ClojureC is optimized at this point, so it would do very badly. I'm focussed at making it work first, then on making it fast. We'll get there.
  - Raphael_Amiard 14y ago
    That brings an interresting other question : How far are you into making it work completely ?
    - mark-probst 14y ago
      I'd say - optimistic estimates (for pessimistic, multiply by 2) - core language 1 more weekend, core library 2 more weekends after that. So far it's been about 3 weekends of work. Contributors are always welcome ;-)
      - Raphael_Amiard 14y ago
        I'll finish up cljs/lua and i might give a hand :)
        mark-probst 14y ago
        I'm looking forward to it! :-)
  - Raphael_Amiard 14y ago
    That brings an interresting other question : How far are you into making it work completely ?
pjmlp 14y ago
Cool, next step directly to native code skipping C. :)
- jared314 14y ago
  Or LLVM
markokocic 14y ago
Clojurescript is an interesting language, with great potential for easy porting. I wish there is a direct JVM port of Clojurescript. I would be able to leave with Clojurescript limitations compared to Clojure if it provides smaller jar files and better performance of compiled code. That would make it really usable on constrained targets like Android.
densh 14y ago
At first one should expect it to be at most as fast as cpython. If you don't do type inference and a bunch of other advanced functional language friendly optimizations you will get quite poor performance compared to JVM. Even haskell which does a lot of those isn't overperforming JVM yet. (Memory usage is much better though.) http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=ghc http://shootout.alioth.debian.org/u64/benchmark.php?test=all...
- dons 14y ago
  And now we can look forward to 20 years of research on very fast runtimes for executing Clojure ... http://research.microsoft.com/apps/pubs/default.aspx?id=79856 http://research.microsoft.com/apps/pubs/default.aspx?id=7985...
- igouy 14y ago
  >>Memory usage is much better though.<< "Don't confuse differences in default memory allocation with differences in Memory-used when the task requires programs to allocate more than the default memory." Look at memory use for reverse-complement, k-nucleotide, regex-dna, binary-trees, mandelbrot.
TazeTSchnitzel 14y ago
Great, now you can compile Clojure to C, then back to JavaScript with Emscripten! :P
- markus2012 14y ago
  Sounds like a nice obfuscator. I'm curious what the end result would look like! :-)
  - TazeTSchnitzel 14y ago
    Well, to do the full obfuscation, we would need to compile Clojure to C, then to JavaScript, then run it through Google Closure compiler. And then try to comprehend the mess that results.
goggles99 14y ago
Interesting but ultimately useless. I see something like this come around a couple of time per year. Always fades into oblivion within a year or two. The C code will not be optimized C (like a seasoned C developer would write). Java is compiled to byte code which is compiled to machine language. This ugly, unoptimized, generated C is compiled into machine language too. The performance (both memory and CPU usage) of the JVM product will be far superior through. Don't think that you will be writing for embedded devices or device drivers with something like this. it will be a mess. Use C for things that should to be done in C. For Desktop apps, Web apps, or services use Closure ETC. I don't like seeing a square peg rammed into a round hole. It does not work well so don't get too exited lisp programmers. This is an interesting hobby - that is about it.
- Xcelerate 14y ago
  Your post adds nothing of value to the discussion. You say this project will fade into oblivion -- how do you know this? Do you know, or are you familiar with, the developer's work? How do you know that the C code, when passed through an optimizing compiler, will not be able to compete with the JVM? Have you done these tests? The author already mentioned on this page that the goal is to get it working, and then focus on performance. As far as I can tell, he is a "seasoned C developer" and should be able to construct the generated C so that it compiles efficiently. Without more than mere pessimistic speculation, your failure to contribute any sort of primary or objective information does little more than to make you appear curiously angry at a project in which you have no interest or affiliation. EDIT: Just read through your comment history. A lot of your posts are written in this sort of negative tone. Why do you do that? If you would provide some links or alternative viewpoints, substantiated with a paper or even a blog post, that would be a lot more insightful.
  - goggles99 14y ago
    Poorly written C code run through an optimizing compiler will only produce slightly better (at best) executing machine code. Why do this? this should compile to native machine language directly if anything. Would you transform it to C first so you could edit the C? Then what if you want to make a big update using the Closure source? it will overwrite your changes. This approach is not practical, perhaps I am failing to understand any logic as to why you would go with this approach. Portability (between processor architectures) is all I can think of but you will never achieve competitive performance if this is the case. Good C coders write their code differently against different processors and platforms because all machines have different capabilities and resources (strength and weaknesses). The whole point these days of writing something in C to achieve the second highest performance (only behind pure assembly language) in both processor gpu/cpu/other co-processors and at the same time using as little memory as possible (both ram and NV storage).
goggles99 14y ago
Why do this? compile to native code directly if anything. Would you transform it to C first so you could edit the C? Then what if you want to make a big update using the Closure source? it will overwrite your changes. This approach is not practical, perhaps I am failing to understand any logic as to why you would go with this approach. Portability (between processor architectures) is all I can think of but you will never achieve high performance if this is the case. C coders write their code differently against different processors because all machines have different capabilities and resources (strength and weaknesses).
- spacemanaki 14y ago
  Compiling to C is not at all an uncommon strategy, there's at least one pretty serious Scheme implementation [1] that does this. Portability between architectures is a pretty big advantage, but in this case another one is leveraging the Boehm GC, rather than rolling your own. Could you use the Boehm GC if you emitted assembly? I would guess maybe it's possible, but you'd have to emit assembly that followed C calling conventions, I would imagine. Emitting C might be easier. Lisp in Small Pieces has a pretty big chapter that goes into detail how to compile Lisp to C. [1] http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Main_Page http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Main_P... see also: http://www.iro.umontreal.ca/~boucherd/mslug/meetings/20041020/90-min-scc/90-min-scc.pdf http://www.iro.umontreal.ca/~boucherd/mslug/meetings/2004102... http://www.iro.umontreal.ca/~boucherd/mslug/meetings/20041020/minutes-en.html http://www.iro.umontreal.ca/~boucherd/mslug/meetings/2004102...
  - takeoutweight 14y ago
    in clojure-scheme I actually compile Clojure to Gambit to make use of Gambit's awesome runtime. It's a very good match for Clojure. https://github.com/takeoutweight/clojure-scheme https://github.com/takeoutweight/clojure-scheme
- dons 14y ago
  C has been widely used as a "portable" target language. LLVM is increasingly used as a more semantically-sane alternative.
goggles99 14y ago
This IMO is an example of using the wrong tool for the job. Closure is awesome and has it's place, C is the industry standard for performance and also has it's place. The lines will never be crossed between these two languages. You are trying to fit a square peg in a round hole. If you improve your skills from this project - great, but please don't try to mislead anyone that this is a viable replacement for C (be up front about this). A language from an entirely different paradigm (Functional language, Garbage collection, tons of abstractions and a huge API) cannot ever be a replacement for or even compete on the same playing field C. We are talking apples to oranges here.

89 points by espeed 14y ago

I'm asking because I'm in the process of writing my own lisp->c compiler: Are you forced to use as environment like: typedef struct environment { struct environment *up; int num_bindings; value_t *bindings [0]; } environment_t; Or would it be possible to have a dynamic vector as your environment (so that you get O(1) lookup) at the cost of making constructing closures more costly (as you'd have to copy the bindings)?

jules 14y ago
Sure, you can do that. Besides the closure allocation cost you have to be careful if your Lisp supports setting variables directly: (define x 2) (define y (lambda (z) (+ x z)) (set! x 3) Now you want the `x` referenced inside the lambda to refer to the new value of `x`. So if closure creation copies bindings in the parent scope then you have to introduce an extra indirection for mutable variables. This is called "assignment conversion". (define x (make-ref 2)) (define y (lambda (z) (+ (read-ref x) z)) (set-ref! x 3) Note also that it is unnecessary to store the number of bindings in the environment, because this number is implied by the lambda associated with the closure.
- DanWaterworth 14y ago
  Awesome, that's exactly the answer I was hoping for. Is this a common way of compiling lisp or does it actually make things slower in the general case?
  - jules 14y ago
    Both methods have pros and cons, and the most advanced implementations use a hybrid method. The copying method is probably the best to start with. It may make things slower in special circumstances, but in the common case it is faster because lambdas tend to have a small number of free variables. It is also way simpler, because linked environments have a tendency to retain garbage if you are not careful. For example: (let ((x 1) (y 2)) (lambda (z) (+ x z)) If you use linked closures naively then the closure will keep `y` in memory even though it's garbage. However, keep in mind that what you usually not think of as free variables are free variables. For example if you have this: (let ((y 4)) (map (lambda (x) (factorial (+ x y))) xs)) Then not only `y` but also `factorial` and `+` are free variables. With some work you can avoid storing those references in the closure. I believe that the state of the art is described in this paper: ftp://ftp.ecn.purdue.edu/qobi/99-105.pdf
    - DanWaterworth 14y ago
      All good points. I will surely read that paper, thanks for all your help.
exDM69 14y ago
Depending on what type of Lisp you're compiling, you may be able to resolve some locations at compile time. See section "lexical addressing" in SICP chapter 5, here: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-35.html#%_sec_5.5.6 http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-35.html...
- deleted 14y ago
  [deleted]
- DanWaterworth 14y ago
  Thanks, I appreciate the link.
mark-probst 14y ago
A dynamic vector will not work in the general case because environments can "fork". You can, however, have shallow environments, where making a new environment based on an old one involves copying some or all of the old environment's fields. That can become a problem, however, if your language supports set!ting local variables - the local variable might be represented in more than one environment, so you'd have to keep track of that and do more than one store. See Andrew Appel's "Compiling with Continuations", for example.
- DanWaterworth 14y ago
  Could you give an example of where an environment can fork. Do you mean when you create a closure?
  - mark-probst 14y ago
    This example is completely contrived, of course: (let [x 1] (defn make-adder [y] (fn [] (+ x y))) (defn make-multiplier [z] (fn [] (* x z))) (defn mutate! [new-x] (set! x new-x))) Now for (make-adder 2) we get a function whose environment includes x and y, whereas the environment for (make-multiplier 2) includes x and z. Assuming you have shallow environments, you'll have one environment [x' y] and another one [x'' z], whereas with deep environments (like in ClojureC) you'd have [[x] y] and [[x] z] where the [x] is shared between them. Now, if you call (mutate! 3) with deep environments you only need to store the 3 once (to x), whereas with shallow environments you'll need to store to x' and x''.
    - DanWaterworth 14y ago
      Sure, you can come up with pathological cases for either option. For example, with deep environments, the following is inefficient: (lambda (a) (lambda (b) (lambda (c) (lambda (d) (lambda (e) a)))))
rayiner 14y ago
The technique used in most Lisp compilers (SBCL, etc) is as follows: Say you have an outer lambda and an inner lambda. All variables (either arguments or local variables introduced by "let") defined in the outer lambda and used in the inner lambda are called the "free variables" of the inner lambda and the "closed over variables" of the outer lambda. Of the closed over variables of the outer lambda, all the ones that are assigned-to in the inner lambda are "assigned to variables." You can perform this classification in one pass before code generation. Once you have annotated each lexical variable with its "closed over" and "assigned to" status, you can generate flat C functions in one code generation pass. The idea is to generate each inner lambda as a C function that accepts a hidden "environment vector" that contains the closed-over variables in the outer lambda. This environment vector is indexed simply by an integer index. Just handle the following AST constructs as follows: LET: if the lexical variable introduced by the LET form is closed-over and assigned-to, it must be demoted to a box because the inner lambda can't modify the outer lambda's stack frame. Whereas you can generate a regular lexical variable directly as a C local variable, you have to generate a closed-over + assigned-to variable as a C variable that points to a single heap-allocated box that references a Lisp value. VARIABLE-REF: for every reference to a variable, you have to look at it's status. If the variable is defined in the current lambda and not assigned-to, you can just emit a use of the C variable directly. If it is defined in the current lambda and assigned-to, you have to emit a load of the value from its box, a pointer to which is in a C variable in the current function. If the variable is defined in an outer lambda (it's a free variable of the current lambda), but not assigned-to, you have to emit the reference as a load (with a known constant index) from the environment vector. If it's a free variable and assigned-to, you have to load a reference to a box from the environment vector, then load the value from the box. LAMBDA: assign each free variable of the lambda an index starting at 0 (which you will use when emitting references to free variables in the body of the lambda). Generate the LAMBDA expression as the creation of an object with two slots: a pointer to the C function of the inner lambda, and a pointer to an environment vector. Emit code to copy each free variable referenced in the inner lambda into the environment vector. Remember you might have multiple levels of nested lambdas, so follow the rules for VARIABLE-REF above when emitting each copy. Then, just structure your calling convention so you can FUNCALL lambda objects and pas the environment vector as a hidden first argument. Note that this requires a bit more environment book-keeping in your compiler than you might otherwise have. You really want each inner lambda to have a complete environment, including free variables. This is so you can keep track of which variable has which index in the environment vector. Say you have: (defun foo() (let ((x 0) (y 1) (z 2)) (labels ((bar () (+ x y)) (baz () (- z x))) ...))) In "bar" your free variable indices probably look like {x: 0, y: 1} while in "baz" your free variable indices probably looks like {z: 0, x: 1}. You need to be able to keep track of these mappings separately, which you can't do if you attach these mappings in the environment structure you use for the LET form. Instead, you need to have a separate environment structure with entries in BAR and BAZ for the free variables of each lambda. In practice this generates very good code, because most lambdas simply don't have very many free variables (at least so long as you don't naively generate LET forms as nested lambdas).

exDM69 14y ago

Clojure is one of the cooler languages out there. There has been one major blocker for me, though: the JVM. I don't want to have anything to do with the JVM. Not because it's bad. It's just that native code is better, especially for the stuff I'm doing. I hope these developments eventually turn into a proper Lisp that you can interface with native C code efficiently. Racket is pretty cool too.

alberich 14y ago
I wonder what would be the advantages of using Clojure without the JVM over, say, Common Lisp. To me, it seems the great advantage of Clojure is its ease of using the large number of java libraries out there. However, if you take those libraries out, wouldn't a mature Lisp implementation like SBCL be much better?
- amouat 14y ago
  There are still some nice things such as destructuring and vectors & maps as base types. The syntax of Clojure is generally a bit cleaner and more concise, though I'm sure some lispers will disagree.
- pjmlp 14y ago
  There a few differences that might be worth still. Clojure is a Lisp-1, and provides better data structures as Common Lisp as far as I am aware. But the JVM benefit is a big one indeed.
  - alberich 14y ago
    I thought that, besides the JVM itself, the advantages of Clojure would be its concurrency model - as CL standard doesn't define anything related to this - and its stronger functional approach. I don't fully grasp the advantages of being a Lisp-1 or Lisp-2 :)
    - pjmlp 14y ago
      > I don't fully grasp the advantages of being a Lisp-1 or Lisp-2 :) It has to do with variable scoping. Lisp-1 only accepts lexical scoping, while Lisp-2 also allows for dynamic scoping. http://www.nhplace.com/kent/Papers/Technical-Issues.html http://www.nhplace.com/kent/Papers/Technical-Issues.html

andrewcooke 14y ago

obvious question here is: comparative benchmarks, anyone?

mark-probst 14y ago
It's too early to do benchmarks. Nothing in ClojureC is optimized at this point, so it would do very badly. I'm focussed at making it work first, then on making it fast. We'll get there.
- Raphael_Amiard 14y ago
  That brings an interresting other question : How far are you into making it work completely ?
  - mark-probst 14y ago
    I'd say - optimistic estimates (for pessimistic, multiply by 2) - core language 1 more weekend, core library 2 more weekends after that. So far it's been about 3 weekends of work. Contributors are always welcome ;-)
    - Raphael_Amiard 14y ago
      I'll finish up cljs/lua and i might give a hand :)
      - mark-probst 14y ago
        I'm looking forward to it! :-)
- Raphael_Amiard 14y ago
  That brings an interresting other question : How far are you into making it work completely ?

pjmlp 14y ago

Cool, next step directly to native code skipping C. :)

jared314 14y ago
Or LLVM

markokocic 14y ago

Clojurescript is an interesting language, with great potential for easy porting. I wish there is a direct JVM port of Clojurescript. I would be able to leave with Clojurescript limitations compared to Clojure if it provides smaller jar files and better performance of compiled code. That would make it really usable on constrained targets like Android.

densh 14y ago

At first one should expect it to be at most as fast as cpython. If you don't do type inference and a bunch of other advanced functional language friendly optimizations you will get quite poor performance compared to JVM. Even haskell which does a lot of those isn't overperforming JVM yet. (Memory usage is much better though.) http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=ghc http://shootout.alioth.debian.org/u64/benchmark.php?test=all...

dons 14y ago
And now we can look forward to 20 years of research on very fast runtimes for executing Clojure ... http://research.microsoft.com/apps/pubs/default.aspx?id=79856 http://research.microsoft.com/apps/pubs/default.aspx?id=7985...
igouy 14y ago
>>Memory usage is much better though.<< "Don't confuse differences in default memory allocation with differences in Memory-used when the task requires programs to allocate more than the default memory." Look at memory use for reverse-complement, k-nucleotide, regex-dna, binary-trees, mandelbrot.

TazeTSchnitzel 14y ago

Great, now you can compile Clojure to C, then back to JavaScript with Emscripten! :P

markus2012 14y ago
Sounds like a nice obfuscator. I'm curious what the end result would look like! :-)
- TazeTSchnitzel 14y ago
  Well, to do the full obfuscation, we would need to compile Clojure to C, then to JavaScript, then run it through Google Closure compiler. And then try to comprehend the mess that results.

goggles99 14y ago

Interesting but ultimately useless. I see something like this come around a couple of time per year. Always fades into oblivion within a year or two. The C code will not be optimized C (like a seasoned C developer would write). Java is compiled to byte code which is compiled to machine language. This ugly, unoptimized, generated C is compiled into machine language too. The performance (both memory and CPU usage) of the JVM product will be far superior through. Don't think that you will be writing for embedded devices or device drivers with something like this. it will be a mess. Use C for things that should to be done in C. For Desktop apps, Web apps, or services use Closure ETC. I don't like seeing a square peg rammed into a round hole. It does not work well so don't get too exited lisp programmers. This is an interesting hobby - that is about it.

Xcelerate 14y ago
Your post adds nothing of value to the discussion. You say this project will fade into oblivion -- how do you know this? Do you know, or are you familiar with, the developer's work? How do you know that the C code, when passed through an optimizing compiler, will not be able to compete with the JVM? Have you done these tests? The author already mentioned on this page that the goal is to get it working, and then focus on performance. As far as I can tell, he is a "seasoned C developer" and should be able to construct the generated C so that it compiles efficiently. Without more than mere pessimistic speculation, your failure to contribute any sort of primary or objective information does little more than to make you appear curiously angry at a project in which you have no interest or affiliation. EDIT: Just read through your comment history. A lot of your posts are written in this sort of negative tone. Why do you do that? If you would provide some links or alternative viewpoints, substantiated with a paper or even a blog post, that would be a lot more insightful.
- goggles99 14y ago
  Poorly written C code run through an optimizing compiler will only produce slightly better (at best) executing machine code. Why do this? this should compile to native machine language directly if anything. Would you transform it to C first so you could edit the C? Then what if you want to make a big update using the Closure source? it will overwrite your changes. This approach is not practical, perhaps I am failing to understand any logic as to why you would go with this approach. Portability (between processor architectures) is all I can think of but you will never achieve competitive performance if this is the case. Good C coders write their code differently against different processors and platforms because all machines have different capabilities and resources (strength and weaknesses). The whole point these days of writing something in C to achieve the second highest performance (only behind pure assembly language) in both processor gpu/cpu/other co-processors and at the same time using as little memory as possible (both ram and NV storage).

goggles99 14y ago

Why do this? compile to native code directly if anything. Would you transform it to C first so you could edit the C? Then what if you want to make a big update using the Closure source? it will overwrite your changes. This approach is not practical, perhaps I am failing to understand any logic as to why you would go with this approach. Portability (between processor architectures) is all I can think of but you will never achieve high performance if this is the case. C coders write their code differently against different processors because all machines have different capabilities and resources (strength and weaknesses).

spacemanaki 14y ago
Compiling to C is not at all an uncommon strategy, there's at least one pretty serious Scheme implementation [1] that does this. Portability between architectures is a pretty big advantage, but in this case another one is leveraging the Boehm GC, rather than rolling your own. Could you use the Boehm GC if you emitted assembly? I would guess maybe it's possible, but you'd have to emit assembly that followed C calling conventions, I would imagine. Emitting C might be easier. Lisp in Small Pieces has a pretty big chapter that goes into detail how to compile Lisp to C. [1] http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Main_Page http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Main_P... see also: http://www.iro.umontreal.ca/~boucherd/mslug/meetings/20041020/90-min-scc/90-min-scc.pdf http://www.iro.umontreal.ca/~boucherd/mslug/meetings/2004102... http://www.iro.umontreal.ca/~boucherd/mslug/meetings/20041020/minutes-en.html http://www.iro.umontreal.ca/~boucherd/mslug/meetings/2004102...
- takeoutweight 14y ago
  in clojure-scheme I actually compile Clojure to Gambit to make use of Gambit's awesome runtime. It's a very good match for Clojure. https://github.com/takeoutweight/clojure-scheme https://github.com/takeoutweight/clojure-scheme
dons 14y ago
C has been widely used as a "portable" target language. LLVM is increasingly used as a more semantically-sane alternative.

goggles99 14y ago

This IMO is an example of using the wrong tool for the job. Closure is awesome and has it's place, C is the industry standard for performance and also has it's place. The lines will never be crossed between these two languages. You are trying to fit a square peg in a round hole. If you improve your skills from this project - great, but please don't try to mislead anyone that this is a viable replacement for C (be up front about this). A language from an entirely different paradigm (Functional language, Garbage collection, tons of abstractions and a huge API) cannot ever be a replacement for or even compete on the same playing field C. We are talking apples to oranges here.