6 ms·

With that approach you won't be able to write a compiler for anything other than a (suboptimal) Lisp derivative, which itself is just a small part of computing

by mojuba 18y ago

With that approach you won't be able to write a compiler for anything other than a (suboptimal) Lisp derivative, which itself is just a small part of computing world. Take a look at the GNU Compiler Collection and see how far it is to write a working optimizing compiler from the Lisp macro paradigm.

narag 18y ago
And even if writing a more complete compiler wasn't difficult enough, there's more much to it. There are complex details like exception handling (stack unwinding, signals...), graphical debuggers, interface with GUI libraries, threads, etc. The fact that only commercial (and expensive) Lisp implementations have all these features is a hint that they're not trivial.
- mojuba 18y ago
  And in fact what we all need is a good, commercial grade open-source Lisp compiler + tools.
  - KirinDave 18y ago
    I'm sorry, but I strongly disagree. Lisp's problems are not technical, they're social. We have a commercial-grade open-source lisp compiler. It's SBCL. And we have really excellent models for tools. If someone would extract SLIME from emacs and into an editor with less history behind it and more popular appeal, you'd have most of the tools you need. Lisp is suffering because its community is fragmented and it has no leaders. Name a popular language that doesn't have an iconic corporation or person behind it, rallying and focusing the community? That condition is actually quite rare.
    - mojuba 18y ago
      SBCL: is it as good as Allegro? I haven't seen an IDE as beautiful and comfortable to work with as Allegro (although I'm barely a Lisp hacker, just a fan).
      - KirinDave 18y ago
        I don't know what metrics you'd use to decide it's "better". Certainly the native code generation on SBCL is pretty darn good. I think Allegro has better tools and libraries, but SBCL is certainly suitable for professional work. At this point we're talking about matters of degrees.
    - rplevy 18y ago
      I think the reason why that is not likely to catch on is that most Lisp hackers enjoy using an editor that can itself be hacked in Lisp. Also, emacs is one of the most stable pieces of software in history. It will probably live forever and continue to evolve. Also, I think that Lisp isn't really suffering. As far as I can tell there has been increasing interest in Lisp. Even without that trend continuing, Lisp is such a masterpiece (and it continues to develop with new innovations), that it is almost definitely here to stay, regardless of whether it becomes a trendy language to use again. The advantages in performance (say SBCL's for example) and expressiveness over other dynamic languages like Ruby and Python afford Lisp developers an actual advantage, as opposed to a merely perceived one.
    - narag 18y ago
      >It's SBCL. And we have really excellent models for tools. If someone would extract SLIME from emacs and into an editor with less history behind it and more popular appeal, you'd have most of the tools you need. SBCL port to Windows is a work in progress. And what you're saying about tools is more or less that they doesn't exist... yet. And comparing what you need to know and set up to start working with open source Lisp and other environments is... not fair.
      - KirinDave 18y ago
        > SBCL port to Windows is a work in progress. So what? Lots of software gets written without ever touching Windows. I've based my entire career on it. An environment can certainly be mature and commercial quality without being multi-platform. > And comparing what you need to know and set up to start working with open source Lisp and other environments is... not fair. I do wish that SBCL had a "everything you need to know in 10 minutes or less", but it comes with ASDF and ASDF-INSTALL pre-configured. I mean, the amount of stuff you need to know to start working with Java is pretty darn daunting as well, given that you need massive assistance frameworks, and there is no good central package repo. The reason more people go into Java is that they have colleges introducing them to it. Yes, more could be done. But I think commercial, professional work is possible right now given the state of SBCL.
        pchristensen 18y ago
        ITA Software is proof of that possibility. Dan Weinreb has written about that often, on comp.lang.lisp and his own blog.
        narag 18y ago
        >So what? Lots of software gets written without ever touching Windows. Except most people doesn't want to write "lots of software", but software that fills certain requirements, often customers' requirements: run on Windows, do threads, have a rich GUI, minimize to tray, detect screensaver, interface to word processor, customer's database, customer's crappy ERP, etc. >the amount of stuff you need to know to start working with Java is pretty darn daunting as well, I've done both starts (Java and Lisp) and I can't disagree more. >But I think commercial, professional work is possible right now given the state of SBCL. It depends on your requirements. For some environments that's not true at all.
- gaius 18y ago
  I'm not sure that's true. Right now I am working on GNU CLISP with bindings to Oracle and a GUI (via Ltk). Sure it's not trivial, but it's not impossible either.
  - narag 18y ago
    That could fill the requirements for "enterprise" software, the kind of work that people happily convert to web apps. Making desktop software can be much much more demanding.
DaniFong 18y ago
I don't claim that this is the only way to learn how to write compilers, it's just one surprisingly effective way. And I actually think skill in generalised, simple, macro-type compilers is more useful, generally, than knowing the ins and outs of optimizing compilers, but that's just me. For the sake of discussion, what are some of the important things for optimizing compilers that you can't do with this approach? I'm having trouble finding any -- both in a literal sense of possibility, and from a practical standpoint.
- mojuba 18y ago
  I may be underestimating macros but I can't imagine any optimization technique that can be done with them when generating low-level output, be it native or virtual machine code. Can you demonstrate (just theoretically, of course) copy propagation, removing loop invariants, automatic inlining of functions, to name a few?
  - gruseom 18y ago
    It's not hard to see how macros could do this. Macros just transform code into other code, which is what optimizations do too. For example, if you have an s-expr representing a loop, imagine a function which accepts that s-expr and returns a new one with invariants moved outside the loop body.
    - mojuba 18y ago
      Ok, but my original point was, "you won't be able to write a compiler for anything other than a (suboptimal) Lisp derivative" (I was probably wrong about suboptimal though, because it only applies to the compiler)
      - gruseom 18y ago
        Well, the easy way to use Lisp macros in a compiler is to start with an s-expr and successively transform it until you've got something that the target platform accepts. "Starting with an s-expr" implies that your source code is in Lisp format. So in that sense, what you're compiling is a Lisp derivative. Is that what you mean? No doubt there are clever ways to leverage Lisp macros in compilers beyond this approach. But most people who like Lisp macros wouldn't bother. They'd just do it the easy way. Edit: my understanding of what DaniFong is getting at is that in Lisp programming, there is no longer a barrier between application development and compiler development. This makes possible a lot of powerful things that you can't do when the application is written in a fixed language by different programmers than the ones who write the compiler. It's a different point, but one I find very interesting. It's not obvious what belongs to application development and what belongs to language development once this technical (and organizational) barrier is removed.

by mojuba 18y ago

narag 18y ago

And even if writing a more complete compiler wasn't difficult enough, there's more much to it. There are complex details like exception handling (stack unwinding, signals...), graphical debuggers, interface with GUI libraries, threads, etc. The fact that only commercial (and expensive) Lisp implementations have all these features is a hint that they're not trivial.

mojuba 18y ago
And in fact what we all need is a good, commercial grade open-source Lisp compiler + tools.
- KirinDave 18y ago
  I'm sorry, but I strongly disagree. Lisp's problems are not technical, they're social. We have a commercial-grade open-source lisp compiler. It's SBCL. And we have really excellent models for tools. If someone would extract SLIME from emacs and into an editor with less history behind it and more popular appeal, you'd have most of the tools you need. Lisp is suffering because its community is fragmented and it has no leaders. Name a popular language that doesn't have an iconic corporation or person behind it, rallying and focusing the community? That condition is actually quite rare.
  - mojuba 18y ago
    SBCL: is it as good as Allegro? I haven't seen an IDE as beautiful and comfortable to work with as Allegro (although I'm barely a Lisp hacker, just a fan).
    - KirinDave 18y ago
      I don't know what metrics you'd use to decide it's "better". Certainly the native code generation on SBCL is pretty darn good. I think Allegro has better tools and libraries, but SBCL is certainly suitable for professional work. At this point we're talking about matters of degrees.
  - rplevy 18y ago
    I think the reason why that is not likely to catch on is that most Lisp hackers enjoy using an editor that can itself be hacked in Lisp. Also, emacs is one of the most stable pieces of software in history. It will probably live forever and continue to evolve. Also, I think that Lisp isn't really suffering. As far as I can tell there has been increasing interest in Lisp. Even without that trend continuing, Lisp is such a masterpiece (and it continues to develop with new innovations), that it is almost definitely here to stay, regardless of whether it becomes a trendy language to use again. The advantages in performance (say SBCL's for example) and expressiveness over other dynamic languages like Ruby and Python afford Lisp developers an actual advantage, as opposed to a merely perceived one.
  - narag 18y ago
    >It's SBCL. And we have really excellent models for tools. If someone would extract SLIME from emacs and into an editor with less history behind it and more popular appeal, you'd have most of the tools you need. SBCL port to Windows is a work in progress. And what you're saying about tools is more or less that they doesn't exist... yet. And comparing what you need to know and set up to start working with open source Lisp and other environments is... not fair.
    - KirinDave 18y ago
      > SBCL port to Windows is a work in progress. So what? Lots of software gets written without ever touching Windows. I've based my entire career on it. An environment can certainly be mature and commercial quality without being multi-platform. > And comparing what you need to know and set up to start working with open source Lisp and other environments is... not fair. I do wish that SBCL had a "everything you need to know in 10 minutes or less", but it comes with ASDF and ASDF-INSTALL pre-configured. I mean, the amount of stuff you need to know to start working with Java is pretty darn daunting as well, given that you need massive assistance frameworks, and there is no good central package repo. The reason more people go into Java is that they have colleges introducing them to it. Yes, more could be done. But I think commercial, professional work is possible right now given the state of SBCL.
      - pchristensen 18y ago
        ITA Software is proof of that possibility. Dan Weinreb has written about that often, on comp.lang.lisp and his own blog.
      - narag 18y ago
        >So what? Lots of software gets written without ever touching Windows. Except most people doesn't want to write "lots of software", but software that fills certain requirements, often customers' requirements: run on Windows, do threads, have a rich GUI, minimize to tray, detect screensaver, interface to word processor, customer's database, customer's crappy ERP, etc. >the amount of stuff you need to know to start working with Java is pretty darn daunting as well, I've done both starts (Java and Lisp) and I can't disagree more. >But I think commercial, professional work is possible right now given the state of SBCL. It depends on your requirements. For some environments that's not true at all.
gaius 18y ago
I'm not sure that's true. Right now I am working on GNU CLISP with bindings to Oracle and a GUI (via Ltk). Sure it's not trivial, but it's not impossible either.
- narag 18y ago
  That could fill the requirements for "enterprise" software, the kind of work that people happily convert to web apps. Making desktop software can be much much more demanding.

DaniFong 18y ago

I don't claim that this is the only way to learn how to write compilers, it's just one surprisingly effective way. And I actually think skill in generalised, simple, macro-type compilers is more useful, generally, than knowing the ins and outs of optimizing compilers, but that's just me. For the sake of discussion, what are some of the important things for optimizing compilers that you can't do with this approach? I'm having trouble finding any -- both in a literal sense of possibility, and from a practical standpoint.

mojuba 18y ago
I may be underestimating macros but I can't imagine any optimization technique that can be done with them when generating low-level output, be it native or virtual machine code. Can you demonstrate (just theoretically, of course) copy propagation, removing loop invariants, automatic inlining of functions, to name a few?
- gruseom 18y ago
  It's not hard to see how macros could do this. Macros just transform code into other code, which is what optimizations do too. For example, if you have an s-expr representing a loop, imagine a function which accepts that s-expr and returns a new one with invariants moved outside the loop body.
  - mojuba 18y ago
    Ok, but my original point was, "you won't be able to write a compiler for anything other than a (suboptimal) Lisp derivative" (I was probably wrong about suboptimal though, because it only applies to the compiler)
    - gruseom 18y ago
      Well, the easy way to use Lisp macros in a compiler is to start with an s-expr and successively transform it until you've got something that the target platform accepts. "Starting with an s-expr" implies that your source code is in Lisp format. So in that sense, what you're compiling is a Lisp derivative. Is that what you mean? No doubt there are clever ways to leverage Lisp macros in compilers beyond this approach. But most people who like Lisp macros wouldn't bother. They'd just do it the easy way. Edit: my understanding of what DaniFong is getting at is that in Lisp programming, there is no longer a barrier between application development and compiler development. This makes possible a lot of powerful things that you can't do when the application is written in a fixed language by different programmers than the ones who write the compiler. It's a different point, but one I find very interesting. It's not obvious what belongs to application development and what belongs to language development once this technical (and organizational) barrier is removed.