5 ms·

Show HN: Compiler using Lisp’s macro system for metaprogramming C-like languages

80 points by dis0rdr 9y ago

decafbad 9y ago
Of course it's a source-to-source compiler. Thanks for not using that moronic word.
jokoon 9y ago
I don't understand, what are lisp/scheme/racket/haskell being used for? I mean how are those languages good from a software engineering point of view? From a project management point of view? Is it possible to hear bad things about those languages? Why aren't they used? Is it because they are too hard to use, or to difficult to approach for a beginner?
- JohnStrange 9y ago
  I can speak for CommonLisp, scheme and Racket. Someone else will have to answer for Haskell. Generally, LISPs are extremely flexible, reasonably fast, compiled languages with macro systems that allow you to develop a domain specific dialect. Some of them, like CommonLisp and Racket, come with a pretty huge set of libraries, tooling and ecosystem. > Why aren't they used? They are used a lot. > Is it because they are too hard to use, or to difficult to approach for a beginner? They tend to be easy to learn but difficult to master for writing idiomatic code, i.e., have a relatively long flat learning curve. > Is it possible to hear bad things about those languages? Sure. In my personal experience, their biggest downside is the same as their biggest selling point: their flexibility. Code is generally hard to maintain and read by others. You can invent some incredibly hacky spaghetti code monstrosities in LISPs if you want to. More disadvantages: high memory use, dynamic typing and dynamic dispatch can suck big time (e.g. Racket OOP methods are mostly checked at runtime, which can lead to testing nightmares if you're not careful) Overall, CommonLisp and Racket are pretty good 'batteries included' languages, whereas some scheme implementations are extremely fast and evolved small extension and scripting languages.
- pweissbrod 9y ago
  Theyre similar to Perl in the sense that theyre extremely powerful and flexible but they tend to be highly customized to the business domain of their design. This type of languages enable a very small team of engineers to be profoundly productive but the learning curve with adopting an existing project can be quite high even for expert programmers due to the lack of consistent patterns you would expect for more Enterprise languages like Java .net or python. The end result is great success for small smart teams but lack of interchangeability and scalability for Enterprises Edit: I once wrote a closure web application for a volunteer project. The experience was wonderful and painless however finding Developers to a system maintain the code base remains a serious challenge today
  - dman 9y ago
    As an industry we dont take maintainability very seriously. You think it would be easy to maintain web apps written in more "mainstream" languages from a couple of years ago?
    - MaxBarraclough 9y ago
      The web industry constantly chases fads, sure, but that's not true of the whole software world.
- setzer22 9y ago
  Haskell and Scheme may be as far from each other as C and Javascript. You seem to suggest they are all the same thing, when in fact the most prominent thing they share is lack of mainstream (That is, a la Java mainstream) usage.
- cryptonector 9y ago
  Haskell and friends (Elm, ..) are fantastic, and perhaps the absolute best programming languages for software engineering. Except there's a small problem: Haskell is either write-only or read-only, depending on what you're looking at. That is, beautiful Haskell code is easy enough to read, but very very difficult to write, while working code you might write may not be very readable at all. Of course, that can happen in any language, but it seems to me that Haskell leads to that sort of situation very easily. On the other hand, the nice thing about Haskell is that once your code compiles it also probably does what you want it to :) I would stay away from dynamically-typed languages (e.g., the Lisp and Scheme families) for software engineering. Dynamic typing means more run-time errors, which means a higher support burden, which is very much what you want to lower when you're a software engineer. And this is why I like Haskell: it's statically-typed. (The main appeal of Lisp is its macro system, really. Haskell gives you the sort of power that the Lisp macro system gives you anyways, though at the cost of more compile-time processing.) If you can't use Haskell, then your best bets are Rust and C/C++. In any case, a software engineer almost necessarily has to be familiar with all of these, and able to use any of them. You really want a strong foundation at the lowest layers (C, and even ASM) in order to work the full stack (libraries, applications, compilers, OS). You don't have to be a full stack engineer, naturally, but it sure helps to be able to adapt to working at different layers, and for this you need a strong conceptual foundation. A lot of layers in a full stack are written in C/C++, but web front-ends involve JavaScript, which is dynamically-typed, so you'll really need to be familiar with all of these, and that's just to get started.
  - SomeHacker44 9y ago
    My opinion: If you can't use Haskell, you should definitely use a null-free language for safety in this day and age, which means Rust and definitely not C/C++.
    - cryptonector 9y ago
      Yes, but that doesn't mean that you shouldn't be able to use C/C++ as needed -- sometimes you have no choice.
- gtycomb 9y ago
  Just by the fact the a language is well engineered doesn't make it a popular language in the industry. The time and the way in which it appears makes a difference. When C appeared (from Bell labs) it fit nicely with the Unix ecosystem. There was a better engineered language already here, PLM, but the industry couldn't take and run with it as easily. The time point of Perl 5 successes is another example (and contrast that with the appearance of Perl6 more recently). Java, when it was introduced, was not well engineered at all but it came at a certain point in object orientation that the software engineering community had been striving toward.
- Q6T46nT668w6i3m 9y ago
  Are you asking why people don’t use functional programming languages? If so, I’d say it’s the same reason most people don’t use procedural programming languages: because multi-paradigm programming languages (that combine procedural and functional features) give us both with minimal cost. :)
- e12e 9y ago
  I'll just leave this here: https://github.com/arclanguage/anarki/blob/master/README.markdown https://github.com/arclanguage/anarki/blob/master/README.mar...
- tytytytytytytyt 9y ago
  > I don't understand, what are lisp/scheme/racket/haskell being used for? You won't use a language unless other people are using it for the same thing?? > I mean how are those languages good from a software engineering point of view? Easier and faster to write code that has less bugs and is easier to maintain. It's not like you can't find these things if you bother using Google for 5 mins. > Is it possible to hear bad things about those languages? They aren't as popular so there's always people complaining that it'd be too hard to hire those people or there might not be enough libraries to do what you want. Again, Google is you friend here, if you use it. > Why aren't they used? Because of, "Nobody ever got fired for choosing IBM". Doing something new = risk, risk is anathema to managers. (1) Also, most programmers first learn a single language and then second learn all the excuses to not use or learn any other language ever. > Is it because they are too hard to use, or to difficult to approach for a beginner? They should be significantly easier, but see point (1) above.
- orthecreedence 9y ago
  > I mean how are those languages good from a software engineering point of view? Common lisp: great for game programming or any type of application where it takes a while to build up a set of "state." You can redefine various aspects of the program while it's running. So, ok, there was a bug in your physics code and your player fell through the map. You can fix the physics bug while the game is still running and place your player back in the same spot. Now you don't have to recompile and try to get your game in the state it was in before. When I get back into game programming, I'm probably going to use CL extensively. I might delegate the low-level stuff to an embeddable game engine (like Orx) but all my high-level logic would be CL.
4lch3m1st 9y ago
I think the title might be misleading here. It doesn't really transpile Common Lisp to C, it requires you to write in a C-like Lisp dialect so it gets translated to C. Might as well have been an entire Lisp dialect with a compiler of its own.
- yorwba 9y ago
  I think the idea is that you write C-with-sexpr-syntax, but by embedding those s-expressions in Common Lisp, you can make use of macros written in Lisp to generate C code. Essentially, it's a saner preprocessor for C that requires you to write in funky syntax. I think this would be even more interesting if it could do the C -> s-expression transformation, so that code already written in C could integrate calls to Lisp macros without needing a full rewrite.
  - cryptonector 9y ago
    A C->S-expression compiler would have to produce very obvious and simple mappings in order for a macro system to be usable. Another way of saying this is that a simple and standard AST would be nice.
    - sedachv 9y ago
      The Vacietis (https://github.com/vsedach/Vacietis/ https://github.com/vsedach/Vacietis/) reader can be easily adapted to do that. You can see examples in the unit tests: https://github.com/vsedach/Vacietis/blob/master/test/reader-tests.lisp https://github.com/vsedach/Vacietis/blob/master/test/reader-... As-is, C block constructs get mapped directly to Common Lisp special forms like tagbody and prog because those implement a superset of C control flow semantics. Pick different names in vacietis.c and you have an AST.
- cryptonector 9y ago
  I don't know what the title was when you commented, but what I see is "Compiler using Lisp’s macro system for metaprogramming C-like languages". That doesn't imply transpiling from Common Lisp (though it does impli transpiling), and it seems like a very pithy description of what C-Mera actually does (which is also what you said in your first sentence). The nice thing about this is that you get the full power of Lisp macros, which is where the metaprogramming comes in, but it still resembles C well enough. Of course, the Lisp compiler can't actually do C type checking in this system, so you still need to map C compiler errors and warnings back to the source -- that may well turn out to be a pain (and almost certainly does). EDIT: I'm curious: why the downvote?
kruhft 9y ago
Reminds me of an older (unfinished) project of mine, sxc: https://github.com/burtonsamograd/sxc https://github.com/burtonsamograd/sxc
PaulHoule 9y ago
TLDR -- there is nothing so special about Lisp (in terms of metaprogramming) other than a syntax that is easy to parse. All languages have tools to manipulate code at the source code level, but people rarely use them. These guys did.

80 points by dis0rdr 9y ago

Of course it's a source-to-source compiler. Thanks for not using that moronic word.

I don't understand, what are lisp/scheme/racket/haskell being used for? I mean how are those languages good from a software engineering point of view? From a project management point of view? Is it possible to hear bad things about those languages? Why aren't they used? Is it because they are too hard to use, or to difficult to approach for a beginner?

JohnStrange 9y ago
I can speak for CommonLisp, scheme and Racket. Someone else will have to answer for Haskell. Generally, LISPs are extremely flexible, reasonably fast, compiled languages with macro systems that allow you to develop a domain specific dialect. Some of them, like CommonLisp and Racket, come with a pretty huge set of libraries, tooling and ecosystem. > Why aren't they used? They are used a lot. > Is it because they are too hard to use, or to difficult to approach for a beginner? They tend to be easy to learn but difficult to master for writing idiomatic code, i.e., have a relatively long flat learning curve. > Is it possible to hear bad things about those languages? Sure. In my personal experience, their biggest downside is the same as their biggest selling point: their flexibility. Code is generally hard to maintain and read by others. You can invent some incredibly hacky spaghetti code monstrosities in LISPs if you want to. More disadvantages: high memory use, dynamic typing and dynamic dispatch can suck big time (e.g. Racket OOP methods are mostly checked at runtime, which can lead to testing nightmares if you're not careful) Overall, CommonLisp and Racket are pretty good 'batteries included' languages, whereas some scheme implementations are extremely fast and evolved small extension and scripting languages.
pweissbrod 9y ago
Theyre similar to Perl in the sense that theyre extremely powerful and flexible but they tend to be highly customized to the business domain of their design. This type of languages enable a very small team of engineers to be profoundly productive but the learning curve with adopting an existing project can be quite high even for expert programmers due to the lack of consistent patterns you would expect for more Enterprise languages like Java .net or python. The end result is great success for small smart teams but lack of interchangeability and scalability for Enterprises Edit: I once wrote a closure web application for a volunteer project. The experience was wonderful and painless however finding Developers to a system maintain the code base remains a serious challenge today
- dman 9y ago
  As an industry we dont take maintainability very seriously. You think it would be easy to maintain web apps written in more "mainstream" languages from a couple of years ago?
  - MaxBarraclough 9y ago
    The web industry constantly chases fads, sure, but that's not true of the whole software world.
setzer22 9y ago
Haskell and Scheme may be as far from each other as C and Javascript. You seem to suggest they are all the same thing, when in fact the most prominent thing they share is lack of mainstream (That is, a la Java mainstream) usage.
cryptonector 9y ago
Haskell and friends (Elm, ..) are fantastic, and perhaps the absolute best programming languages for software engineering. Except there's a small problem: Haskell is either write-only or read-only, depending on what you're looking at. That is, beautiful Haskell code is easy enough to read, but very very difficult to write, while working code you might write may not be very readable at all. Of course, that can happen in any language, but it seems to me that Haskell leads to that sort of situation very easily. On the other hand, the nice thing about Haskell is that once your code compiles it also probably does what you want it to :) I would stay away from dynamically-typed languages (e.g., the Lisp and Scheme families) for software engineering. Dynamic typing means more run-time errors, which means a higher support burden, which is very much what you want to lower when you're a software engineer. And this is why I like Haskell: it's statically-typed. (The main appeal of Lisp is its macro system, really. Haskell gives you the sort of power that the Lisp macro system gives you anyways, though at the cost of more compile-time processing.) If you can't use Haskell, then your best bets are Rust and C/C++. In any case, a software engineer almost necessarily has to be familiar with all of these, and able to use any of them. You really want a strong foundation at the lowest layers (C, and even ASM) in order to work the full stack (libraries, applications, compilers, OS). You don't have to be a full stack engineer, naturally, but it sure helps to be able to adapt to working at different layers, and for this you need a strong conceptual foundation. A lot of layers in a full stack are written in C/C++, but web front-ends involve JavaScript, which is dynamically-typed, so you'll really need to be familiar with all of these, and that's just to get started.
- SomeHacker44 9y ago
  My opinion: If you can't use Haskell, you should definitely use a null-free language for safety in this day and age, which means Rust and definitely not C/C++.
  - cryptonector 9y ago
    Yes, but that doesn't mean that you shouldn't be able to use C/C++ as needed -- sometimes you have no choice.
gtycomb 9y ago
Just by the fact the a language is well engineered doesn't make it a popular language in the industry. The time and the way in which it appears makes a difference. When C appeared (from Bell labs) it fit nicely with the Unix ecosystem. There was a better engineered language already here, PLM, but the industry couldn't take and run with it as easily. The time point of Perl 5 successes is another example (and contrast that with the appearance of Perl6 more recently). Java, when it was introduced, was not well engineered at all but it came at a certain point in object orientation that the software engineering community had been striving toward.
Q6T46nT668w6i3m 9y ago
Are you asking why people don’t use functional programming languages? If so, I’d say it’s the same reason most people don’t use procedural programming languages: because multi-paradigm programming languages (that combine procedural and functional features) give us both with minimal cost. :)
e12e 9y ago
I'll just leave this here: https://github.com/arclanguage/anarki/blob/master/README.markdown https://github.com/arclanguage/anarki/blob/master/README.mar...
tytytytytytytyt 9y ago
> I don't understand, what are lisp/scheme/racket/haskell being used for? You won't use a language unless other people are using it for the same thing?? > I mean how are those languages good from a software engineering point of view? Easier and faster to write code that has less bugs and is easier to maintain. It's not like you can't find these things if you bother using Google for 5 mins. > Is it possible to hear bad things about those languages? They aren't as popular so there's always people complaining that it'd be too hard to hire those people or there might not be enough libraries to do what you want. Again, Google is you friend here, if you use it. > Why aren't they used? Because of, "Nobody ever got fired for choosing IBM". Doing something new = risk, risk is anathema to managers. (1) Also, most programmers first learn a single language and then second learn all the excuses to not use or learn any other language ever. > Is it because they are too hard to use, or to difficult to approach for a beginner? They should be significantly easier, but see point (1) above.
orthecreedence 9y ago
> I mean how are those languages good from a software engineering point of view? Common lisp: great for game programming or any type of application where it takes a while to build up a set of "state." You can redefine various aspects of the program while it's running. So, ok, there was a bug in your physics code and your player fell through the map. You can fix the physics bug while the game is still running and place your player back in the same spot. Now you don't have to recompile and try to get your game in the state it was in before. When I get back into game programming, I'm probably going to use CL extensively. I might delegate the low-level stuff to an embeddable game engine (like Orx) but all my high-level logic would be CL.

4lch3m1st 9y ago

I think the title might be misleading here. It doesn't really transpile Common Lisp to C, it requires you to write in a C-like Lisp dialect so it gets translated to C. Might as well have been an entire Lisp dialect with a compiler of its own.

yorwba 9y ago
I think the idea is that you write C-with-sexpr-syntax, but by embedding those s-expressions in Common Lisp, you can make use of macros written in Lisp to generate C code. Essentially, it's a saner preprocessor for C that requires you to write in funky syntax. I think this would be even more interesting if it could do the C -> s-expression transformation, so that code already written in C could integrate calls to Lisp macros without needing a full rewrite.
- cryptonector 9y ago
  A C->S-expression compiler would have to produce very obvious and simple mappings in order for a macro system to be usable. Another way of saying this is that a simple and standard AST would be nice.
  - sedachv 9y ago
    The Vacietis (https://github.com/vsedach/Vacietis/ https://github.com/vsedach/Vacietis/) reader can be easily adapted to do that. You can see examples in the unit tests: https://github.com/vsedach/Vacietis/blob/master/test/reader-tests.lisp https://github.com/vsedach/Vacietis/blob/master/test/reader-... As-is, C block constructs get mapped directly to Common Lisp special forms like tagbody and prog because those implement a superset of C control flow semantics. Pick different names in vacietis.c and you have an AST.
cryptonector 9y ago
I don't know what the title was when you commented, but what I see is "Compiler using Lisp’s macro system for metaprogramming C-like languages". That doesn't imply transpiling from Common Lisp (though it does impli transpiling), and it seems like a very pithy description of what C-Mera actually does (which is also what you said in your first sentence). The nice thing about this is that you get the full power of Lisp macros, which is where the metaprogramming comes in, but it still resembles C well enough. Of course, the Lisp compiler can't actually do C type checking in this system, so you still need to map C compiler errors and warnings back to the source -- that may well turn out to be a pain (and almost certainly does). EDIT: I'm curious: why the downvote?

kruhft 9y ago

Reminds me of an older (unfinished) project of mine, sxc: https://github.com/burtonsamograd/sxc https://github.com/burtonsamograd/sxc

PaulHoule 9y ago

TLDR -- there is nothing so special about Lisp (in terms of metaprogramming) other than a syntax that is easy to parse. All languages have tools to manipulate code at the source code level, but people rarely use them. These guys did.