10 ms·

Show HN: A Rust macro that parses Java-like syntax and runs it as a Rust program

195 points by jD91mZM2 8y ago

deleted 8y ago
[deleted]
azhenley 8y ago
I don't know what I would do with this yet, but this is certainly a cool feat!
jandrese 8y ago
But the issue with Rust ins't the syntax, it's where you need to change your entire thought model to match what Rust expects. Well, ok, the syntax can be pretty horrific at times, but going to something Java-like doesn't seem like much improvement.
- tbodt 8y ago
  This isn't really intended to be used for any actual code, it's just a fun exercise.
- mmirate 8y ago
  The issue contrariwise with most other languages that can achieve similar performance characteristics as Rust, is that if you don't think about your code in a similar level of detail as the Rust compiler can "think" about Rust code ... then you get undefined behavior, memory leaks, and all sorts of really exciting run-time bugs that your tests may or may not detect.
zerb 8y ago
I was actually just toying with a similar thing myself. I'm experimenting with a system that generates abilities for a game within certain design parameters, then tests the games' balance by running lots of example games. I figured compiling a DSL via macro compiled down would be worth the upfront cost given how much testing I planned on giving each generated variant. Thanks for laying some groundwork in this part!
dleslie 8y ago
How well do LSP implementations cope with complex macros like this?
- jD91mZM2 8y ago
  I think that I actually managed to get the language server to take 100% CPU once, I think it was stuck recursing. Don't want to dig deeper because I had to REISUB my computer, heh. But other than that one very rare time it works pretty well. Can't always show you the most useful things, because sometimes it depends on invocation and line number but it's doing well with syntax errors.
- valarauca1 8y ago
  In short they don’t, neither does the compiler. I wrote a rather large macro to auto-generate a Java-Ops parser. The I spent the next 4 days re-writing it so I didn’t have to give the Rust compiler 4GiB of stack just to compile. Deeply recursive structures (boardering on 1mil+ recursion depth gets rough)
- dmead 8y ago
  this is the wrong way to do translation problems in lisp.
monocasa 8y ago
Reminds me of this cool hack: a 6502 assembler as a rust macro. https://github.com/jonas-schievink/rustasm6502 https://github.com/jonas-schievink/rustasm6502
dan-robertson 8y ago
There are a few things that are annoying about writing macros* (in particular when I last checked there was no way to break the hygiene which is something that is occasionally useful for eg making lots of functions that follow a similar template) in Rust but in general I think they strike a really nice balance for a language with complex syntax: 1. Macros only happen in certain points and don’t need to be known to be parsed so parsers can be made which don’t need to understand macro syntax (or know which macros are in scope before parsing). I think this is better than e.g. ocamlp4 (writing a plugin for an alternative compiler frontend) or perl6 (subclassing the grammar of the language and adding your own rules which is a bit nice but also a bit crazy. Perl has the advantage of being so dynamic that any kind of analysis tools are not written to parse the code but to introspect it). 2. Macros don’t require any code to be compiled and then run at compile time. This also means that macros are not programs that have to understand the type of the ast and operate on it, something that would be particularly painful in rust for writing something simple. Languages where macros are not like this are ocaml with ppx, Rust, and Julia (where the ast is super vaguely defined and hard to work with, especially in the way it magically deals with module scoping and quoting, quaisiquoting, and “hygiene”) 3. The input to a macro is not a parse tree but rather a “token tree” which is essentially a sequence of tokens where delimiters are matched and converted into “tokens,” something that feels vaguely like TeX. the only other macro system that works at all like this is Lisp but that’s because the language (and ast) is sequences of tokens with parenthesised bits grouped into single tokens so it doesn’t really count. 4. Macros are based on rewrite rules. These make it quite hard to write macros but I think they’re the best we have for not needing to invent a huge language just to write macros in, and for having macros be hygienic. Scheme macros are also rewrite based but it is easier to understand how they work because the shape of their input it more obvious One thing that makes me sad about rust macros is having to specify the “type” of things that are matched against, e.g. “expr”. I can see why it’s needed but it does feel somewhat limiting. Another thing is that macro invocations look different from the rest of the syntax so a macro feels like it’s tacked on rather than a part of the language. * by macro I mean the kind made of rewrite rules and not compiler plugins
- jD91mZM2 8y ago
  I gotta say what I really like about macros are that ( and )s are matched. If you say: while ($($cond:tt)*) { You'll actually match both while (true) { and while (5 * (2 + 3)) { . It counts the parenthesis :D
mpweiher 8y ago
Cool. On the other hand, projects like these suggest to me that Macro systems like Rust's are a bit too powerful. Not that I know how to make it just enough less powerful.
- simias 8y ago
  In my experience Rust macros are painful enough to write to deter people from abusing them too much. Whether that's a bug or a feature, I'll let you decide.
- kibwen 8y ago
  Rust's macro system is actually quite limited (I don't know if it's Turing-complete, but the artificial recursion limit is low enough that even if it is, it's not going to be all that abusable); macros are mostly just good for getting rid of repetitive boilerplate, and community norms (and implementation papercuts) discourage people from reaching for macros willy-nilly.
  - gamegoblin 8y ago
    They are Turing complete, see https://github.com/durka/brainmunch https://github.com/durka/brainmunch
    - Dylan16807 8y ago
      Not that "Turing complete" matters much. Raw text find-and-replace is Turing complete.
      - tomsmeding 8y ago
        Like the language /// ? (https://esolangs.org/wiki//// https://esolangs.org/wiki////)
        Dylan16807 8y ago
        Interesting, but my thoughts were along a simpler route of making a literal Turing machine out of text. Represent your tape with digits, and your head with a letter. Have an arrow from the letter to the active cell. [ 0 0 0 1 1 A->2 2 2 ] Then the text replacements look like this A->2 becomes 1 A-> 2<-A becomes 1 A-> B->2 becomes <-D 3 2<-B becomes <-D 3 So two text replacements for every state/symbol combo. And the halt state has no replacements. Easy! (Plus a replacement from [<- to [0<- and from ->] to ->0] to make the tape infinite.)
sharpercoder 8y ago
Would it be possible to write a compiler from Java to Rust? C# to rust? The goal then would be to get fast(er) execution times.
- timClicks 8y ago
  Rust is a good base for many languages, but in these cases it would probably still imply implementing the JVM/the CLR, rather than "Java" or "C#". There are probably research experiments that have involved ahead of time compilation, but if they had succeeded then I expect most people would have used them. GraalVM looks very promising.
- dan-robertson 8y ago
  Rust get’s a speed advantage over those languages from three things: 1. Being safer for writing concurrent programs and therefore having programs which are concurrent 2. (mostly) Type safe memory management which means most things are allocated on the stack and those things that are allocated in the heap don’t need to be gc’d (but note that gc doesn’t mean slow memory management but it is harder to have a shared heap in concurrent programs as some synchronisation is needed), and rust programs are typically written in a way that doesn’t allocate much on the heap. 3. Specialisation of generic code to specific data types 1 and 2 are hard for Java and c# as programs are not normally written in ways which would satisfy rust’s checks and so a compiler would just compile these to runtime checks like the c# or Java compilers would use. For number 3. C# already does this. Note that actually if C# or Java program were translated into a rust program which does the same work then the Rust program would probably be slower because the Java and C# compilers are much better at optimising programs written in the Java/C# style. A fourth advantage rust gets (part of 3 and part of the type system) is a lack of subclassing and vtables which are typically fast at runtime but hard to inline. One way to make a Java or C# program faster is to be conscious of the hidden work being done and trying to write programs in a way that safety checks can be eliminated and allocation can be made automatic.
- fasquoika 8y ago
  > Would it be possible to write a compiler from Java to Rust? C# to rust? It's possible to write a compiler from any Turing-complete language to any other Turing-complete language > The goal then would be to get fast(er) execution times. You'd improve startup times from not having to warm up the JIT, but otherwise it would very unlikely you'd get a noticeable performance improvement. The speed of a language is at least as dependent on its execution model and idioms as its compiler. Not to mention that the JVM already does plenty of optimizations
gwbas1c 8y ago
Perhaps I don't understand something? (I've only done a very small amount of Rust.) What exactly does this do? Let you write Java-style code in Rust? Why would I use this? (Is it because writing code like this is easier than straight rust?)
- fasquoika 8y ago
  > Let you write Java-style code in Rust? Yep > Why would I use this? You wouldn't. It's a toy
dionna767675 8y ago
Get 24/7 Unlimited Roadside Assistance For As Little As $9.95 a Month Unlimited 24/7 roadside assistance. (Whether you’ve got a flat tire, out of gas, locked out of your car, or need towing, just call us, and we’ll be there in 45 minutes or less) Get reimbursed for calling your own tow company up to $100. If your car is undriveable due to an accident we will cover your rental car up to $500. Or, if you’re far from home, we’ll pay up to $500 for food, lodging or transportation click below for more information https://roadsidekings.com https://roadsidekings.com

195 points by jD91mZM2 8y ago

deleted 8y ago

[deleted]

azhenley 8y ago

I don't know what I would do with this yet, but this is certainly a cool feat!

jandrese 8y ago

But the issue with Rust ins't the syntax, it's where you need to change your entire thought model to match what Rust expects. Well, ok, the syntax can be pretty horrific at times, but going to something Java-like doesn't seem like much improvement.

tbodt 8y ago
This isn't really intended to be used for any actual code, it's just a fun exercise.
mmirate 8y ago
The issue contrariwise with most other languages that can achieve similar performance characteristics as Rust, is that if you don't think about your code in a similar level of detail as the Rust compiler can "think" about Rust code ... then you get undefined behavior, memory leaks, and all sorts of really exciting run-time bugs that your tests may or may not detect.

zerb 8y ago

I was actually just toying with a similar thing myself. I'm experimenting with a system that generates abilities for a game within certain design parameters, then tests the games' balance by running lots of example games. I figured compiling a DSL via macro compiled down would be worth the upfront cost given how much testing I planned on giving each generated variant. Thanks for laying some groundwork in this part!

dleslie 8y ago

How well do LSP implementations cope with complex macros like this?

jD91mZM2 8y ago
I think that I actually managed to get the language server to take 100% CPU once, I think it was stuck recursing. Don't want to dig deeper because I had to REISUB my computer, heh. But other than that one very rare time it works pretty well. Can't always show you the most useful things, because sometimes it depends on invocation and line number but it's doing well with syntax errors.
valarauca1 8y ago
In short they don’t, neither does the compiler. I wrote a rather large macro to auto-generate a Java-Ops parser. The I spent the next 4 days re-writing it so I didn’t have to give the Rust compiler 4GiB of stack just to compile. Deeply recursive structures (boardering on 1mil+ recursion depth gets rough)
dmead 8y ago
this is the wrong way to do translation problems in lisp.

monocasa 8y ago

Reminds me of this cool hack: a 6502 assembler as a rust macro. https://github.com/jonas-schievink/rustasm6502 https://github.com/jonas-schievink/rustasm6502

dan-robertson 8y ago

There are a few things that are annoying about writing macros* (in particular when I last checked there was no way to break the hygiene which is something that is occasionally useful for eg making lots of functions that follow a similar template) in Rust but in general I think they strike a really nice balance for a language with complex syntax: 1. Macros only happen in certain points and don’t need to be known to be parsed so parsers can be made which don’t need to understand macro syntax (or know which macros are in scope before parsing). I think this is better than e.g. ocamlp4 (writing a plugin for an alternative compiler frontend) or perl6 (subclassing the grammar of the language and adding your own rules which is a bit nice but also a bit crazy. Perl has the advantage of being so dynamic that any kind of analysis tools are not written to parse the code but to introspect it). 2. Macros don’t require any code to be compiled and then run at compile time. This also means that macros are not programs that have to understand the type of the ast and operate on it, something that would be particularly painful in rust for writing something simple. Languages where macros are not like this are ocaml with ppx, Rust, and Julia (where the ast is super vaguely defined and hard to work with, especially in the way it magically deals with module scoping and quoting, quaisiquoting, and “hygiene”) 3. The input to a macro is not a parse tree but rather a “token tree” which is essentially a sequence of tokens where delimiters are matched and converted into “tokens,” something that feels vaguely like TeX. the only other macro system that works at all like this is Lisp but that’s because the language (and ast) is sequences of tokens with parenthesised bits grouped into single tokens so it doesn’t really count. 4. Macros are based on rewrite rules. These make it quite hard to write macros but I think they’re the best we have for not needing to invent a huge language just to write macros in, and for having macros be hygienic. Scheme macros are also rewrite based but it is easier to understand how they work because the shape of their input it more obvious One thing that makes me sad about rust macros is having to specify the “type” of things that are matched against, e.g. “expr”. I can see why it’s needed but it does feel somewhat limiting. Another thing is that macro invocations look different from the rest of the syntax so a macro feels like it’s tacked on rather than a part of the language. * by macro I mean the kind made of rewrite rules and not compiler plugins

jD91mZM2 8y ago
I gotta say what I really like about macros are that ( and )s are matched. If you say: while ($($cond:tt)*) { You'll actually match both while (true) { and while (5 * (2 + 3)) { . It counts the parenthesis :D

mpweiher 8y ago

Cool. On the other hand, projects like these suggest to me that Macro systems like Rust's are a bit too powerful. Not that I know how to make it just enough less powerful.

simias 8y ago
In my experience Rust macros are painful enough to write to deter people from abusing them too much. Whether that's a bug or a feature, I'll let you decide.
kibwen 8y ago
Rust's macro system is actually quite limited (I don't know if it's Turing-complete, but the artificial recursion limit is low enough that even if it is, it's not going to be all that abusable); macros are mostly just good for getting rid of repetitive boilerplate, and community norms (and implementation papercuts) discourage people from reaching for macros willy-nilly.
- gamegoblin 8y ago
  They are Turing complete, see https://github.com/durka/brainmunch https://github.com/durka/brainmunch
  - Dylan16807 8y ago
    Not that "Turing complete" matters much. Raw text find-and-replace is Turing complete.
    - tomsmeding 8y ago
      Like the language /// ? (https://esolangs.org/wiki//// https://esolangs.org/wiki////)
      - Dylan16807 8y ago
        Interesting, but my thoughts were along a simpler route of making a literal Turing machine out of text. Represent your tape with digits, and your head with a letter. Have an arrow from the letter to the active cell. [ 0 0 0 1 1 A->2 2 2 ] Then the text replacements look like this A->2 becomes 1 A-> 2<-A becomes 1 A-> B->2 becomes <-D 3 2<-B becomes <-D 3 So two text replacements for every state/symbol combo. And the halt state has no replacements. Easy! (Plus a replacement from [<- to [0<- and from ->] to ->0] to make the tape infinite.)

sharpercoder 8y ago

Would it be possible to write a compiler from Java to Rust? C# to rust? The goal then would be to get fast(er) execution times.

timClicks 8y ago
Rust is a good base for many languages, but in these cases it would probably still imply implementing the JVM/the CLR, rather than "Java" or "C#". There are probably research experiments that have involved ahead of time compilation, but if they had succeeded then I expect most people would have used them. GraalVM looks very promising.
dan-robertson 8y ago
Rust get’s a speed advantage over those languages from three things: 1. Being safer for writing concurrent programs and therefore having programs which are concurrent 2. (mostly) Type safe memory management which means most things are allocated on the stack and those things that are allocated in the heap don’t need to be gc’d (but note that gc doesn’t mean slow memory management but it is harder to have a shared heap in concurrent programs as some synchronisation is needed), and rust programs are typically written in a way that doesn’t allocate much on the heap. 3. Specialisation of generic code to specific data types 1 and 2 are hard for Java and c# as programs are not normally written in ways which would satisfy rust’s checks and so a compiler would just compile these to runtime checks like the c# or Java compilers would use. For number 3. C# already does this. Note that actually if C# or Java program were translated into a rust program which does the same work then the Rust program would probably be slower because the Java and C# compilers are much better at optimising programs written in the Java/C# style. A fourth advantage rust gets (part of 3 and part of the type system) is a lack of subclassing and vtables which are typically fast at runtime but hard to inline. One way to make a Java or C# program faster is to be conscious of the hidden work being done and trying to write programs in a way that safety checks can be eliminated and allocation can be made automatic.
fasquoika 8y ago
> Would it be possible to write a compiler from Java to Rust? C# to rust? It's possible to write a compiler from any Turing-complete language to any other Turing-complete language > The goal then would be to get fast(er) execution times. You'd improve startup times from not having to warm up the JIT, but otherwise it would very unlikely you'd get a noticeable performance improvement. The speed of a language is at least as dependent on its execution model and idioms as its compiler. Not to mention that the JVM already does plenty of optimizations

gwbas1c 8y ago

Perhaps I don't understand something? (I've only done a very small amount of Rust.) What exactly does this do? Let you write Java-style code in Rust? Why would I use this? (Is it because writing code like this is easier than straight rust?)

fasquoika 8y ago
> Let you write Java-style code in Rust? Yep > Why would I use this? You wouldn't. It's a toy

dionna767675 8y ago

Get 24/7 Unlimited Roadside Assistance For As Little As $9.95 a Month Unlimited 24/7 roadside assistance. (Whether you’ve got a flat tire, out of gas, locked out of your car, or need towing, just call us, and we’ll be there in 45 minutes or less) Get reimbursed for calling your own tow company up to $100. If your car is undriveable due to an accident we will cover your rental car up to $500. Or, if you’re far from home, we’ll pay up to $500 for food, lodging or transportation click below for more information https://roadsidekings.com https://roadsidekings.com