7 ms·

Nimrod: A new approach to metaprogramming

218 points by ryeguy 13y ago

eliah-lakhin 13y ago
Very interesting work. By the way, I'm planning to work on the project with the close purposes, but I have started with a more low-level tool first. My first work is a library to construct incremental parsers: https://github.com/Eliah-Lakhin/papa-carlo https://github.com/Eliah-Lakhin/papa-carlo The idea is that the language's compiler based on this library will be easier integrated into the exist code editors and IDEs. Maybe we should somehow unite our efforts with the author of Nimrod?
fxtentacle 13y ago
nimrod sounds even better than google go. why did you start it? what's the desired program that made you design nimrod?
wwwtyro 13y ago
Cool project. How does it accomplish the realtime GC max pauses of 1-2 ms after compiling to javascript? Wouldn't the js implementation's GC have the final say on GC pauses? Thanks.
- TylerE 13y ago
  I believe the JS target is considered a bit of a demo/toy - e.g. large swathes of the stdlib are unavailable.
- RyanZAG 13y ago
  Just being optimistic, probably. You could write a fairly trivial program that builds and discards massive trees that will force GC to take greater than 1-2ms simply on freeing memory. Maybe they mean in regular operation or something - but then what is regular? I guess they have to claim impossible feats to get interest given just how many languages there are already.
  - pcwalton 13y ago
    It's deferred reference counting, so the 1-2ms claim is perfectly reasonable because all it has to scan is the stack (though if they want to break cycles they will have to do something costlier). The downsides are all of the downsides of reference counting, plus the fact that last I looked Nimrod's GC is not thread safe. That last one is a huge downside IMHO. I think DRC is interesting but I'm not much of a fan of it, honestly: it doesn't solve the big downsides of RC (cycles and thread safety being super expensive) while giving up RC's biggest advantage (prompt reclamation).
    - RyanZAG 13y ago
      Deferred means you eventually have to do the work. Same for reference counting - when you free that last element it will have to actually do the freeing. If you've filled up ram and are allocating objects fast then you can either crash or do some garbage collection and if the tree is large it will have to take more than 1-2ms.
      - pcwalton 13y ago
        Yes, you can always overrun an incremental garbage collector, forcing it to start collecting more often to hit the deadline. Each collection may take only a couple of milliseconds, but there's no garbage collector that can say "no more than 1-2 ms per frame" in the case of a game, for example. If you bound max pause times, you may have to pause more often. There's no free lunch.
      - ryeguy 13y ago
        I think the slides are saying you have the ability to pass that option to the runtime. You can specify an upper bound for gc time, see here: http://nimrod-lang.org/gc.html http://nimrod-lang.org/gc.html
      - rbehrends 13y ago
        Deferred means you eventually have to do the work. Not quite. What you save with deferred reference counting is what is generally the biggest cost of reference counting: assigning references to local variables and having local variables go out of scope. Deferred reference counting only requires an RC update if you store a reference in a heap object or in a global variable. You will, of course, necessarily incur the overhead for allocation and deallocation, which one can make cheaper (bump allocation in generational and copying GC, memory regions) but can't entirely get rid of, especially in non-copying/compacting schemes.
- lars 13y ago
  I have no idea what Nimrod is doing, but I suppose one could output JS where a heap is simulated through a TypedArray, such that the JS GC only has a single memory reference to keep track of. Then you could give guarantees on GC performance, given reasonable assumptions about the JS runtime.
  - pcwalton 13y ago
    There's no way to scan the stack in that case to find the root set, though, unless you spill all pointers to the typed array at safe points (which costs performance).
- rcthompson 13y ago
  Probably by compiling to JS code that avoids the JS garbage collector, the way asm.js does.
ibudiallo 13y ago
Impossible to navigate the site on mobile . For some reason it says swipe so it is supposed to work well on mobile device.
- twotwotwo 13y ago
  It's using http://nimrod-lang.org/talk01/Slidy2/scripts/slidy.js http://nimrod-lang.org/talk01/Slidy2/scripts/slidy.js -- that does indeed seem to try to support swipe navigation on mobile, but I had trouble with it on Chrome Beta for Android.
ericb 13y ago
This looks really neat! Looks like a cross-platform language thats an even more acceptable lisp than ruby (hello macros!), with multiple build-language targets including C/javascript/C++ and a fast garbage collector? Like Go, but even cooler?
- ericb 13y ago
  My enthusiasm is greatly lessened. The IDE doesn't compile on Mac OS X. Even worse, the developers don't care, and keep closing the issue. https://github.com/nimrod-code/Aporia/issues/37#ref-issue-17624074 https://github.com/nimrod-code/Aporia/issues/37#ref-issue-17... As a rule, I avoid any project where the developers have too much of the it-doesn't-work-and-I-don't-care attitude, mainly because you can't build on top of a foundation that isn't maintained fastidiously--you'll waste all your time fixing the foundation and won't get productive work done. Culture matters.
  - mortyseinfeld 13y ago
    Don't worry. You'd be better off using a normal text editor anyway. GTK+ is flaky on non Linux/BSD systems. They should note what some of the OpenDylan folks are doing and write a plugin for Intellij.
progman 13y ago
Some very interesting features of Nimrod which I would like to emphasize: - clean Pythonic syntax (whitespace relevant but tabs are forbidden) - native Perl syntax for regular expressions (slide 42) - subrange types as in Ada - set types as in Pascal - strings as case (switch) selectors - easy C interface with automatic type conversion (only functions as parameters need additional compiler pragmas) - typed macros (templates) - object code: native, C, Javascript (probably more to come) I am working with Nimrod right now, and it is really one of the most productive languages that I ever encountered. It seems to mix the best features of other popular languages. Nimrod actually is where Python should be. I wonder if it is suitable for small embedded systems also. By the way, one important difference to Perl regular expressions is that re"..." always parses the whole expression where Perl parses partially. Fortunately Perl's behavior can easily be simulated with a template.
- TylerE 13y ago
  For embedded use it's probably ok, anywhere C is ok. It statically links the bits of the stdlib you use but dead code elimination seems to work pretty well. A simple word counting script I just wrote compiles down to about 52k on 64bit OS X. (Src: https://gist.github.com/tylereaves/7711302 https://gist.github.com/tylereaves/7711302)
  - progman 13y ago
    Thanks - but your example is Python code ??
    - TylerE 13y ago
      No, it's Nimrod. I set it to python in gist because that's the closest to nimrod for syntax highlighting purposes. Edit: Apparenetly not...there are alot more modes in there but the UI exposing them is easy to miss
  - flyinglizard 13y ago
    I also wonder about Nimrod on embedded. The dealbreaker is dynamic memory allocation; if Nimrod does that as a non-negotiable part of its runtime, it can't be used.
rdtsc 13y ago
A GC with a deadline is what caught my attention, seemingly can specify a max pause time (in ms) and GC wont' take longer than that. That is very good for soft realtime stuff -- games, audio processing. Also, Nimrod has a very nice library. Definitely has enough "batteries" in it to get started. http://nimrod-lang.org/lib.html http://nimrod-lang.org/lib.html Just a few impressive ones: http client, server, json parsing, actor support, redis db driver, zmq, cairo graphics, OpenGL wrappers, even Lua and Python support. That looks very appealing, anyone using, what is your experience with it?
- rbehrends 13y ago
  I have been using Nimrod for some personal projects and internal tools since about version 0.9.0. It is quite usable -- especially the current Github version -- but as you may expect from a 0.x project, it still has some rough edges. These haven't stopped me from being productive with it, but you should be prepared for the occasional "huh?" situation (the most common problem I've had was that when I experimented with and learned the language, malformed code that I inadvertently wrote would trigger a compiler error, e.g. an illegal AST exception). If you want to do some serious experimentation with Nimrod, I recommend using 0.9.3 (the Github version). It is considerably more mature and stable than 0.9.2. The one oddity that I'm still getting used to is Nimrod's preference for value types. I.e., the string and seq[T] types use copying for assignment [1] by default, when most other languages assume reference semantics (you can use reference assignment instead, but have to say so explicitly). This may create inadvertent overhead if you are unaware of it, but also avoids some pitfalls you otherwise may run into with mutable strings. As for the default GC, it's not that you can actually guarantee a maximum pause time [2]. The GC uses deferred reference counting (i.e., eliding most RC operations and batching the ones that are needed). As long as your data structures are acyclic, that allows for a very low upper bound on GC pause times. Pause times for dealing with cyclic structures depend on the size of the cycles involved (and the size of data structures attached to those cycles). A practical workaround for when you have to deal extensively with cyclic data structures is to make use of the fact that each thread has its own heap: thus, putting all time-critical stuff in one thread where cyclic data structures aren't used, and the rest in another thread. [1] Copying is not used for passing arguments to a procedure or for returning a result, only actual assignment. [2] You can influence it to some extent by changing the value of ZctThreshold in system/gc.nim, which specifies the size of the zero count table. Each GC iteration will still have to scan the stack, of course. Edit: Oops, to correct myself, Nimrod does actually have a GC_setMaxPause() procedure, though I still don't think it can guarantee hard upper bounds on pause times.
goggles99 13y ago
Am I in a time warp? Nimrod is over 5 years old. Yet most comments here think it's new and groundbreaking with statements like nimrod sounds even better than google go. Nimrod is a cool language, but am I missing something here? Original site with almost same description verbiage: https://web.archive.org/web/20110704041631/http://force7.de/nimrod/ https://web.archive.org/web/20110704041631/http://force7.de/...
- gfodor 13y ago
  Why is it confusing that an obscure programming language, on any given day, will be seen for the first time by a lot of people when it gets onto the front page of HN?
- evincarofautumn 13y ago
  That’s normal. Languages often take five or ten years to attain any kind of popularity. It’s probably still new to many people, and even if they’ve heard of it before, they probably haven’t tried it yet.
rcthompson 13y ago
It says the GC is implemented in Nimrod. Does the GC code avoid using the GC itself?
- drakeandrews 13y ago
  I presume by allocating memory yourself. It does mention that if you don't use the GC, it is removed from the final program.
discreteevent 13y ago
I think I could get along with this language.
twotwotwo 13y ago
It has a lot of cool stuff, and I'm intrigued by its approach to threading. (Per-thread heaps and message passing, with an unsafe shared heap if you need it.) How does dynamic typing work in Nimrod, if it has it? What's the equivalent of being able to call Write() on any io.Writer in Go, for example? [Edit: just found the explanation at http://nimrod-lang.org/tut2.html#dynamic-dispatch http://nimrod-lang.org/tut2.html#dynamic-dispatch but have not yet read and digested.] I'd love to see some open-source projects that've been written in it--the Nimrod tools and stdlib are obviously a good start. I think one of the things that really helped Go was having lots of n00b-friendly content (the tour, a short "spec", Effective Go doc, blog posts, talks); you could call it "marketing", but it really helped me, at least.
msellout 13y ago
The html slideshow fails to work on Chrome on Samsung Galaxy S4. It tells me to swipe left to move to the next slide, but nothing happens.
eonil 13y ago
Nimrod is the one which is closest to my hypothetic ideal language in features.
- dirkk0 13y ago
  Yes, that's what I thought, too. My only complaint is that my internal Python (awesome) vs. JavaScript (not so awesome but becoming standard) war gets tackled from a third side now. Another language to care about - not exactly what I needed. Having said that: Nimrod looks beautiful and maybe it's time to hop on the bandwagon.
humpty44 13y ago
This language looks very very interesting. Expressive and efficient languages are hard to come by. I really like the name too.
catwell 13y ago
Every time I see that name (Nimrod) I think it is about http://www.messagelab.monash.edu.au/Nimrod http://www.messagelab.monash.edu.au/Nimrod Why can't people find original names for software? :(

218 points by ryeguy 13y ago

Very interesting work. By the way, I'm planning to work on the project with the close purposes, but I have started with a more low-level tool first. My first work is a library to construct incremental parsers: https://github.com/Eliah-Lakhin/papa-carlo https://github.com/Eliah-Lakhin/papa-carlo The idea is that the language's compiler based on this library will be easier integrated into the exist code editors and IDEs. Maybe we should somehow unite our efforts with the author of Nimrod?

fxtentacle 13y ago

nimrod sounds even better than google go. why did you start it? what's the desired program that made you design nimrod?

wwwtyro 13y ago

Cool project. How does it accomplish the realtime GC max pauses of 1-2 ms after compiling to javascript? Wouldn't the js implementation's GC have the final say on GC pauses? Thanks.

TylerE 13y ago
I believe the JS target is considered a bit of a demo/toy - e.g. large swathes of the stdlib are unavailable.
RyanZAG 13y ago
Just being optimistic, probably. You could write a fairly trivial program that builds and discards massive trees that will force GC to take greater than 1-2ms simply on freeing memory. Maybe they mean in regular operation or something - but then what is regular? I guess they have to claim impossible feats to get interest given just how many languages there are already.
- pcwalton 13y ago
  It's deferred reference counting, so the 1-2ms claim is perfectly reasonable because all it has to scan is the stack (though if they want to break cycles they will have to do something costlier). The downsides are all of the downsides of reference counting, plus the fact that last I looked Nimrod's GC is not thread safe. That last one is a huge downside IMHO. I think DRC is interesting but I'm not much of a fan of it, honestly: it doesn't solve the big downsides of RC (cycles and thread safety being super expensive) while giving up RC's biggest advantage (prompt reclamation).
  - RyanZAG 13y ago
    Deferred means you eventually have to do the work. Same for reference counting - when you free that last element it will have to actually do the freeing. If you've filled up ram and are allocating objects fast then you can either crash or do some garbage collection and if the tree is large it will have to take more than 1-2ms.
    - pcwalton 13y ago
      Yes, you can always overrun an incremental garbage collector, forcing it to start collecting more often to hit the deadline. Each collection may take only a couple of milliseconds, but there's no garbage collector that can say "no more than 1-2 ms per frame" in the case of a game, for example. If you bound max pause times, you may have to pause more often. There's no free lunch.
    - ryeguy 13y ago
      I think the slides are saying you have the ability to pass that option to the runtime. You can specify an upper bound for gc time, see here: http://nimrod-lang.org/gc.html http://nimrod-lang.org/gc.html
    - rbehrends 13y ago
      Deferred means you eventually have to do the work. Not quite. What you save with deferred reference counting is what is generally the biggest cost of reference counting: assigning references to local variables and having local variables go out of scope. Deferred reference counting only requires an RC update if you store a reference in a heap object or in a global variable. You will, of course, necessarily incur the overhead for allocation and deallocation, which one can make cheaper (bump allocation in generational and copying GC, memory regions) but can't entirely get rid of, especially in non-copying/compacting schemes.
lars 13y ago
I have no idea what Nimrod is doing, but I suppose one could output JS where a heap is simulated through a TypedArray, such that the JS GC only has a single memory reference to keep track of. Then you could give guarantees on GC performance, given reasonable assumptions about the JS runtime.
- pcwalton 13y ago
  There's no way to scan the stack in that case to find the root set, though, unless you spill all pointers to the typed array at safe points (which costs performance).
rcthompson 13y ago
Probably by compiling to JS code that avoids the JS garbage collector, the way asm.js does.

ibudiallo 13y ago

Impossible to navigate the site on mobile . For some reason it says swipe so it is supposed to work well on mobile device.

twotwotwo 13y ago
It's using http://nimrod-lang.org/talk01/Slidy2/scripts/slidy.js http://nimrod-lang.org/talk01/Slidy2/scripts/slidy.js -- that does indeed seem to try to support swipe navigation on mobile, but I had trouble with it on Chrome Beta for Android.

ericb 13y ago

This looks really neat! Looks like a cross-platform language thats an even more acceptable lisp than ruby (hello macros!), with multiple build-language targets including C/javascript/C++ and a fast garbage collector? Like Go, but even cooler?

ericb 13y ago
My enthusiasm is greatly lessened. The IDE doesn't compile on Mac OS X. Even worse, the developers don't care, and keep closing the issue. https://github.com/nimrod-code/Aporia/issues/37#ref-issue-17624074 https://github.com/nimrod-code/Aporia/issues/37#ref-issue-17... As a rule, I avoid any project where the developers have too much of the it-doesn't-work-and-I-don't-care attitude, mainly because you can't build on top of a foundation that isn't maintained fastidiously--you'll waste all your time fixing the foundation and won't get productive work done. Culture matters.
- mortyseinfeld 13y ago
  Don't worry. You'd be better off using a normal text editor anyway. GTK+ is flaky on non Linux/BSD systems. They should note what some of the OpenDylan folks are doing and write a plugin for Intellij.

progman 13y ago

Some very interesting features of Nimrod which I would like to emphasize: - clean Pythonic syntax (whitespace relevant but tabs are forbidden) - native Perl syntax for regular expressions (slide 42) - subrange types as in Ada - set types as in Pascal - strings as case (switch) selectors - easy C interface with automatic type conversion (only functions as parameters need additional compiler pragmas) - typed macros (templates) - object code: native, C, Javascript (probably more to come) I am working with Nimrod right now, and it is really one of the most productive languages that I ever encountered. It seems to mix the best features of other popular languages. Nimrod actually is where Python should be. I wonder if it is suitable for small embedded systems also. By the way, one important difference to Perl regular expressions is that re"..." always parses the whole expression where Perl parses partially. Fortunately Perl's behavior can easily be simulated with a template.

TylerE 13y ago
For embedded use it's probably ok, anywhere C is ok. It statically links the bits of the stdlib you use but dead code elimination seems to work pretty well. A simple word counting script I just wrote compiles down to about 52k on 64bit OS X. (Src: https://gist.github.com/tylereaves/7711302 https://gist.github.com/tylereaves/7711302)
- progman 13y ago
  Thanks - but your example is Python code ??
  - TylerE 13y ago
    No, it's Nimrod. I set it to python in gist because that's the closest to nimrod for syntax highlighting purposes. Edit: Apparenetly not...there are alot more modes in there but the UI exposing them is easy to miss
- flyinglizard 13y ago
  I also wonder about Nimrod on embedded. The dealbreaker is dynamic memory allocation; if Nimrod does that as a non-negotiable part of its runtime, it can't be used.

rdtsc 13y ago

A GC with a deadline is what caught my attention, seemingly can specify a max pause time (in ms) and GC wont' take longer than that. That is very good for soft realtime stuff -- games, audio processing. Also, Nimrod has a very nice library. Definitely has enough "batteries" in it to get started. http://nimrod-lang.org/lib.html http://nimrod-lang.org/lib.html Just a few impressive ones: http client, server, json parsing, actor support, redis db driver, zmq, cairo graphics, OpenGL wrappers, even Lua and Python support. That looks very appealing, anyone using, what is your experience with it?

rbehrends 13y ago
I have been using Nimrod for some personal projects and internal tools since about version 0.9.0. It is quite usable -- especially the current Github version -- but as you may expect from a 0.x project, it still has some rough edges. These haven't stopped me from being productive with it, but you should be prepared for the occasional "huh?" situation (the most common problem I've had was that when I experimented with and learned the language, malformed code that I inadvertently wrote would trigger a compiler error, e.g. an illegal AST exception). If you want to do some serious experimentation with Nimrod, I recommend using 0.9.3 (the Github version). It is considerably more mature and stable than 0.9.2. The one oddity that I'm still getting used to is Nimrod's preference for value types. I.e., the string and seq[T] types use copying for assignment [1] by default, when most other languages assume reference semantics (you can use reference assignment instead, but have to say so explicitly). This may create inadvertent overhead if you are unaware of it, but also avoids some pitfalls you otherwise may run into with mutable strings. As for the default GC, it's not that you can actually guarantee a maximum pause time [2]. The GC uses deferred reference counting (i.e., eliding most RC operations and batching the ones that are needed). As long as your data structures are acyclic, that allows for a very low upper bound on GC pause times. Pause times for dealing with cyclic structures depend on the size of the cycles involved (and the size of data structures attached to those cycles). A practical workaround for when you have to deal extensively with cyclic data structures is to make use of the fact that each thread has its own heap: thus, putting all time-critical stuff in one thread where cyclic data structures aren't used, and the rest in another thread. [1] Copying is not used for passing arguments to a procedure or for returning a result, only actual assignment. [2] You can influence it to some extent by changing the value of ZctThreshold in system/gc.nim, which specifies the size of the zero count table. Each GC iteration will still have to scan the stack, of course. Edit: Oops, to correct myself, Nimrod does actually have a GC_setMaxPause() procedure, though I still don't think it can guarantee hard upper bounds on pause times.

goggles99 13y ago

Am I in a time warp? Nimrod is over 5 years old. Yet most comments here think it's new and groundbreaking with statements like nimrod sounds even better than google go. Nimrod is a cool language, but am I missing something here? Original site with almost same description verbiage: https://web.archive.org/web/20110704041631/http://force7.de/nimrod/ https://web.archive.org/web/20110704041631/http://force7.de/...

gfodor 13y ago
Why is it confusing that an obscure programming language, on any given day, will be seen for the first time by a lot of people when it gets onto the front page of HN?
evincarofautumn 13y ago
That’s normal. Languages often take five or ten years to attain any kind of popularity. It’s probably still new to many people, and even if they’ve heard of it before, they probably haven’t tried it yet.

rcthompson 13y ago

It says the GC is implemented in Nimrod. Does the GC code avoid using the GC itself?

drakeandrews 13y ago
I presume by allocating memory yourself. It does mention that if you don't use the GC, it is removed from the final program.

discreteevent 13y ago

I think I could get along with this language.

twotwotwo 13y ago

It has a lot of cool stuff, and I'm intrigued by its approach to threading. (Per-thread heaps and message passing, with an unsafe shared heap if you need it.) How does dynamic typing work in Nimrod, if it has it? What's the equivalent of being able to call Write() on any io.Writer in Go, for example? [Edit: just found the explanation at http://nimrod-lang.org/tut2.html#dynamic-dispatch http://nimrod-lang.org/tut2.html#dynamic-dispatch but have not yet read and digested.] I'd love to see some open-source projects that've been written in it--the Nimrod tools and stdlib are obviously a good start. I think one of the things that really helped Go was having lots of n00b-friendly content (the tour, a short "spec", Effective Go doc, blog posts, talks); you could call it "marketing", but it really helped me, at least.

msellout 13y ago

The html slideshow fails to work on Chrome on Samsung Galaxy S4. It tells me to swipe left to move to the next slide, but nothing happens.

eonil 13y ago

Nimrod is the one which is closest to my hypothetic ideal language in features.

dirkk0 13y ago
Yes, that's what I thought, too. My only complaint is that my internal Python (awesome) vs. JavaScript (not so awesome but becoming standard) war gets tackled from a third side now. Another language to care about - not exactly what I needed. Having said that: Nimrod looks beautiful and maybe it's time to hop on the bandwagon.

humpty44 13y ago

This language looks very very interesting. Expressive and efficient languages are hard to come by. I really like the name too.

catwell 13y ago

Every time I see that name (Nimrod) I think it is about http://www.messagelab.monash.edu.au/Nimrod http://www.messagelab.monash.edu.au/Nimrod Why can't people find original names for software? :(