10 ms·

Benefits of Writing a DSL in Ruby

80 points by edawerd 12y ago

CalinBalauru 12y ago
One of the best articlets I've read about the ruby dsl's was on practicingruby. You guys can check it out here https://practicingruby.com/articles/domain-specific-apis https://practicingruby.com/articles/domain-specific-apis is free now.
vinceguidry 12y ago
For any struggling to understand Ruby's metaprogramming, the rather short and succinct Metaprogramming Ruby is an excellent read. There's a method to the madness of define_method, method_missing, the various eval methods, the difference between Blocks, Procs, and Lambdas. Most of it revolves around manipulating scopes. It also shows how you can iterate from the ugliest, most powerful, most dangerous eval method by pulling out the functionality you need into less-powerful, safer manipulations. It's all Ruby code until it's actually executed, so it can all be manipulated at runtime. http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Pros/dp/1934356476 http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Pros...
- tcopeland 12y ago
  Yup this is great stuff, just be sure to get the 2nd edition which just came out: http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Facets/dp/1941222129/ref=dp_ob_title_bk http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Face... It focuses on modern Rubies, so it talks about Module#prepend and such. Nice to keep moving forwards.
- thirdtruck 12y ago
  Seconding. The book revolutionized my understanding of Ruby, and inspired me to create a whole visual novel framework: https://github.com/thirdtruck/rubyai https://github.com/thirdtruck/rubyai.
tcopeland 12y ago
That's a great writeup, well done. Along the same lines, here's the ever-entertaining Rich Kilmer talking about DSL experiences on some government projects: http://www.infoq.com/presentations/kilmer-ruby-dsls http://www.infoq.com/presentations/kilmer-ruby-dsls
habosa 12y ago
Wow. That was an amazing article. I have been doing Rails/Ruby for years and never understood any of that. Thanks for putting that together, please consider doing more tutorials for other interesting things going on in ZenPayroll.
bshimmin 12y ago
A good read for anyone just starting out with some of the more intricate parts of Ruby. I often think Ruby is unsurpassed for writing readable, English-like DSLs - perhaps because of the combination of the convenient lassitude of Ruby's syntax (not requiring brackets - in many cases - or semi-colons) and the dangerous power of its metaprogramming abilities. The only other language that perhaps comes close and immediately springs to mind, before I've had my first coffee of the day, is (appropriately) CoffeeScript. Any other takers?
- lennel 12y ago
  on the jvm i would say groovy is probably one of the better languages for dsl's. Gradle is a dsl and groovy has inbuilt Builder classes to help you in the creation of dsl's Clojure being a lisp is extremely powerful in this regard as well, but if you are programming in any lisp I would expect a fair amount of meta programming.
- 0xfaded 12y ago
  Languages with expressive type systems, Haskell likely the most mainstream, allow for deep embedding of an DSL, such that the type system of your DSL can be embedded into the parent language. IMHO absence of static analysis is a huge downside of using an interpreted language. Having type safety in an embedded DSL can only be dreampt of in a language like ruby. http://tom.lokhorst.eu/2009/09/deeply-embedded-dsls http://tom.lokhorst.eu/2009/09/deeply-embedded-dsls
  - sklogic 12y ago
    What does typing have to do with interpreted vs. compiled? Hugs is an interpreter. And it's not a problem to build typed eDSLs on top of an untyped core language - see Typed Racket or Shen for example.
- malisper 12y ago
  Lisp provides a couple really powerful DSLs. Some of more easily understandable ones I can think of: Loop: http://www.gigamonkeys.com/book/loop-for-black-belts.html http://www.gigamonkeys.com/book/loop-for-black-belts.html Symbolic Manipulation: http://norvig.com/paip/macsymar.lisp http://norvig.com/paip/macsymar.lisp Generating HTML: http://allegroserve.sourceforge.net/aserve-dist/doc/htmlgen.html http://allegroserve.sourceforge.net/aserve-dist/doc/htmlgen....
thewarrior 12y ago
Any ruby experts know why the AttributeScope class is required ? Cant the functionality be included in the containing class ?
- likeclockwork 12y ago
  AttributeScope provides the common language for defining attributes. It's shared functionality, validation rules for the attributes. If it were included in the containing class it would only be available to that class. Company or Employee specific attribute-creation methods could be placed on CompanyScope or EmployeeScope.
emson 12y ago
If you would like to see an example for a command line invoice tool I wrote, have a look at this: https://github.com/emson/invoicerb https://github.com/emson/invoicerb I wrote it because I used an invoice app, that went bust and I needed something really simple to send out invoices and I was in complete control of. Also I didn't want to use a database so just used Ruby DSL meta files that described my invoice. Admittedly it probably needs a bit more work... but it was fun to do.
VeejayRampay 12y ago
Ruby is really great for metaprogramming. To the point where you have to refrain yourself from sprinkling too much of it all over the place at times.
rubiquity 12y ago
That seems like unnecessary use of metaprogamming and DSLs in my opinion[0]. That problem could be solved with good ol' object-orientation. Also, capturing blocks and instance_eval are notoriously slow. 0 - https://www.youtube.com/watch?v=yuh9COzp5vo https://www.youtube.com/watch?v=yuh9COzp5vo 1 - https://speakerdeck.com/sferik/writing-fast-ruby https://speakerdeck.com/sferik/writing-fast-ruby
- grandpa 12y ago
  > Also, capturing blocks and instance_eval are notoriously slow. ...and, in this case, are evaluated once, at startup. It's so fast that you couldn't possibly notice a difference.
deleted 12y ago
[deleted]
tel 12y ago
While I agree with all of the benefits of DSL writing, I kind of feel like DSLs in Ruby are a bit silly now. There is always a thin line between "lots of function abstraction" and DSL (e.g. a shallow embedding is pretty tough to distinguish from... well, just a library) and that line is even thinner in the fast-and-loose world of Ruby DSLs. It's a vague enough question to lead to nothing but religious wars, but I really want to push people who are interested in (e)DSLs to consider their implementation in typed languages. Haskell is, of course, a favorite of mine, but you can achieve similar things in Scala and the ML family. You can even go much further using something like Idris, mostly by using its stronger dependently typed system.
- sklogic 12y ago
  Haskell is not very suitable for the efficient and straightforward eDSL implementation. Template Haskell is a nice thing, but it is staged (cannot use the template definition from the same module) and enforces explicit template syntax. And the usual Haskell DSLs are in fact all just ad hoc interpreters, missing all the benefits of having an underlying compiled runtime environment. And typing is not a big deal, once you have a proper metaprogramming, you can easily add typing on top of your underlying language.
  - tel 12y ago
    You can take advantage of the Haskell compiler with a shallow embedding, you can do your own compilation steps with a deep embedding, you can do partial evaluation to speed up compilation. You can introduce your own type system, of course, but it's really nice to have most of that infrastructure pre-built for you (otherwise why embed?).
    - sklogic 12y ago
      The only reasonable ways of implementing fully compiled DSLs in Haskell are standalone DSL compilers (e.g., as in Happy) or Template Haskell (restricted syntax, stages, limited reflection). Both ways are not as convenient as with the more metaprogramming-oriented languages, plus the lack of reflection limits the potential for integrating DSL into the host language and the other DSLs. Yet, TH is really nice. I cannot stand languages without any metaprogramming capabilities at all, and TH just ticks the box for me.
qewrffewqwfqew 12y ago
I don't get it. How is this a DSL? Programmer with a couple of decades' experience here, but no ruby. I have no idea what half the tokens in the example mean, or what changes would be valid beyond editing the four obvious values.
- isbadawi 12y ago
  "DSL" is slang for "API" in the ruby community [0]. [0]: https://gist.github.com/geeksam/24ef10be8c773a2c1bd4 https://gist.github.com/geeksam/24ef10be8c773a2c1bd4

80 points by edawerd 12y ago

One of the best articlets I've read about the ruby dsl's was on practicingruby. You guys can check it out here https://practicingruby.com/articles/domain-specific-apis https://practicingruby.com/articles/domain-specific-apis is free now.

vinceguidry 12y ago

For any struggling to understand Ruby's metaprogramming, the rather short and succinct Metaprogramming Ruby is an excellent read. There's a method to the madness of define_method, method_missing, the various eval methods, the difference between Blocks, Procs, and Lambdas. Most of it revolves around manipulating scopes. It also shows how you can iterate from the ugliest, most powerful, most dangerous eval method by pulling out the functionality you need into less-powerful, safer manipulations. It's all Ruby code until it's actually executed, so it can all be manipulated at runtime. http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Pros/dp/1934356476 http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Pros...

tcopeland 12y ago
Yup this is great stuff, just be sure to get the 2nd edition which just came out: http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Facets/dp/1941222129/ref=dp_ob_title_bk http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Face... It focuses on modern Rubies, so it talks about Module#prepend and such. Nice to keep moving forwards.
thirdtruck 12y ago
Seconding. The book revolutionized my understanding of Ruby, and inspired me to create a whole visual novel framework: https://github.com/thirdtruck/rubyai https://github.com/thirdtruck/rubyai.

tcopeland 12y ago

That's a great writeup, well done. Along the same lines, here's the ever-entertaining Rich Kilmer talking about DSL experiences on some government projects: http://www.infoq.com/presentations/kilmer-ruby-dsls http://www.infoq.com/presentations/kilmer-ruby-dsls

habosa 12y ago

Wow. That was an amazing article. I have been doing Rails/Ruby for years and never understood any of that. Thanks for putting that together, please consider doing more tutorials for other interesting things going on in ZenPayroll.

bshimmin 12y ago

A good read for anyone just starting out with some of the more intricate parts of Ruby. I often think Ruby is unsurpassed for writing readable, English-like DSLs - perhaps because of the combination of the convenient lassitude of Ruby's syntax (not requiring brackets - in many cases - or semi-colons) and the dangerous power of its metaprogramming abilities. The only other language that perhaps comes close and immediately springs to mind, before I've had my first coffee of the day, is (appropriately) CoffeeScript. Any other takers?

lennel 12y ago
on the jvm i would say groovy is probably one of the better languages for dsl's. Gradle is a dsl and groovy has inbuilt Builder classes to help you in the creation of dsl's Clojure being a lisp is extremely powerful in this regard as well, but if you are programming in any lisp I would expect a fair amount of meta programming.
0xfaded 12y ago
Languages with expressive type systems, Haskell likely the most mainstream, allow for deep embedding of an DSL, such that the type system of your DSL can be embedded into the parent language. IMHO absence of static analysis is a huge downside of using an interpreted language. Having type safety in an embedded DSL can only be dreampt of in a language like ruby. http://tom.lokhorst.eu/2009/09/deeply-embedded-dsls http://tom.lokhorst.eu/2009/09/deeply-embedded-dsls
- sklogic 12y ago
  What does typing have to do with interpreted vs. compiled? Hugs is an interpreter. And it's not a problem to build typed eDSLs on top of an untyped core language - see Typed Racket or Shen for example.
malisper 12y ago
Lisp provides a couple really powerful DSLs. Some of more easily understandable ones I can think of: Loop: http://www.gigamonkeys.com/book/loop-for-black-belts.html http://www.gigamonkeys.com/book/loop-for-black-belts.html Symbolic Manipulation: http://norvig.com/paip/macsymar.lisp http://norvig.com/paip/macsymar.lisp Generating HTML: http://allegroserve.sourceforge.net/aserve-dist/doc/htmlgen.html http://allegroserve.sourceforge.net/aserve-dist/doc/htmlgen....

thewarrior 12y ago

Any ruby experts know why the AttributeScope class is required ? Cant the functionality be included in the containing class ?

likeclockwork 12y ago
AttributeScope provides the common language for defining attributes. It's shared functionality, validation rules for the attributes. If it were included in the containing class it would only be available to that class. Company or Employee specific attribute-creation methods could be placed on CompanyScope or EmployeeScope.

emson 12y ago

If you would like to see an example for a command line invoice tool I wrote, have a look at this: https://github.com/emson/invoicerb https://github.com/emson/invoicerb I wrote it because I used an invoice app, that went bust and I needed something really simple to send out invoices and I was in complete control of. Also I didn't want to use a database so just used Ruby DSL meta files that described my invoice. Admittedly it probably needs a bit more work... but it was fun to do.

VeejayRampay 12y ago

Ruby is really great for metaprogramming. To the point where you have to refrain yourself from sprinkling too much of it all over the place at times.

rubiquity 12y ago

That seems like unnecessary use of metaprogamming and DSLs in my opinion[0]. That problem could be solved with good ol' object-orientation. Also, capturing blocks and instance_eval are notoriously slow. 0 - https://www.youtube.com/watch?v=yuh9COzp5vo https://www.youtube.com/watch?v=yuh9COzp5vo 1 - https://speakerdeck.com/sferik/writing-fast-ruby https://speakerdeck.com/sferik/writing-fast-ruby

grandpa 12y ago
> Also, capturing blocks and instance_eval are notoriously slow. ...and, in this case, are evaluated once, at startup. It's so fast that you couldn't possibly notice a difference.

deleted 12y ago

[deleted]

tel 12y ago

While I agree with all of the benefits of DSL writing, I kind of feel like DSLs in Ruby are a bit silly now. There is always a thin line between "lots of function abstraction" and DSL (e.g. a shallow embedding is pretty tough to distinguish from... well, just a library) and that line is even thinner in the fast-and-loose world of Ruby DSLs. It's a vague enough question to lead to nothing but religious wars, but I really want to push people who are interested in (e)DSLs to consider their implementation in typed languages. Haskell is, of course, a favorite of mine, but you can achieve similar things in Scala and the ML family. You can even go much further using something like Idris, mostly by using its stronger dependently typed system.

sklogic 12y ago
Haskell is not very suitable for the efficient and straightforward eDSL implementation. Template Haskell is a nice thing, but it is staged (cannot use the template definition from the same module) and enforces explicit template syntax. And the usual Haskell DSLs are in fact all just ad hoc interpreters, missing all the benefits of having an underlying compiled runtime environment. And typing is not a big deal, once you have a proper metaprogramming, you can easily add typing on top of your underlying language.
- tel 12y ago
  You can take advantage of the Haskell compiler with a shallow embedding, you can do your own compilation steps with a deep embedding, you can do partial evaluation to speed up compilation. You can introduce your own type system, of course, but it's really nice to have most of that infrastructure pre-built for you (otherwise why embed?).
  - sklogic 12y ago
    The only reasonable ways of implementing fully compiled DSLs in Haskell are standalone DSL compilers (e.g., as in Happy) or Template Haskell (restricted syntax, stages, limited reflection). Both ways are not as convenient as with the more metaprogramming-oriented languages, plus the lack of reflection limits the potential for integrating DSL into the host language and the other DSLs. Yet, TH is really nice. I cannot stand languages without any metaprogramming capabilities at all, and TH just ticks the box for me.

qewrffewqwfqew 12y ago

I don't get it. How is this a DSL? Programmer with a couple of decades' experience here, but no ruby. I have no idea what half the tokens in the example mean, or what changes would be valid beyond editing the four obvious values.

isbadawi 12y ago
"DSL" is slang for "API" in the ruby community [0]. [0]: https://gist.github.com/geeksam/24ef10be8c773a2c1bd4 https://gist.github.com/geeksam/24ef10be8c773a2c1bd4