3 ms·

A DSL can be immensely useful in terms of useability. But it is not trivial to be done right. You need to define a consistent grammar, make sure it's intuitive

A DSL can be immensely useful in terms of useability. But it is not trivial to be done right. You need to define a consistent grammar, make sure it's intuitive to your users and free of all sorts of fun bugs that happen when odd constructs are put together.

argd678 7y ago
Also if you don’t write a debugger and report line numbers on errors, you have not fully understood your problem domain.
throwaway1492 7y ago
I think you're describing what Fowler[1] calls an "external" dsl ie separate lexer, parser, and code generator. The alternative being "internal" dsl, something that uses a languages own constructs to create stuff; ie like how Ruby is is used in several well known frameworks. [1] https://www.goodreads.com/book/show/8082269-domain-specific-languages https://www.goodreads.com/book/show/8082269-domain-specific-...
- grumdan 7y ago
  I think this is still applicable advice for embedded DSLs too, since the grammar of the DSL will be made up of the ways in which you can compose the operations you provide, whether they are parsed by the host language's parser or your own. Overall, I agree that it's important to think about how to make sure that operations compose in an unintuitive, expected way. I find this hard to figure out without thinking of it in terms of a grammar and semantics, whether it's embedded or not.
dkersten 7y ago
I inadvertently created a scripting language recently. It started out as simple commands like "thing param" (kinda like TCL, I guess), but it slowly evolved (needed conditional commands, state, etc) and after a while I wanted proper expression parsing because my ad-hoc command parser was way too brittle. So I wrote an EBNF grammar and used Clojures Instaparse to parse it into an AST and then I added functions, loops and other more traditional programming constructs because why not. I've cleaned up the grammar a few times and now its a simple but relatively full featured scripting language (with some domain-specific features). There are basically three parts: the text parser (EBNF grammar and instaparse), AST transformation that takes the AST that the grammar created and cleans it up (removing or simplifying intermediary nodes, turning string numbers into numbers, extracting function definitions, etc) and the evaluator (which, I'm quite proud of: its a pure function of the scripts inputs + mutable state and it generates a collection of "actions" and new mutable state). Now onto why I mention this: > free of all sorts of fun bugs that happen when odd constructs are put together I actually had an interesting experience with this, but in a good way. As I said above, I hoist the code for functions out of the AST (the final AST is basically a map of function to function body AST, where one of these is the main script code) and the declarations are replaced by the transformation step with a local variable binding that gets set to some data describing how to call the function (key into the map, list of parameters, other stuff), so when you do 'f()', it looks up 'f' in local state, gets this data and figures out how to call the function from that. What this meant was that I could pass functions around. And so I got higher order functions almost for free, without having planned for it, just because I thought "hmm, what if I do this?" (the current version did add some special logic to capture state, for closures, but I did that after when I decided why not support it for real). As an aside, my little language is a synchronous language[1], which acts as if its runtime is instantaneous (ie, the world does not change until it completes), which makes a lot of things easier (both for me and for users) and makes sense in the domain, since its basically commands that get triggered when certain conditions are met. This almost makes it worth writing a custom language over using an existing one, but if I were to start again, I'd probably just use Lua or Javascript and save some time, even if the final language isn't so easy for my target audience. [1] https://en.wikipedia.org/wiki/Synchronous_programming_language https://en.wikipedia.org/wiki/Synchronous_programming_langua...

argd678 7y ago

Also if you don’t write a debugger and report line numbers on errors, you have not fully understood your problem domain.

throwaway1492 7y ago

I think you're describing what Fowler[1] calls an "external" dsl ie separate lexer, parser, and code generator. The alternative being "internal" dsl, something that uses a languages own constructs to create stuff; ie like how Ruby is is used in several well known frameworks. [1] https://www.goodreads.com/book/show/8082269-domain-specific-languages https://www.goodreads.com/book/show/8082269-domain-specific-...

grumdan 7y ago
I think this is still applicable advice for embedded DSLs too, since the grammar of the DSL will be made up of the ways in which you can compose the operations you provide, whether they are parsed by the host language's parser or your own. Overall, I agree that it's important to think about how to make sure that operations compose in an unintuitive, expected way. I find this hard to figure out without thinking of it in terms of a grammar and semantics, whether it's embedded or not.

dkersten 7y ago

I inadvertently created a scripting language recently. It started out as simple commands like "thing param" (kinda like TCL, I guess), but it slowly evolved (needed conditional commands, state, etc) and after a while I wanted proper expression parsing because my ad-hoc command parser was way too brittle. So I wrote an EBNF grammar and used Clojures Instaparse to parse it into an AST and then I added functions, loops and other more traditional programming constructs because why not. I've cleaned up the grammar a few times and now its a simple but relatively full featured scripting language (with some domain-specific features). There are basically three parts: the text parser (EBNF grammar and instaparse), AST transformation that takes the AST that the grammar created and cleans it up (removing or simplifying intermediary nodes, turning string numbers into numbers, extracting function definitions, etc) and the evaluator (which, I'm quite proud of: its a pure function of the scripts inputs + mutable state and it generates a collection of "actions" and new mutable state). Now onto why I mention this: > free of all sorts of fun bugs that happen when odd constructs are put together I actually had an interesting experience with this, but in a good way. As I said above, I hoist the code for functions out of the AST (the final AST is basically a map of function to function body AST, where one of these is the main script code) and the declarations are replaced by the transformation step with a local variable binding that gets set to some data describing how to call the function (key into the map, list of parameters, other stuff), so when you do 'f()', it looks up 'f' in local state, gets this data and figures out how to call the function from that. What this meant was that I could pass functions around. And so I got higher order functions almost for free, without having planned for it, just because I thought "hmm, what if I do this?" (the current version did add some special logic to capture state, for closures, but I did that after when I decided why not support it for real). As an aside, my little language is a synchronous language[1], which acts as if its runtime is instantaneous (ie, the world does not change until it completes), which makes a lot of things easier (both for me and for users) and makes sense in the domain, since its basically commands that get triggered when certain conditions are met. This almost makes it worth writing a custom language over using an existing one, but if I were to start again, I'd probably just use Lua or Javascript and save some time, even if the final language isn't so easy for my target audience. [1] https://en.wikipedia.org/wiki/Synchronous_programming_language https://en.wikipedia.org/wiki/Synchronous_programming_langua...