7 ms·

Design Guidelines for Domain Specific Languages (2014)

133 points by lr0 3y ago

valenterry 3y ago
Unfortunately, they don't mention the most easy way of defining a DSL: by doing it in the host language itself. Not all programming languages are capable of allowing that, but some do, and in that case it is by far the most easy and safe solution. It is still much much easier than combining existing language like adding OCL into another language, as suggest in the paper; and you get compiler/IDE support, tooling support and everything else from the "host language" out of the box.
- MrJohz 3y ago
  I'd go a step further, and suggest that most languages are able to embed some level of DSL, although you might not be able to reach quite the expressiveness that you had in mind. Generally a combination of block closures and being able to chain methods together is usually enough to create something reasonably expressive, although even then there are alternatives - I've seen DSLs in Python that use the class syntax and metaclasses to define quite complex this. You won't be generating your own syntax or anything, but you can often get pretty far.
- victorNicollet 3y ago
  This is definitely the best idea for "drop down" DSLs (by drop down, I mean that you are working in a general-purpose language and then, for one specific situation, you drop down into the DSL because it is more productive). I think the article is more oriented towards standalone DSLs (where the entry point is the DSL itself), as those get actual maintenance benefits from not being able to invoke arbitrary code in the host language.
  - valenterry 3y ago
    Yes. My point is just that I would judge that in 90% of the cases, "drop down" DSLs are the best solution for the problem at hand. So when talking about DSLs they should be mentioned, or at least the term DSL should be defined accordingly, which did not happen in the paper.
- chriswarbo 3y ago
  Alan Kay describes OOP as defining interactions with/between values via mini languages/algebras. In a language like Smalltalk (using whitespace and mixfix syntax) this very much feels like crafting a lightweight/hosted DSL. Other languages require a lot more squinting, to get past the host language's punctuation and ceremony. For example, compare something like the following: canvas drawA: circle at: topLeft in: red. canvas.draw(circle, at=topLeft, in=red);
  - ljm 3y ago
    I think the discoverability of Smalltalk code within the IDE lends nicely to that too. It'd be quite obscure to many these days, but there's something to be said about the ability to browse and experiment with a catalogue of classes like a graphical REPL on steroids.
- Too 3y ago
  Doesn’t necessarily have to be the host language. Just any other existing language. If it’s static and declarative, just use yaml. (Yes it sucks, but you get it point) If it’s imperative, just use the host language. Then you have some hybrid solutions. HCL allowing rich expression in an otherwise very declarative language. A big list of constant dataclasses in python is another quick way to define complex data structures without having to reinvent anything. This even gives you type safety for free. Heck, MongoDB managed to build big business on a database whose only query language is super verbose json. This isn’t just negative, because by leveraging existing language, a lot of tooling around is ready to plug in on day one.
  - valenterry 3y ago
    Actually, I disagree. The _by far_ easiest way is to use the host language itself. Using _any_ other language, e.g. yaml, forces you to do a lot of things that you get for free when using the host language, including syntax, type-checking/linting, build-system if you need one and especially all the tooling that comes with it for free (e.g. autocomplete in IDEs). In fact, there is almost no benefit at all of using yaml _for DSLs_. (And yes, there is widely-available syntaxhighlighting for yaml and others, but that doesn't over anything specific for the DSL, or otherwise you will have to customize it)
    - Too 3y ago
      Host language is rarely ever declarative. Declarative languages are often necessary because they don’t leave any escape hatches that tempt you into writing incorrect code. Build system dependency graphs one common example. It’s also needed to pass data between different systems without recompiling and to constrain the usage boundaries, especially around security. Imagine how absurd it would be if mongodb queries were written in the host language C++ instead of json. For in-house stuff agree. There is no point to make excessive config layers if only developers will touch it anyway.
      - valenterry 3y ago
        > For in-house stuff agree. There is no point to make excessive config layers if only developers will touch it anyway. Yeah, I'm specifically talking about in-house DSLs or those that are only used by/from specific users. That doesn't have to be developers, but they should be using the host language as well - and in that case, an imperative host language can still make it able to use a declarative DSL. (in the most basic way for instance: generate a datastructure that essentially describes a json/yaml or whatever) For non-inhouse DSL, I agree that it is usually worthwhile to make them standalone.
- librasteve 3y ago
  Raku has a built in Grammar capability that integrates well with regex / tokens / rules in an composable OO context. As mentioned you get all the IDE support (IntelliJ based) including a grammar inspector tool.
victorNicollet 3y ago
A few strong opinions, a lot stronger than they were in my 2017 talk https://youtu.be/_gmMJwjg0Pk https://youtu.be/_gmMJwjg0Pk, surely that must be the experience speaking :) - Creating a DSL is a commitment. It is never done, it's either in development or abandoned. Plan for this, both in terms of team size and knowledge management. - Don't allow escape hatches or ways to access arbitrary functionality in the host language. When (not if) these are used, they ruin backwards compatibility and make maintenance of DSL code a nightmare. - Don't use a PEG (parsing expression grammar), they are terrible for backwards compatibility. Hand-written parsers are even worse. Using an LR parser generator is worth the time investment. - Support static analysis, or at the very least don't make it impossible to implement later. It improves productivity, and every error message is an opportunity to teach the DSL to the users. - You need a language server. The bare minimum is: highlighting, auto-completion, hints on hover, find references, rename. - It should be trivial to reproduce a production run on a development machine. This means the language should be deterministic, and the production context should be preserved for later reuse, and the development machine should be safe for accessing production data. - Thousands of unit tests, thousands of regression tests. It's fine to break these rules, if you can live with the consequences. And here's a brand new one from this year: use RAG to have ChatGPT "read" your documentation, and ask it to produce scripts for you. When the script doesn't compile, feed the error message back to ChatGPT and try again until you get a working script. Think deeply about why ChatGPT made some mistakes, as it's often the case that humans would have the same misunderstandings. Improve your documentation, error messages or syntax to eliminate them, and then try again to see if it works. We have been doing this for 6 months now, and there have already been some significant improvements that are only obvious in hindsight. And we've now opened a PhD position to investigate whether we can have a separate "LLM-friendly" syntax for the language, dedicated to making code generation easier.
- cc_ashby 3y ago
  Hey Victor, these are amazing tips. I am currently building a DSL for LLMs, and would love your feedback/consulting. I could not find your email online. Please feel free to reply to this or reach me here: [redacted] on X [email-redacted] Would love advisement from someone who has thought about DSLs extensively!
mejutoco 3y ago
I first saw this talk of DSLs with Ruby, where it is/was popular to use missing method and others to build your own "language". I do not mean it in a negative way, but at the time it seemed like calling a utility class DSL was excessive. It just seemed like regular programming, using the language constructs. It also was not enforcing many boundaries (there are escape hatches). Looking at all of this, would it not be easier to encode the DSL in something like json, and turn it into a data/format problem? Something like terraform. The implementation can then be in any language that can read json. One could imagine, for example, regex defined in json as a DSL.
- victorNicollet 3y ago
  I would say JSON + schema, rather than just JSON. Like S-expressions, JSON saves you from having to implement a tokenizer, and a tokenizer is one of the easier parts of a DSL. You still need to implement something like a syntax, because things that are handled by the syntax now get pushed into later stages of the DSL. However, if you can provide a JSON schema for your language, then the problem is mostly solved ! On the other hand, it's not enough to read JSON, you need a JSON parser library that provides you with the position (line:column) of every JSON value, so that you can emit user-friendly errors.
- chriswarbo 3y ago
  I think every language (DSL or not) should support translation to/from a simple data representation for its syntax trees (whether s-expressions, JSON, XML, etc.). Not necessarily the same representation, but just something structured that existing tools can walk and transform, such that (a) we don't resort to grepping through bytes (and manually filtering out matches that are comments or strings) and (b) adding a keyword to a language won't break all existing tooling that's unable to parse it. I wrote about this at http://www.chriswarbo.net/blog/2017-01-31-syntax_trees.html http://www.chriswarbo.net/blog/2017-01-31-syntax_trees.html
LelouBil 3y ago
I built a DSL for a Visual Novel game I am making, by using ANTLR for parsing and then executing it using C#. Are there generic frameworks for implementing static analysis that exists out there ? I did a rudimentary version of it with some simple checks, but I have troubles wrapping my head about checking conditional branches and stuff like this.
- no_wizard 3y ago
  check out Rascal[0] [0]: https://www.rascal-mpl.org/ https://www.rascal-mpl.org/
  - LelouBil 3y ago
    Looks interesting but I think I lack a lot of knowledge to be able to use it right now..
scottfr 3y ago
I've created a few DSL's and I've regretted not closely emulating an existing language that users already know. When creating a new DSL you should find the language that is most familiar to your users. That could be Python, Javascript, or SQL. There is a good chance though that it is spreadsheet formulas. Whatever that language, align capabilities you add to your DSL with the syntax and function naming of that language only diverging where needed to add the custom capabilities which motivated the DSL. The closer you align with what your users already know, the more easily they will be able to adopt it. An added benefit of this that emerged in the past year is GPT's will be able to write your DSL with less prompting.
swader999 3y ago
I want to build a DSL for a really specific reporting use case. It's daunting though. Not sure if what I have in mind is a true DSL, perhaps just a souped up builder pattern.
- eternityforest 3y ago
  Some kind of multi step data transform pipeline thing I assume?
  - swader999 3y ago
    Perhaps, it's a build your own report thingy for end users or perhaps internal (non technical) support to use. I've never seen these things work so I'm deferring this feature until the end as best I can.
    - eternityforest 3y ago
      I wonder if those users could learn SQL+Jinja2+a few spreadsheet-like helper functions to use in the templates? End users can use Excel, which is close to a real language, but declarative and reactive and heavily framework-ed. I'm not sure how a DSL would be much easier than Jinja2. I'm thinking of adding user Jinja templates to my SQLite browser utility. Zero code GUI only config is my favorite way to do things, but after that I'm not sure why you'd want to not just jump straight to a mainstream language that people can Google. In the 2000s Myspace users learned HTML just fine. Mainstream languages are already designed to be easy for anyone to use.

133 points by lr0 3y ago

Unfortunately, they don't mention the most easy way of defining a DSL: by doing it in the host language itself. Not all programming languages are capable of allowing that, but some do, and in that case it is by far the most easy and safe solution. It is still much much easier than combining existing language like adding OCL into another language, as suggest in the paper; and you get compiler/IDE support, tooling support and everything else from the "host language" out of the box.

MrJohz 3y ago
I'd go a step further, and suggest that most languages are able to embed some level of DSL, although you might not be able to reach quite the expressiveness that you had in mind. Generally a combination of block closures and being able to chain methods together is usually enough to create something reasonably expressive, although even then there are alternatives - I've seen DSLs in Python that use the class syntax and metaclasses to define quite complex this. You won't be generating your own syntax or anything, but you can often get pretty far.
victorNicollet 3y ago
This is definitely the best idea for "drop down" DSLs (by drop down, I mean that you are working in a general-purpose language and then, for one specific situation, you drop down into the DSL because it is more productive). I think the article is more oriented towards standalone DSLs (where the entry point is the DSL itself), as those get actual maintenance benefits from not being able to invoke arbitrary code in the host language.
- valenterry 3y ago
  Yes. My point is just that I would judge that in 90% of the cases, "drop down" DSLs are the best solution for the problem at hand. So when talking about DSLs they should be mentioned, or at least the term DSL should be defined accordingly, which did not happen in the paper.
chriswarbo 3y ago
Alan Kay describes OOP as defining interactions with/between values via mini languages/algebras. In a language like Smalltalk (using whitespace and mixfix syntax) this very much feels like crafting a lightweight/hosted DSL. Other languages require a lot more squinting, to get past the host language's punctuation and ceremony. For example, compare something like the following: canvas drawA: circle at: topLeft in: red. canvas.draw(circle, at=topLeft, in=red);
- ljm 3y ago
  I think the discoverability of Smalltalk code within the IDE lends nicely to that too. It'd be quite obscure to many these days, but there's something to be said about the ability to browse and experiment with a catalogue of classes like a graphical REPL on steroids.
Too 3y ago
Doesn’t necessarily have to be the host language. Just any other existing language. If it’s static and declarative, just use yaml. (Yes it sucks, but you get it point) If it’s imperative, just use the host language. Then you have some hybrid solutions. HCL allowing rich expression in an otherwise very declarative language. A big list of constant dataclasses in python is another quick way to define complex data structures without having to reinvent anything. This even gives you type safety for free. Heck, MongoDB managed to build big business on a database whose only query language is super verbose json. This isn’t just negative, because by leveraging existing language, a lot of tooling around is ready to plug in on day one.
- valenterry 3y ago
  Actually, I disagree. The _by far_ easiest way is to use the host language itself. Using _any_ other language, e.g. yaml, forces you to do a lot of things that you get for free when using the host language, including syntax, type-checking/linting, build-system if you need one and especially all the tooling that comes with it for free (e.g. autocomplete in IDEs). In fact, there is almost no benefit at all of using yaml _for DSLs_. (And yes, there is widely-available syntaxhighlighting for yaml and others, but that doesn't over anything specific for the DSL, or otherwise you will have to customize it)
  - Too 3y ago
    Host language is rarely ever declarative. Declarative languages are often necessary because they don’t leave any escape hatches that tempt you into writing incorrect code. Build system dependency graphs one common example. It’s also needed to pass data between different systems without recompiling and to constrain the usage boundaries, especially around security. Imagine how absurd it would be if mongodb queries were written in the host language C++ instead of json. For in-house stuff agree. There is no point to make excessive config layers if only developers will touch it anyway.
    - valenterry 3y ago
      > For in-house stuff agree. There is no point to make excessive config layers if only developers will touch it anyway. Yeah, I'm specifically talking about in-house DSLs or those that are only used by/from specific users. That doesn't have to be developers, but they should be using the host language as well - and in that case, an imperative host language can still make it able to use a declarative DSL. (in the most basic way for instance: generate a datastructure that essentially describes a json/yaml or whatever) For non-inhouse DSL, I agree that it is usually worthwhile to make them standalone.
librasteve 3y ago
Raku has a built in Grammar capability that integrates well with regex / tokens / rules in an composable OO context. As mentioned you get all the IDE support (IntelliJ based) including a grammar inspector tool.

victorNicollet 3y ago

A few strong opinions, a lot stronger than they were in my 2017 talk https://youtu.be/_gmMJwjg0Pk https://youtu.be/_gmMJwjg0Pk, surely that must be the experience speaking :) - Creating a DSL is a commitment. It is never done, it's either in development or abandoned. Plan for this, both in terms of team size and knowledge management. - Don't allow escape hatches or ways to access arbitrary functionality in the host language. When (not if) these are used, they ruin backwards compatibility and make maintenance of DSL code a nightmare. - Don't use a PEG (parsing expression grammar), they are terrible for backwards compatibility. Hand-written parsers are even worse. Using an LR parser generator is worth the time investment. - Support static analysis, or at the very least don't make it impossible to implement later. It improves productivity, and every error message is an opportunity to teach the DSL to the users. - You need a language server. The bare minimum is: highlighting, auto-completion, hints on hover, find references, rename. - It should be trivial to reproduce a production run on a development machine. This means the language should be deterministic, and the production context should be preserved for later reuse, and the development machine should be safe for accessing production data. - Thousands of unit tests, thousands of regression tests. It's fine to break these rules, if you can live with the consequences. And here's a brand new one from this year: use RAG to have ChatGPT "read" your documentation, and ask it to produce scripts for you. When the script doesn't compile, feed the error message back to ChatGPT and try again until you get a working script. Think deeply about why ChatGPT made some mistakes, as it's often the case that humans would have the same misunderstandings. Improve your documentation, error messages or syntax to eliminate them, and then try again to see if it works. We have been doing this for 6 months now, and there have already been some significant improvements that are only obvious in hindsight. And we've now opened a PhD position to investigate whether we can have a separate "LLM-friendly" syntax for the language, dedicated to making code generation easier.

cc_ashby 3y ago
Hey Victor, these are amazing tips. I am currently building a DSL for LLMs, and would love your feedback/consulting. I could not find your email online. Please feel free to reply to this or reach me here: [redacted] on X [email-redacted] Would love advisement from someone who has thought about DSLs extensively!

mejutoco 3y ago

I first saw this talk of DSLs with Ruby, where it is/was popular to use missing method and others to build your own "language". I do not mean it in a negative way, but at the time it seemed like calling a utility class DSL was excessive. It just seemed like regular programming, using the language constructs. It also was not enforcing many boundaries (there are escape hatches). Looking at all of this, would it not be easier to encode the DSL in something like json, and turn it into a data/format problem? Something like terraform. The implementation can then be in any language that can read json. One could imagine, for example, regex defined in json as a DSL.

victorNicollet 3y ago
I would say JSON + schema, rather than just JSON. Like S-expressions, JSON saves you from having to implement a tokenizer, and a tokenizer is one of the easier parts of a DSL. You still need to implement something like a syntax, because things that are handled by the syntax now get pushed into later stages of the DSL. However, if you can provide a JSON schema for your language, then the problem is mostly solved ! On the other hand, it's not enough to read JSON, you need a JSON parser library that provides you with the position (line:column) of every JSON value, so that you can emit user-friendly errors.
chriswarbo 3y ago
I think every language (DSL or not) should support translation to/from a simple data representation for its syntax trees (whether s-expressions, JSON, XML, etc.). Not necessarily the same representation, but just something structured that existing tools can walk and transform, such that (a) we don't resort to grepping through bytes (and manually filtering out matches that are comments or strings) and (b) adding a keyword to a language won't break all existing tooling that's unable to parse it. I wrote about this at http://www.chriswarbo.net/blog/2017-01-31-syntax_trees.html http://www.chriswarbo.net/blog/2017-01-31-syntax_trees.html

LelouBil 3y ago

I built a DSL for a Visual Novel game I am making, by using ANTLR for parsing and then executing it using C#. Are there generic frameworks for implementing static analysis that exists out there ? I did a rudimentary version of it with some simple checks, but I have troubles wrapping my head about checking conditional branches and stuff like this.

no_wizard 3y ago
check out Rascal[0] [0]: https://www.rascal-mpl.org/ https://www.rascal-mpl.org/
- LelouBil 3y ago
  Looks interesting but I think I lack a lot of knowledge to be able to use it right now..

scottfr 3y ago

I've created a few DSL's and I've regretted not closely emulating an existing language that users already know. When creating a new DSL you should find the language that is most familiar to your users. That could be Python, Javascript, or SQL. There is a good chance though that it is spreadsheet formulas. Whatever that language, align capabilities you add to your DSL with the syntax and function naming of that language only diverging where needed to add the custom capabilities which motivated the DSL. The closer you align with what your users already know, the more easily they will be able to adopt it. An added benefit of this that emerged in the past year is GPT's will be able to write your DSL with less prompting.

swader999 3y ago

I want to build a DSL for a really specific reporting use case. It's daunting though. Not sure if what I have in mind is a true DSL, perhaps just a souped up builder pattern.

eternityforest 3y ago
Some kind of multi step data transform pipeline thing I assume?
- swader999 3y ago
  Perhaps, it's a build your own report thingy for end users or perhaps internal (non technical) support to use. I've never seen these things work so I'm deferring this feature until the end as best I can.
  - eternityforest 3y ago
    I wonder if those users could learn SQL+Jinja2+a few spreadsheet-like helper functions to use in the templates? End users can use Excel, which is close to a real language, but declarative and reactive and heavily framework-ed. I'm not sure how a DSL would be much easier than Jinja2. I'm thinking of adding user Jinja templates to my SQLite browser utility. Zero code GUI only config is my favorite way to do things, but after that I'm not sure why you'd want to not just jump straight to a mainstream language that people can Google. In the 2000s Myspace users learned HTML just fine. Mainstream languages are already designed to be easy for anyone to use.