5 ms·

Languages that encourage making DSLs are a two-edged sword. On the one hand, you get to make a language that is more clear and fine-tuned to your use-case. On t

Languages that encourage making DSLs are a two-edged sword. On the one hand, you get to make a language that is more clear and fine-tuned to your use-case. On the other, you have an ad-hoc language with no support that you have to maintain along with the documentation (considering that you can't expect anyone else to know the DSL ahead of time). As I've gotten older, I've determined that well-designed APIs in a well-known language are a better alternative to DSLs.

vidarh 1y ago
An API is just as much a DSL.
- bunderbunder 1y ago
  Kind of, except that a non-DSL API doesn't create any new syntax. Which means that you get to keep all sorts of quality-of-life tools like syntax highlighting and correctness checking in the editor, autoformatting, possibly some amount of linting, etc. A few years ago I revisited Racket after a long hiatus, and that was maybe the biggest thing I noticed. I really don't like syntax macros as much as I did back in the day. Once I decide to use `define-syntax` I've then got to decide whether I also want to wade into dealing with also implementing a syntax colorer or an indenter as part of my #lang. And if I do decide to do that, then I've got a bunch more work, and am also probably committing to working in DrRacket (otherwise I'd rather stay in emacs) because that's the only editor that supports those features, and it just turns into a whole quagmire. And it's arguably even worse outside of Racket, where I might have to implement a whole language server and editor plugin to accomplish the same. Versus, if I can do what I need to do with a reasonably tidy API, then I can get those quality of life things without all the extra maintenance burden. None of this was a big deal 20 years ago. My expectations were different back then, because I hadn't been spoiled by things like the language server protocol and everyone (finally) agreeing that autoformatting is a Good Thing.
  - AnimalMuppet 1y ago
    Even without the new spiffs, I still don't see the point of DSLs. From where I sit, I see exactly zero problems where I think that new syntax is what I need to be able to write a solution.
    - ljlolel 1y ago
      Regex
      - AnimalMuppet 1y ago
        Fair point, though I learned regexes a long time ago, so that they weren't on my "problems that I can solve with new syntax" list, but instead on the "syntax that's already there" list.
      - miki123211 1y ago
        TBH, I feel like regexes would be much easier to understand in a more literate form with standard syntax. Something like: user_part = re.repeat(re.alnum | re.chars(".-_+")) domain_segment = re.repeat(re.alnum) domain = re.list(domain_segment,separator=".",minimum=2) email_address = user_part + "@" + domain Where, in a real program, `domain` would be defined in a "standard library of constructions" that you can just import and re-use in more complicated regexes. Something like this can be implemented in any language with operator overloading, no DSL required. Without operator overloading, the syntax would be a bit more awkward, but still nicer than the current regexp madness.
        ChadNauseam 1y ago
        I don't quite understand where regex gets its reputation from. I think that once you remember the meaning of the operators, it's not too bad. (And the concise syntax is actually very helpful.) I get that the meaning of the operators is not clear unless you're already familiar with regex, but neither is the meaning of !, ?, %, &, |, ^, ~, &&, ||, <<, >>, *, //, &, ++ (prefix), ++ (postfix), and so on. You learn these because you need them once, and then they're burned into your mind forever. Regex was similar for me.
- 90s_dev 1y ago
  No, an API uses existing rules, but a DSL uses its own ad hoc rules. GP is right. Don't make DSLs, make APIs, which are: * More composable * More reusable * More simple to reason about * More natively supported * More portable * More readable * More maintainable
  - tuveson 1y ago
    There are plenty of places where it makes more sense to use DSLs (or where they’re flat-out required). SQL and regex both spring to mind. Pretty much any HTML templating language is simpler to use than concatenation a bunch of strings together. JSX is usually easier read than directly calling React functions directly. A line of a shell script can be much nicer (and more portable) than 20 lines of a more general purpose language. Those are things that spring to mind that I think are unequivocally DSLs, but if you’re willing to consider markup languages as DSLs, the list could get a lot longer.
    - 90s_dev 1y ago
      Most people think of DSLs as a language you can create within a language. All the things you named are either entire languages themselves like HTML, SQL, and the shell family, or formal extensions of existing languages like JSX. A DSL would be something you could create inside JS tagged literals or Ruby.
      - ChadNauseam 1y ago
        I don't think of DSLs as strictly being embedded in another programming language. The term I would use for that is "eDSL", short for Embedded Domain Specific Language. See: https://wiki.haskell.org/Embedded_domain_specific_language https://wiki.haskell.org/Embedded_domain_specific_language
      - fuzztester 1y ago
        Martin Fowler on DSLs, including internal and external DSLs: https://martinfowler.com/books/dsl.html https://martinfowler.com/books/dsl.html https://martinfowler.com/dsl.html https://martinfowler.com/dsl.html Also see: https://en.m.wikipedia.org/wiki/Domain-specific_language https://en.m.wikipedia.org/wiki/Domain-specific_language including the References section.
- efitz 1y ago
  No, an API is a contract. Just like an object or a class or a structure is not a DSL. None of these modify the syntax or keywords of the language used to implement them, and only objects and classes actually have the ability to override operators. Just grouping things together in a particular order and giving the group a name does not make a DSL.
  - vidarh 1y ago
    The API creates a language just the same modifying syntax, by introducing new words with new meaning. The point is that however you want to group it, the effect is the same: You introduce a whole new vocabulary that you need to learn the rules of.
- deleted 1y ago
  [deleted]
- jayd16 1y ago
  A DSL is expected to add or break language rules. An API is not. An API can be so complex as to be thought of as a DSL but that makes it bad.
SkyPuncher 1y ago
I've come to hate DSLs. With rare exception, almost every DSL has become a brittle subset of the parent language.
- paddy_m 1y ago
  Sometimes you want a specific limited subset of the parent language, especially when you don't want to expose a huge possibility space to non technical users. Exposing only a limited subset can also be helpful to sandbox for security reasons.
satoru42 1y ago
I like the idea of DSL, but in real projects, I hate almost every DSL introduced by coworkers.
- jolt42 1y ago
  Because they are really really difficult to get right almost as difficult as building a new language. But when they work well they can be a real boost.
6gvONxR4sf7o 1y ago
I’m right there with you. Modern languages have such a plethora of tooling available that DSLs and any API approaching a DSL have this massive gap unless they really embrace whatever patterns the host language’s tooling enables. Not long ago, i had to work with a coworker’s mini language and function runner engine, which was basically a mini programming language. Except without a debugger or type checker or stack traces or any of the other million niceties we’d have had if we just used the host language to execute things ‘natively.’ That said, while the level of tooling for big languages goes up, the bar for creating yesteryear’s tooling is going down, with all the LSP tooling we have now, for example. Maybe someday we’ll get languages with tools where libraries have nice tooling without crazy dev effort, and then we’ll change our tune on DSLs.
cess11 1y ago
An API is an interface, metaprogramming is a technique for building such interfaces by treating code as data. They aren't interchangeable concepts. Metaprogramming comes in handy when ordinary programming results in a lot of complex, hence unmaintainable, code or can't achieve something due to how the language is implemented. It can vastly simplify how you express application logic, e.g. when interfacing with foreign programming languages such as SQL or generating code from data. Any sane metaprogramming regime allows you to do something like macro expansion, i.e. look at the code the metaprogramming is compiled into in the current context.
agos 1y ago
this is a very important lesson. I remember the frustration of dealing with the Gradle and whatever was the name of the DSL used by Fastlane. Gradle especially kept changing so documentation was never really useful and there was zero help from the tooling
- michaelcampbell 1y ago
  I've run the same frustration road. And at least the last time I used it in anger (which was, granted, perhaps double digit years ago), the docs started out with and loudly proclaimed "here's how you write your own tasks!" Wait, what? I want to have a build system do at least the 80% case with minimal/zero work; why am I jumping right in to my bespoke needs? Not sure if this was just bad doc, or the tool actually didn't do the 80% very well then or what, but it's stuck with me.
- mdaniel 1y ago
  > and there was zero help from the tooling That's mostly my experience with ruby, too: you can type anything, don't worry about it! I wanted to mention that Gradle since I think about version 7 supports mostly static typing via Kotlin but regrettably it's lipstick on a pig since almost inevitably one needs to interact with the Groovy side of the house and then all bets are off
andai 1y ago
I heard that in Lisp every program becomes a DSL?
- 7thaccount 1y ago
  I wouldn't say that. It's just its trivial to make DSLs in languages like the various lisps (common lisp, racket, chez scheme...etc). That's usually what people complain about (besides the parenthesis) in that it's easy to not be very disciplined. Lisp is homoiconic, so there isn't really a distinction between programs and data. For example, a snippet of code like a for-loop iterating through a list is also a list that can be inspected and modified. Or something along those lines (there's an XKCD comic that captures the spirit where the person says "it's all CARs" as in lisp you can build everything from CAR, CDR, and CONS I think). You'll have to dig into that on your own. The terms are historically relevant, but seem antiquated now. I've never really understood macros (compile or runtime ones) all that well though, so hopefully someone else in the comics can clarify my mumbo jumbo. https://www.explainxkcd.com/wiki/index.php/224:_Lisp https://www.explainxkcd.com/wiki/index.php/224:_Lisp
drdrek 1y ago
Even DSLs of huge and successful projects are more often than not bad. As a rule if you think that you can create a better query language than SQL, please spare us. JQL, Elastic Query DSL, KQL... Don't.
drob518 1y ago
It has been interesting to watch the Clojure community’s take on DSLs over time. Clojure is a Lisp, and Lisps are notorious for inventing DSLs, particularly after PG’s various papers promoted that as a superpower of sorts. Clojure has gone the other direction and, while it supports macros, it generally discourages their use unless you’ve tried everything else first. But it still encourages DSLs built from the native Clojure data structures. So, for instance Hiccup for HTML generation.
- jolt42 1y ago
  The discouragement of macros I think is a knee-jerk reaction. Build around data structures and functions first, and then use the thinnest macros around that can be quite maintainable.
thomastjeffery 1y ago
I've been thinking about this a lot. DSLs solve an important problem, but they do it in the wrong direction. It's impossible to write objective fact. Everything we write is subjected to the context it is expressed in, including the grammar that we use to write it. A DSL accommodates this by letting you make a new grammar on the fly. The trouble is, this doesn't help us get out of the context we are writing in already: it only lets us enter a new one that is nested inside. So what if we could actually get out? What if we could write from outside of the context we are writing? That's the idea I'm working on. I think it's possible, but it's such an abstract idea that it's tricky to get a handle on.

vidarh 1y ago

An API is just as much a DSL.

bunderbunder 1y ago
Kind of, except that a non-DSL API doesn't create any new syntax. Which means that you get to keep all sorts of quality-of-life tools like syntax highlighting and correctness checking in the editor, autoformatting, possibly some amount of linting, etc. A few years ago I revisited Racket after a long hiatus, and that was maybe the biggest thing I noticed. I really don't like syntax macros as much as I did back in the day. Once I decide to use `define-syntax` I've then got to decide whether I also want to wade into dealing with also implementing a syntax colorer or an indenter as part of my #lang. And if I do decide to do that, then I've got a bunch more work, and am also probably committing to working in DrRacket (otherwise I'd rather stay in emacs) because that's the only editor that supports those features, and it just turns into a whole quagmire. And it's arguably even worse outside of Racket, where I might have to implement a whole language server and editor plugin to accomplish the same. Versus, if I can do what I need to do with a reasonably tidy API, then I can get those quality of life things without all the extra maintenance burden. None of this was a big deal 20 years ago. My expectations were different back then, because I hadn't been spoiled by things like the language server protocol and everyone (finally) agreeing that autoformatting is a Good Thing.
- AnimalMuppet 1y ago
  Even without the new spiffs, I still don't see the point of DSLs. From where I sit, I see exactly zero problems where I think that new syntax is what I need to be able to write a solution.
  - ljlolel 1y ago
    Regex
    - AnimalMuppet 1y ago
      Fair point, though I learned regexes a long time ago, so that they weren't on my "problems that I can solve with new syntax" list, but instead on the "syntax that's already there" list.
    - miki123211 1y ago
      TBH, I feel like regexes would be much easier to understand in a more literate form with standard syntax. Something like: user_part = re.repeat(re.alnum | re.chars(".-_+")) domain_segment = re.repeat(re.alnum) domain = re.list(domain_segment,separator=".",minimum=2) email_address = user_part + "@" + domain Where, in a real program, `domain` would be defined in a "standard library of constructions" that you can just import and re-use in more complicated regexes. Something like this can be implemented in any language with operator overloading, no DSL required. Without operator overloading, the syntax would be a bit more awkward, but still nicer than the current regexp madness.
      - ChadNauseam 1y ago
        I don't quite understand where regex gets its reputation from. I think that once you remember the meaning of the operators, it's not too bad. (And the concise syntax is actually very helpful.) I get that the meaning of the operators is not clear unless you're already familiar with regex, but neither is the meaning of !, ?, %, &, |, ^, ~, &&, ||, <<, >>, *, //, &, ++ (prefix), ++ (postfix), and so on. You learn these because you need them once, and then they're burned into your mind forever. Regex was similar for me.
90s_dev 1y ago
No, an API uses existing rules, but a DSL uses its own ad hoc rules. GP is right. Don't make DSLs, make APIs, which are: * More composable * More reusable * More simple to reason about * More natively supported * More portable * More readable * More maintainable
- tuveson 1y ago
  There are plenty of places where it makes more sense to use DSLs (or where they’re flat-out required). SQL and regex both spring to mind. Pretty much any HTML templating language is simpler to use than concatenation a bunch of strings together. JSX is usually easier read than directly calling React functions directly. A line of a shell script can be much nicer (and more portable) than 20 lines of a more general purpose language. Those are things that spring to mind that I think are unequivocally DSLs, but if you’re willing to consider markup languages as DSLs, the list could get a lot longer.
  - 90s_dev 1y ago
    Most people think of DSLs as a language you can create within a language. All the things you named are either entire languages themselves like HTML, SQL, and the shell family, or formal extensions of existing languages like JSX. A DSL would be something you could create inside JS tagged literals or Ruby.
    - ChadNauseam 1y ago
      I don't think of DSLs as strictly being embedded in another programming language. The term I would use for that is "eDSL", short for Embedded Domain Specific Language. See: https://wiki.haskell.org/Embedded_domain_specific_language https://wiki.haskell.org/Embedded_domain_specific_language
    - fuzztester 1y ago
      Martin Fowler on DSLs, including internal and external DSLs: https://martinfowler.com/books/dsl.html https://martinfowler.com/books/dsl.html https://martinfowler.com/dsl.html https://martinfowler.com/dsl.html Also see: https://en.m.wikipedia.org/wiki/Domain-specific_language https://en.m.wikipedia.org/wiki/Domain-specific_language including the References section.
efitz 1y ago
No, an API is a contract. Just like an object or a class or a structure is not a DSL. None of these modify the syntax or keywords of the language used to implement them, and only objects and classes actually have the ability to override operators. Just grouping things together in a particular order and giving the group a name does not make a DSL.
- vidarh 1y ago
  The API creates a language just the same modifying syntax, by introducing new words with new meaning. The point is that however you want to group it, the effect is the same: You introduce a whole new vocabulary that you need to learn the rules of.
deleted 1y ago
[deleted]
jayd16 1y ago
A DSL is expected to add or break language rules. An API is not. An API can be so complex as to be thought of as a DSL but that makes it bad.

SkyPuncher 1y ago

I've come to hate DSLs. With rare exception, almost every DSL has become a brittle subset of the parent language.

paddy_m 1y ago
Sometimes you want a specific limited subset of the parent language, especially when you don't want to expose a huge possibility space to non technical users. Exposing only a limited subset can also be helpful to sandbox for security reasons.

satoru42 1y ago

I like the idea of DSL, but in real projects, I hate almost every DSL introduced by coworkers.

jolt42 1y ago
Because they are really really difficult to get right almost as difficult as building a new language. But when they work well they can be a real boost.

6gvONxR4sf7o 1y ago

I’m right there with you. Modern languages have such a plethora of tooling available that DSLs and any API approaching a DSL have this massive gap unless they really embrace whatever patterns the host language’s tooling enables. Not long ago, i had to work with a coworker’s mini language and function runner engine, which was basically a mini programming language. Except without a debugger or type checker or stack traces or any of the other million niceties we’d have had if we just used the host language to execute things ‘natively.’ That said, while the level of tooling for big languages goes up, the bar for creating yesteryear’s tooling is going down, with all the LSP tooling we have now, for example. Maybe someday we’ll get languages with tools where libraries have nice tooling without crazy dev effort, and then we’ll change our tune on DSLs.

cess11 1y ago

An API is an interface, metaprogramming is a technique for building such interfaces by treating code as data. They aren't interchangeable concepts. Metaprogramming comes in handy when ordinary programming results in a lot of complex, hence unmaintainable, code or can't achieve something due to how the language is implemented. It can vastly simplify how you express application logic, e.g. when interfacing with foreign programming languages such as SQL or generating code from data. Any sane metaprogramming regime allows you to do something like macro expansion, i.e. look at the code the metaprogramming is compiled into in the current context.

agos 1y ago

this is a very important lesson. I remember the frustration of dealing with the Gradle and whatever was the name of the DSL used by Fastlane. Gradle especially kept changing so documentation was never really useful and there was zero help from the tooling

michaelcampbell 1y ago
I've run the same frustration road. And at least the last time I used it in anger (which was, granted, perhaps double digit years ago), the docs started out with and loudly proclaimed "here's how you write your own tasks!" Wait, what? I want to have a build system do at least the 80% case with minimal/zero work; why am I jumping right in to my bespoke needs? Not sure if this was just bad doc, or the tool actually didn't do the 80% very well then or what, but it's stuck with me.
mdaniel 1y ago
> and there was zero help from the tooling That's mostly my experience with ruby, too: you can type anything, don't worry about it! I wanted to mention that Gradle since I think about version 7 supports mostly static typing via Kotlin but regrettably it's lipstick on a pig since almost inevitably one needs to interact with the Groovy side of the house and then all bets are off

andai 1y ago

I heard that in Lisp every program becomes a DSL?

7thaccount 1y ago
I wouldn't say that. It's just its trivial to make DSLs in languages like the various lisps (common lisp, racket, chez scheme...etc). That's usually what people complain about (besides the parenthesis) in that it's easy to not be very disciplined. Lisp is homoiconic, so there isn't really a distinction between programs and data. For example, a snippet of code like a for-loop iterating through a list is also a list that can be inspected and modified. Or something along those lines (there's an XKCD comic that captures the spirit where the person says "it's all CARs" as in lisp you can build everything from CAR, CDR, and CONS I think). You'll have to dig into that on your own. The terms are historically relevant, but seem antiquated now. I've never really understood macros (compile or runtime ones) all that well though, so hopefully someone else in the comics can clarify my mumbo jumbo. https://www.explainxkcd.com/wiki/index.php/224:_Lisp https://www.explainxkcd.com/wiki/index.php/224:_Lisp

drdrek 1y ago

Even DSLs of huge and successful projects are more often than not bad. As a rule if you think that you can create a better query language than SQL, please spare us. JQL, Elastic Query DSL, KQL... Don't.

drob518 1y ago

It has been interesting to watch the Clojure community’s take on DSLs over time. Clojure is a Lisp, and Lisps are notorious for inventing DSLs, particularly after PG’s various papers promoted that as a superpower of sorts. Clojure has gone the other direction and, while it supports macros, it generally discourages their use unless you’ve tried everything else first. But it still encourages DSLs built from the native Clojure data structures. So, for instance Hiccup for HTML generation.

jolt42 1y ago
The discouragement of macros I think is a knee-jerk reaction. Build around data structures and functions first, and then use the thinnest macros around that can be quite maintainable.

thomastjeffery 1y ago

I've been thinking about this a lot. DSLs solve an important problem, but they do it in the wrong direction. It's impossible to write objective fact. Everything we write is subjected to the context it is expressed in, including the grammar that we use to write it. A DSL accommodates this by letting you make a new grammar on the fly. The trouble is, this doesn't help us get out of the context we are writing in already: it only lets us enter a new one that is nested inside. So what if we could actually get out? What if we could write from outside of the context we are writing? That's the idea I'm working on. I think it's possible, but it's such an abstract idea that it's tricky to get a handle on.