5 ms·

I've written two DSLs (one with a team) and I'd consider them both successful. They solved the problem and no one cursed them out. I think the most important fa

I've written two DSLs (one with a team) and I'd consider them both successful. They solved the problem and no one cursed them out. I think the most important factor is they were both small. They were very similar. I even reused the code. One was writing rules to validate giant forms, the other was writing rules to for decisions based on form responses. Ok, just ranting on DSLs. Good DSLs take someone from can't to can. A DSL that's meant to save time is way less likely to be useful because it's very likely to not save you time. In both of my DSLs, it's that we needed to get complex domain behavior into the program. So you either need to teach a programmer the domain, partner a programmer with a domain expert, or teach a domain expert how to program. Putting the power in the hands of the domain expert is attractive when there's a lot of work to be done. It frees up programmers to do other things and tightens the feedback loop. If it's a deep domain, it's not like you want to send your programmer to school to learn how to do this. If it's shallow, you can probably have someone cheaper do it. A DSL comes with a lot of cognitive overhead. If the other option is learning a full programming language, this becomes more reasonable. A time saving DSL is where someone already knows how to write code, they just want to write less of it. This is generally not as good because the savings are marginal. Then when some programmer wants to change something, they have to learn/remember this whole DSL instead of more straightforward code. Actually, this makes a simpler rule of thumb. A DSL for programmers is less likely to be a good idea than a DSL for non-programmers.

f1shy 2y ago
DSL are just great, have never failed for me. I have done it several times, the last one just the past year, for programming complex ASICs. I‘ve seen uncountable times working like a charm. I cannot understand why seems it was bad for him…
- travisjungroth 2y ago
  I think there are two big reasons DSLs often underdeliver. The first is not understanding the cognitive overhead budget. If it's something where it's being used infrequently or by a lot of new people, that's a lot of overhead to be spent each time. Sometimes people think of writing DSLs for tests because they have to write a bunch, but it's really easy for this to suck. I have a test turn red and now I have to learn a DSL to deal with it? Ew. The second is fuzzier. It's putting a DSL over something complex and hoping this will fix things. Writing SQL queries for this system takes a bunch of time and is error prone? Just put a DSL over it! Except all those details and errors are probably going to leak right through your DSL. You want to master the domain before you put a DSL over it.
  - f1shy 2y ago
    >> The first is not understanding the cognitive overhead budget. If it's something where it's being used infrequently or by a lot of new people, that's a lot of overhead to be spent each time. Sometimes people think of writing DSLs for tests because they have to write a bunch, but it's really easy for this to suck. I have a test turn red and now I have to learn a DSL to deal with it? What is the alternative to the DSL with lower cognitive load? I do not follow. Every single DSL I’ve seen REDUCES the cognitive load, by allowing to express the concept in the mere language of the problem at hand, for which the SME should be more than familiar with. About the second point: I see many critics in this thread based on DSls above SQL. Whatever somebody is doing above SQL and selling as a DSL, it is not. Period. I cannot think in any possible way of doing a DSL above a query language. No doubt people hate the idea. Is a BAD one.
    - travisjungroth 2y ago
      > What is the alternative to the DSL with lower cognitive load? In the test example, writing it directly in the programming language. This will usually lead to code that is more verbose and repetitive, but understanding the first example will be faster. I think of cognitive load like a line. X is the number of cases you’re working with, Y is cognitive load [0]. For someone who already knows a programming language, the DSL is going to have a higher Y intercept since you have to learn something new before you understand the first case. Hopefully, it’s a shallower slope so as you deal with more cases the upfront cost gets paid back. If you have lots of people dealing with one case or doing it infrequently enough they have to relearn each time, this payoff never happens. This model extends past DSLs to all abstractions. It’s why people often end up happier with test code that’s less abstract/DRY. The access pattern supports it. Looking at it this way also explains why a DSL for a non-programmer is more likely to be useful. The intercept can be lower than an actual programming language, so you’re ahead from the start. [0] It’s really more of a curve, but the line model works conceptually.
  - epgui 2y ago
    These examples sound like things that aren’t really DSLs… Or in other words it sounds like someone is trying to make something “simpler than it actually is”. DSLs are supposed to be for making it easier to perform computation in a specific context. Software tests have about as many degrees of freedom as the programming language they are written in, so I’m not sure they are an ideal use case for a DSL— not without a lot of discipline at least. For a DSL to make sense, IMHO, you need to be able to write down a complete and correct specification for it. I doubt that is even possible in the given examples :shrug:
- bjourne 2y ago
  There are hundreds of DSLs for ASIC design but not a single one of them has ever been used for actual tapeout. It's 100% unheard of. Hence, I doubt your DSL saved any time over using an RTL language directly. Sorry for sounding harsh, but if you work in the area you understand my skepticism about ASIC design DSLs.
- layer8 2y ago
  Are your DSLs used by other people and do they share your opinion? In my experience DSLs are nice to work with for the creator, but it’s much more work in documentation, training, intelligible error handling, and so on, to make a DSL that’s easy for others to learn and use. I do like DSLs, but the value proposition is often difficult, IMO.
- throw16180339 2y ago
  > I cannot understand why seems it was bad for him… There are many poorly designed libraries, and DSL design is no easier. While I haven’t personally encountered any, I’m sure there are numerous half-baked DSLs out there.
- crabbone 2y ago
  There's a difference between "works once for my very specific problem" and "works most of the time for a wide range of problems". DSLs, in my experience, usually fail in the later definition. It's very hard to make a small language that precisely captures its domain of application, will produce easy to manage programs no matter the size, would be easy to analyze in terms of performance and side-effects.
Majromax 2y ago
I think the theory of domain-specific languages is very valuable, even if there's rarely a need for a full implementation of one. As I see it, a DSL is just the end-state of a programmer creating abstractions and reusable components to ultimately solve the real problem. The nouns and verbs granted by a programming interface constrain how one thinks, so a flexible and intuitive vocabulary and grammar can make the "real program" powerful and easy to maintain. Conversely, a rigid and irregular interface makes the "real program" a brittle maintenance nightmare.
- travisjungroth 2y ago
  I agree. The line between a DSL and regular old programming abstractions is fuzzy. Learning some language design is very helpful because you’ll see every abstraction is a little piece of language.
d0mine 2y ago
Here's a few DSL (for programmers) examples that might make sense (the context is embedding DSLs into Python literal strings): https://news.ycombinator.com/item?id=41214365 https://news.ycombinator.com/item?id=41214365 For example, bash, SQL DSLs may be immediately useful by protecting against shell,sql injection: shutil.run(sh"command {arg}") may translate to subprocess.run(["command", os.fspath(arg)]) No shell--no shell injection. The assumption is that it enables sh"a | b > {c}" syntax (otherwise just call subprocess.run directly). Implementing it in pure Python by hand would be more verbose, less readable, more error-prone).
- travisjungroth 2y ago
  Yes, there are definitely counterexamples. It’s not black and white at all.
lifeisstillgood 2y ago
>>> A DSL for programmers is less likely to be a good idea than a DSL for non-programmers. Nail on the head time - somewhere else in the thread is jooq which is (yet another) SQL DSL where you end up with from(table).where(student=bob) This is a perfect example of why the programmer should (just?) learn SQL instead of the DSL - and your comment nails it
- PaulHoule 2y ago
  (1) JooQ shines for query generation. For instance at work we have a search engine that can search maybe 50 fields that are mostly materialized but involve the occasional subquery. Also you can easily write a template method like transitiveQuery(selectClause,relationshipGenerator,where1,...) and have type inference do the right thing in the compiler and IDE and all of that. You get "hygenic macros", you can write an Java class which is parameterized by one or more related tables which can be specialized by adding more parameters, subclassing, etc. (2) Circa 2005 coding PHP I came to the conclusion that the ORM I needed was insert(table,{col1: val1, col2:val2, ...}) because writing simple inserts and updates against SQL is for birds, let freshers do it and they will forget to escape something. Such a "framework" can be so simple that you can bend it to the needs of your application. JooQ gives you something like that but backed by the Java type system.
  - lifeisstillgood 2y ago
    I do get it (I think!) - but there is a world of difference between “I have 20 years SQL experience and do not want to spend hours maintaining 200 SQL templates, and believe the overhead of this DSL is worth the trade off” vs “use the DSL and you won’t have to teach junior Devs SQL!” My comment is more aimed at the second part. SQL Is tied to the implementation and demands coders understand it all. A DSL can allow domain experts to express their understanding without having to worry about software trade offs. The most successful “DSL” I know of like this is fitnesse tests - just a large number of simple tests where domain experts can spreadsheet style throw in the “gotchas”. Something like that but more spool is fixated is a holy grail - Behaviour driven tests like cucumber come close but there is that weird intermediate translation from English phrase to random function - now you have to understand the function to use the phrase and suddenly you are reading real Code to be able to use the fake code and it never feels clean One day I will be clever enough to be able to write a really good test DSL It’s just whenever I think of “Given used is logged in, visit “textbox” and enter “word” .. it just looks like BDD test not a DSL. Like I said, one day I will be clever enough

f1shy 2y ago

DSL are just great, have never failed for me. I have done it several times, the last one just the past year, for programming complex ASICs. I‘ve seen uncountable times working like a charm. I cannot understand why seems it was bad for him…

travisjungroth 2y ago
I think there are two big reasons DSLs often underdeliver. The first is not understanding the cognitive overhead budget. If it's something where it's being used infrequently or by a lot of new people, that's a lot of overhead to be spent each time. Sometimes people think of writing DSLs for tests because they have to write a bunch, but it's really easy for this to suck. I have a test turn red and now I have to learn a DSL to deal with it? Ew. The second is fuzzier. It's putting a DSL over something complex and hoping this will fix things. Writing SQL queries for this system takes a bunch of time and is error prone? Just put a DSL over it! Except all those details and errors are probably going to leak right through your DSL. You want to master the domain before you put a DSL over it.
- f1shy 2y ago
  >> The first is not understanding the cognitive overhead budget. If it's something where it's being used infrequently or by a lot of new people, that's a lot of overhead to be spent each time. Sometimes people think of writing DSLs for tests because they have to write a bunch, but it's really easy for this to suck. I have a test turn red and now I have to learn a DSL to deal with it? What is the alternative to the DSL with lower cognitive load? I do not follow. Every single DSL I’ve seen REDUCES the cognitive load, by allowing to express the concept in the mere language of the problem at hand, for which the SME should be more than familiar with. About the second point: I see many critics in this thread based on DSls above SQL. Whatever somebody is doing above SQL and selling as a DSL, it is not. Period. I cannot think in any possible way of doing a DSL above a query language. No doubt people hate the idea. Is a BAD one.
  - travisjungroth 2y ago
    > What is the alternative to the DSL with lower cognitive load? In the test example, writing it directly in the programming language. This will usually lead to code that is more verbose and repetitive, but understanding the first example will be faster. I think of cognitive load like a line. X is the number of cases you’re working with, Y is cognitive load [0]. For someone who already knows a programming language, the DSL is going to have a higher Y intercept since you have to learn something new before you understand the first case. Hopefully, it’s a shallower slope so as you deal with more cases the upfront cost gets paid back. If you have lots of people dealing with one case or doing it infrequently enough they have to relearn each time, this payoff never happens. This model extends past DSLs to all abstractions. It’s why people often end up happier with test code that’s less abstract/DRY. The access pattern supports it. Looking at it this way also explains why a DSL for a non-programmer is more likely to be useful. The intercept can be lower than an actual programming language, so you’re ahead from the start. [0] It’s really more of a curve, but the line model works conceptually.
- epgui 2y ago
  These examples sound like things that aren’t really DSLs… Or in other words it sounds like someone is trying to make something “simpler than it actually is”. DSLs are supposed to be for making it easier to perform computation in a specific context. Software tests have about as many degrees of freedom as the programming language they are written in, so I’m not sure they are an ideal use case for a DSL— not without a lot of discipline at least. For a DSL to make sense, IMHO, you need to be able to write down a complete and correct specification for it. I doubt that is even possible in the given examples :shrug:
bjourne 2y ago
There are hundreds of DSLs for ASIC design but not a single one of them has ever been used for actual tapeout. It's 100% unheard of. Hence, I doubt your DSL saved any time over using an RTL language directly. Sorry for sounding harsh, but if you work in the area you understand my skepticism about ASIC design DSLs.
layer8 2y ago
Are your DSLs used by other people and do they share your opinion? In my experience DSLs are nice to work with for the creator, but it’s much more work in documentation, training, intelligible error handling, and so on, to make a DSL that’s easy for others to learn and use. I do like DSLs, but the value proposition is often difficult, IMO.
throw16180339 2y ago
> I cannot understand why seems it was bad for him… There are many poorly designed libraries, and DSL design is no easier. While I haven’t personally encountered any, I’m sure there are numerous half-baked DSLs out there.
crabbone 2y ago
There's a difference between "works once for my very specific problem" and "works most of the time for a wide range of problems". DSLs, in my experience, usually fail in the later definition. It's very hard to make a small language that precisely captures its domain of application, will produce easy to manage programs no matter the size, would be easy to analyze in terms of performance and side-effects.

Majromax 2y ago

I think the theory of domain-specific languages is very valuable, even if there's rarely a need for a full implementation of one. As I see it, a DSL is just the end-state of a programmer creating abstractions and reusable components to ultimately solve the real problem. The nouns and verbs granted by a programming interface constrain how one thinks, so a flexible and intuitive vocabulary and grammar can make the "real program" powerful and easy to maintain. Conversely, a rigid and irregular interface makes the "real program" a brittle maintenance nightmare.

travisjungroth 2y ago
I agree. The line between a DSL and regular old programming abstractions is fuzzy. Learning some language design is very helpful because you’ll see every abstraction is a little piece of language.

d0mine 2y ago

Here's a few DSL (for programmers) examples that might make sense (the context is embedding DSLs into Python literal strings): https://news.ycombinator.com/item?id=41214365 https://news.ycombinator.com/item?id=41214365 For example, bash, SQL DSLs may be immediately useful by protecting against shell,sql injection: shutil.run(sh"command {arg}") may translate to subprocess.run(["command", os.fspath(arg)]) No shell--no shell injection. The assumption is that it enables sh"a | b > {c}" syntax (otherwise just call subprocess.run directly). Implementing it in pure Python by hand would be more verbose, less readable, more error-prone).

travisjungroth 2y ago
Yes, there are definitely counterexamples. It’s not black and white at all.

lifeisstillgood 2y ago

>>> A DSL for programmers is less likely to be a good idea than a DSL for non-programmers. Nail on the head time - somewhere else in the thread is jooq which is (yet another) SQL DSL where you end up with from(table).where(student=bob) This is a perfect example of why the programmer should (just?) learn SQL instead of the DSL - and your comment nails it

PaulHoule 2y ago
(1) JooQ shines for query generation. For instance at work we have a search engine that can search maybe 50 fields that are mostly materialized but involve the occasional subquery. Also you can easily write a template method like transitiveQuery(selectClause,relationshipGenerator,where1,...) and have type inference do the right thing in the compiler and IDE and all of that. You get "hygenic macros", you can write an Java class which is parameterized by one or more related tables which can be specialized by adding more parameters, subclassing, etc. (2) Circa 2005 coding PHP I came to the conclusion that the ORM I needed was insert(table,{col1: val1, col2:val2, ...}) because writing simple inserts and updates against SQL is for birds, let freshers do it and they will forget to escape something. Such a "framework" can be so simple that you can bend it to the needs of your application. JooQ gives you something like that but backed by the Java type system.
- lifeisstillgood 2y ago
  I do get it (I think!) - but there is a world of difference between “I have 20 years SQL experience and do not want to spend hours maintaining 200 SQL templates, and believe the overhead of this DSL is worth the trade off” vs “use the DSL and you won’t have to teach junior Devs SQL!” My comment is more aimed at the second part. SQL Is tied to the implementation and demands coders understand it all. A DSL can allow domain experts to express their understanding without having to worry about software trade offs. The most successful “DSL” I know of like this is fitnesse tests - just a large number of simple tests where domain experts can spreadsheet style throw in the “gotchas”. Something like that but more spool is fixated is a holy grail - Behaviour driven tests like cucumber come close but there is that weird intermediate translation from English phrase to random function - now you have to understand the function to use the phrase and suddenly you are reading real Code to be able to use the fake code and it never feels clean One day I will be clever enough to be able to write a really good test DSL It’s just whenever I think of “Given used is logged in, visit “textbox” and enter “word” .. it just looks like BDD test not a DSL. Like I said, one day I will be clever enough