28 ms·

Fantastic DSLs and where to find them

180 points by SerCe 9y ago

sillysaurus3 9y ago
Anyone else dislike the immutability fad? Mutability has its place, and simply hiding it behind abstractions tacked on to the language (vars, refs, agents, and atoms in Clojure's case) isn't a productive way to deal with it. It has a lot of benefits, but the downsides are significant. Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way. It's the old debate of whether the programmer should be constrained by the language, or the language should serve the programmer. Maybe it's a matter of opinion. Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? What if you want to change it? The cost goes up significantly, and I'm not sure the purported ability to sidestep a few bugs is worth it. If safety is an issue, good test coverage can be a sufficient substitute for types. (The article is excellent, by the way.)
- striking 9y ago
  RE: Typed functions: Kotlin lives on top of a statically typed architecture, so I think you do have to make that function typed. Some languages get around this requirement by inferring types, but type inference itself can then become Turing complete... So it's not like there's a "healthy option". It's more that you're choosing the poison you prefer.
- adrianN 9y ago
  If you have sufficiently good test coverage to replace a type system you have to change tests whenever you'd have to change the type. No costs saved. Thinking more before coding is beneficial in my experience.
- norswap 9y ago
  Agreed that immutability seems overrated right now. The pendulum will swing back in full force, eventually. But of course, the art is in putting mutability and immutability in the right places. There are no fast rules, but in general mutability of local variables is harmless, and you probably need some mutability at the top level as well (as per: "functional core, imperative shell"). Mutability in other places can still be useful though. I don't agree or really follow your comment on function parameters. It seems an argument against typing in general.
  - panic 9y ago
    I think type systems derived from linear/affine/uniqueness typing will help a lot here. Mutability is fine -- it's sharing that's the problem.
  - sillysaurus3 9y ago
    Re: function parameters, here's a concrete example. Clojure's `reduce` takes a function `f` as an argument: https://clojuredocs.org/clojure.core/reduce https://clojuredocs.org/clojure.core/reduce When you pass an empty list, `f` is called with no arguments. That way, `(reduce + [])` can return 0, whereas `(reduce * [])` can return 1. It's very easy to write such a function in Clojure, thanks to a lack of types. What would that look like in Kotlin? A function that can either take zero arguments or takes two arguments of types specified by the input list?
    - Nullabillity 9y ago
      The usual implementation of fold/reduce takes a separate seed argument (as does clojure, optionally!), which IMO is far more sensible than having the same function do two completely separate things depending on the number of arguments.
      - deleted 9y ago
        [deleted]
      - sillysaurus3 9y ago
        Sure, let's roll with that. How would you write a function parameter that takes two arguments, whose type is determined by the seed argument? Also, with reduce, the return value of the function can determine what the type of the input arguments should be. For example if you pass in a `(fn (x y) (list x y))`, you end up with: > (reduce (fn (x y) (list x y)) (list "a" "b" "c" "d")) ("a" ("b" ("c" "d"))) Let's throw in a print statement to print out the `y` parameter: > (reduce (fn (x y) (print (str y)) (list x y)) (list "a" "b" "c" "d")) "d" ("c" "d") ("b" ("c" "d")) ("a" ("b" ("c" "d"))) So `y` starts out as a string, then a list of strings, then a list whose first element is a string and the second element is a list of strings, and so on. I'm not sure what the type of that function would even look like. And if there's no way to express something as simple as `reduce` without resorting to `f: (x: Any, y: Any) -> Any`, are we certain it's good design?
- nicktelford 9y ago
  Immutability is useful as a default, but languages should absolutely provide mutable alternatives for situations the programmer deems appropriate. The reason immutability has become popular so recently, and the reason why I say it's the best default assumption is the rise in multi-threaded programming. There are a growing number of strategies for dealing with mutability in a multi-threaded environment in a way that is both safe, and intuitive to reason about, the two that springs to mind are the rediscovery of actors (e.g. Akka), and go's goroutines. Mutability certainly has its place, but so does immutability. Previously, immutability has been largely ignored in favour of the convenience of mutablity. I wouldn't say immutability is being heavily favoured now; more that it's back where it belongs, as part of a binary choice.
  - tdb7893 9y ago
    Even in single threaded programs immutability can make things easier to reason about. I've had to look at some old Java projects and the easiest time I've had reasoning about them was a project that used mostly immutable objects. Ever since then I've used immutable as the default for my Java objects and it's been better for me. Your mileage may vary obviously
  - lostcolony 9y ago
    But even actors benefit from immutability. Imagine if the message you received and matched on was not the message you processed (because it was mutated in between). You either want immutability at least for message handling, or messages to be full copies, and there are tradeoffs to either choice.
    - nicktelford 9y ago
      Messages, of course, should be immutable for exactly those reasons. Actors encapsulate mutable state; they essentially serialize access to that state such that access is single-threaded.
      - lostcolony 9y ago
        Actors don't necessarily encapsulate mutable state; they may in fact encapsulate immutable state. An actor can be treated (as in Erlang) as essentially just a(n optionally) recursive function with a mailbox, where mutation can only occur across the function boundary (i.e., when you recurse, you pass in the new value). At this point the only benefit to allowing mutability is to allow you convenience things like for loops, and for performance reasons. From the perspective outside of the actor, the state behaves the same way whether it's mutable or immutable; you do not gain new behaviors by making it mutable (i.e., for sharing data or similar), you just make a more complex coding model (in that you're mixing mutable and immutable and have to know which is which).
- zeptomu 9y ago
  If your data is immutable or not is significant for program analysis and correctness, so I would say the distinction makes absolutely sense. > Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way. Languages that provide immutable data structures have often 2 variants, an immutable and mutable. Why can't you just take the mutable in that case? I think it's more about making immutability or mutability explicit.
- pmontra 9y ago
  You can have both mutability and immutability in the same language. Example from Ruby: $ irb 2.4.1 :001 > a = [1, 2, 3] => [1, 2, 3] 2.4.1 :002 > a.map {|x| x + 1} => [2, 3, 4] 2.4.1 :003 > a => [1, 2, 3] 2.4.1 :004 > a.map! {|x| x + 1} => [2, 3, 4] 2.4.1 :005 > a => [2, 3, 4] 2.4.1 :006 > a.freeze # make a immutable => [2, 3, 4] 2.4.1 :007 > a.map! {|x| x + 1} RuntimeError: can't modify frozen Array Ruby is fundamentally a language based on mutability, but it shows that it should be possible to have both ways. Questions: 1) Is really there a language which is fully agnostic about mutability and immutability? 2) If not, it's maybe because designers have strong opinions about this matter? Actually IMHO designing a fully agnostic language would demonstrate strong opinions too.
  - moosingin3space 9y ago
    I'd argue most languages are agnostic about (im)mutability. It's libraries that aren't.
    - pmontra 9y ago
      Most but some don't. This fails in Erlang X = X + 1. because even variables are immutable. This is ok in Elixir even if it kind of fakes it [1] x = x + 1 and it's perfectly ok in Ruby and most other languages. [1] http://blog.plataformatec.com.br/2016/01/comparing-elixir-and-erlang-variables/ http://blog.plataformatec.com.br/2016/01/comparing-elixir-an...
- typon 9y ago
  > It's the old debate of whether the programmer should be constrained by the language, or the language should serve the programmer. Maybe it's a matter of opinion. I like to think of it as the language saving the programmer from herself
- sporkenfang 9y ago
  > Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? What if you want to change it? This is actually my favorite thing about languages with strong type systems where type inference is less emphasized -- if you're looking at a function definition, even if you're not terribly familiar with the language, you know what the thing is supposed to do. Sure, it's nice to let the compiler infer your types at compile or runtime, but it's really nice to just be able to read the code and be able to see what a function is at least meant to do, based on what it returns.
  - mpweiher 9y ago
    ...and in fact this documentation effect is (so far) the only positive effect of static typing that has had at least some scientific validation.
    - reitanqild 9y ago
      ? This sounds really fishy to me. Do you have more details? Source: have programmed professionally in major static and non-static languages and as the years pass I appreciate static languages more and more for each time I waste time debugging what I have learned to think of as stupid, completely avoidable errors.
- reitanqild 9y ago
  Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? (Java programmer here.) Then return a sufficiently "broad" type up to, if necessary, Object. That said I almost never fall that far back, I'd see that as a sign that I should stop and think. Oh, and the advantage of a typed languages like Java: you can have extremely good tool support. Netbeans (or IntelliJ or Eclipse) will happily help you to change types if it later turns out you were wrong.
- colanderman 9y ago
  > What if you want to change it? If you need to change the argument/return types of a function, you have a LOT more work to do than just changing a one-line signature.
- lomnakkus 9y ago
  > What if you want to change it? Then you change it and let the compiler/IDE help you find all the other bits of code you need to change. I don't see the problem.
- zintinio5 9y ago
  Immutability is absolutely fantastic as a default (especially for typical business logic). I'm not sure I want to go back to mutability except in constrained environments or time-sensitive software. Mutability should be harder to reach for because it forces you to think hard about the implications to other parts of your program. I can think of a number of times we've had to parallelize a routine or share resources between workers, and it has been almost trivial _because_ of immutability. In contrast to that, I inherited a hairball of C code using pthreads, mutexes, global variables, etc which had to be completely gutted and rewritten as it was impossible to understand the dependencies in the software (we were witnessing segfaults, deadlock, etc).
- kqr 9y ago
  > Anyone else dislike the immutability fad? [...] Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way. Immutability does not prevent you from incrementally creating new versions of a map! At this point, you are conflating no less than three different and orthogonal concepts: A) A convenient way to write code that incrementally creates new versions of a value. This is trivial in most languages using immutable values, but it often requires unfamiliar idioms, giving people the false impression it cannot be done. B) A performance optimisation that performs such local incremental updates in place. This is a real concern. Any immutable-by-default language should support a safe (!!) way for the programmer to ensure updates happen in place. C) A mechanism whereby new versions of values can be distributed to other parts of a program. This one is tricky as all hell to do safely, and most good solutions work just as well with immutable values. "But kqr, why did you take something so simple and made it so complicated?" It was complicated from the start. Traditional languages have just ignored this complexity and solved it in an "undefined behaviour" sort of way: let's just do wha the metal happens to do, however unsafe!" You'll note tha such languages have no way to do safe in place updates, and value propagation is never sychronised or otherwise guaranteed not to cause partial views. It's a low-level hack that's far outstayed its welcome in the age of huge systems with all sorts of interaction effects.
  - mpweiher 9y ago
    > It was complicated from the start. No it wasn't. It's only "complicated" from a certain very specific and constrained POV, which you seem to have adopted wholesale and now assume as the only valid reality. Mutability is not just how "the metal" works, it's also how the rest of the world works. If I eat a banana, the contents of my stomach (and by extension I) change. You don't get a new me that's no longer hungry + the old me that hasn't changed. If I put gas in my car's tank, the car changes. I don't get a new car. Etc. You are also conflating immutability with atomic updates, as far as I can tell.
    - inopinatus 9y ago
      You are perceiving yourself as having mutated after eating the banana, merely because you are in temporal lockstep with your stomach. Most of what you've just stated as "fact" and "how the world works" is a matter of perception. Time, after all, is (we believe) simply another dimension of spacetime.
- emidln 9y ago
  > Mutability has its place, and simply hiding it behind abstractions tacked on to the language (vars, refs, agents, and atoms in Clojure's case) isn't a productive way to deal with it. In Clojure if you want mutability, you don't use an PersistentHashMap i.e. `{}` inside an atom `(atom {})`, you import and instantiate a mutable java class `java.util.concurrent.ConcurrentHashMap` or `java.util.HashMap` `(let [m (doto (java.util.HashMap.) (.put "foo" "bar") (.put "spam "eggs"))] .... )` and bang on it just like you would in Java. Clojure doesn't attempt to solve mutability because the host platform has already done a good job at that. Clojure provides semantics around state managment of persistent data if you need that sort of thing, but that's not necessarily a good replacement for mutability if you actually need a mutable thing.
- codecurve 9y ago
  Few points on immutability and Clojure. The abstraction that hides mutable data structures is transient[1], not vars, refs, agents and atoms. Those abstractions are all to do with mutable references, not mutable data. They're there to make sure that you have safe ways to coordinate state in concurrent environments. I agree that immutability can be annoying in languages that weren't designed with it in mind, but the languages that were almost always make it easier, safer and performant to create copies rather than mutate data. If you're writing Clojure or Haskell and you think you're going to save yourself time by "just creating a map and mutating its structure" then you're misunderstanding the purpose of the language. These constraints are what enable the guarantees and contracts those languages make. [1]: https://clojure.org/reference/transients https://clojure.org/reference/transients
  - sillysaurus3 9y ago
    But it does save time. JS is proof of that. There are no immutable data structures and the world rolls on. People will hate on JS, but it's effective. You can learn to get into the "immutability mindset" if you train yourself to, but are you certain it's worth the time investment? It seems like there's at least a chance that it's not.
    - codecurve 9y ago
      Sure, no arguments there. It does save time in JavaScript and a large part of that is because the language has been designed around mutability. Part of that trade-off is that JavaScript can't make the same guarantees about what happens when you pass an object into a function. It's harder to be confident that a given program is correct. Immutability is just a part of the "simple made easy"[1] ethos of Clojure and I think most Clojure programmers will argue that taking the time to understand that philosophy _is_ worth the investment. [1] https://www.youtube.com/watch?v=rI8tNMsozo0 https://www.youtube.com/watch?v=rI8tNMsozo0
      - ruleabidinguser 9y ago
        Ah, so you see why there are not many clojure programmers
- borplk 9y ago
  You pay for not specifying those parameters and so on in other ways later on. By leaving those things out you are broadening the problem model and denying the compiler information that it needs from you as the programmer so that it can do a good job of solving the problem. You have to remember that the computer can't synthesise knowledge about the problem domain that you deny it in the first place. So sure tightening things down by specifying them may be tedious but it is the reality of the problem that you are solving. Convenience of leaving it out will translate to instability such as run-time exceptions not to mention a massively reduced ability in tooling support (like IDE auto-complete).
- dtech 9y ago
  > Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? Why are you writing the code if you don't know yet what your input will be? A function transforms input into output, how can it do that correctly when you don't know what its input is? If you don't know the input type yet, you should first be writing the things that will deliver the input, or leave the function as an unimplemented placeholder and define only the output type. > What if you want to change it This is one of the best reasons to use types. Directly after the change you'll be told by the compiler which other code needs changes. Compare this to un-typed languages where you'll need a lot of tests to ensure the same or risk runtime errors.
norswap 9y ago
The crispness and terseness is definitely a big part of Kotlin's appeal. What I find suprising is that these ideas don't exactly seem breakthrough. Why aren't more language implementing them or taking them further?
- pmontra 9y ago
  Because they're not new. Check this XML DSL for Ruby, March 2008 http://www.codecommit.com/blog/ruby/xmlbuilder-a-ruby-dsl-case-study http://www.codecommit.com/blog/ruby/xmlbuilder-a-ruby-dsl-ca... What's happened here is that it's being done in a statically typed language which is harder than in a dynamically typed on. So, good job done.
  - tormeh 9y ago
    It's not new in statically typed languages either. There are already too many DSLs in Scala...
athenot 9y ago
(For those who were wondering, the article is about DSLs in Kotlin). DSLs can be very elegant ways to improve code readability (as long as the assumptions and language they use is meaningful to the team). This is an area that I wish node.js had invested in. Perl6 has "slangs" which are extremely powerful, see [1] for some examples that just flow elegantly. Combine that with custom operators to write code like "4.7kΩ ± 5%" that would make sense to electronicians [2] and you can get some really user-friendly syntaxes. (Again, it bears repeating that a DSL improves readability by sacrificing generality, so a DSL is for audience that's usually a subset of that language's users. That electronics code would be useless to the average web developer.) [1] https://github.com/tony-o/perl6-slang-sql https://github.com/tony-o/perl6-slang-sql [2] https://perl6advent.wordpress.com/tag/dsl/ https://perl6advent.wordpress.com/tag/dsl/
- kqr 9y ago
  > a DSL improves readability by sacrificing generality, so a DSL is for audience that's usually a subset of that language's users. You'd think that'd be self-evident from the "domain specific" in the name, but I guess a lot of things are obvious once they're pointed out to you...
- wahern 9y ago
  Wow. The Perl6 slang feature is awesome. I can't believe I missed it. I haven't had a chance to use Perl 6 but I would really like an excuse. I love Lua's LPeg module. Perl 6 has deeply embedded PEGs (i.e. Grammars, aka parsing combinators) into it's design. Knowing first-hand how powerful PEGs can be if the engine and language integration is implemented well, I've never doubted the awesomeness and elegance of Perl 6. It's unclear to me if Perl 6 Grammars' Action feature is as powerful as LPeg's capturing and inline transformation primitives[1], but the fact that you can plug a grammar into Perl 6's parser is crazy awesome. I'm not surprised Slang exists--I've known it was possible; just surprised that it's a module and idiomatic pattern and I hadn't read about it before. [1] See http://www.inf.puc-rio.br/~roberto/lpeg/#captures http://www.inf.puc-rio.br/~roberto/lpeg/#captures Most PEG engines just return your raw parse tree as a complete data structure and require you to fix it up manually. LPeg has really well thought-out extensions that allow you to express capture and tree transformations much more naturally; specifically, inline (both syntactically and runtime execution) with the pattern definition(s).
agentgt 9y ago
I used to be fervent embedded DSL fan many years ago (particularly in college with Scheme/Racket) but since then I have grown to hate them particularly Scala, Groovy and Ruby DSLs. External DSLs (that is a DSL independent of the host language)... I love them. An external DSL is like a protocol or the ultimate formal API (an example would be HTTP). Why do I dislike embedded DSLs: I guess the crux could be that you are essentially mixing two languages when the reality it is just one so you are doing non canonical things in the host to make the guest language look pretty (eg going to town with implicits, extension functions, and operator overloading (scala)). Basically a whole bunch of confusing stuff to normal developers just to make the DSL look pretty. With Scheme/Lisp/Clojure it is OK because embedded DSLs using sexp look like lisp. In Haskell it is sort of OK because Monads and various other things Haskell provides (laziness). With Scala its the absolute worse with operator overloading, implicits, and various other gotchas. In my experience it also gets really confusing trying to figure what is going on when a embedded DSL breakk as well as extremely difficult to secure an embedded DSL. `ie System.exit(1)` is just one call away. There are exceptions of course. DSLs that mimic external DSLs one-to-one. One example being jOOQ as it aims to be one to one with an external DSL ... SQL.
- pluma 9y ago
  Embedded DSLs often seem like the typical "bike shed coding styles" taken to the extreme. In my opinion they often value form over function, creating an entirely new (subset of the) language just because. I can obsess about code formatting as much as anybody else but creating my own subset of the language just for a specific domain just seems excessive. It puts additional burden on anyone else trying to work with it because they now not only have to worry about the "host" language but also the idiosyncrasies of the embedded DSL.
  - swsieber 9y ago
    Counter argument for statically enforced dsl (e.g. Kotlin): I have an idea for a DSL - parse the templates for a site to generate type information about what buttons, inputs, interpolated values, etc can be found on each page. Combine that with parsed route information to allow one to generate dsl binding for a all the pages / routes. If we're doing that with Kotlin, then afterwards we can get a really nice, statically enforced testing api (for Selenium, etc), with IDE completion. And you don't even have to run them to make sure that the element selectors work (barring of course any bugs in your parsing setup). I had this idea several months ago but haven't had the opportunity to use it. I'm itching to because it seems like a feasible, if not slightly insane, way to making writing automated end to end tests easier to write and maintain. tl;dr if you're going to do some code gen for code generation, why not make the interface a DSL?
lasermike026 9y ago
I like ruby's rspec and Rake. Very powerful.
lostcolony 9y ago
Out of curiosity (because this is what I was hoping from from the article), has anyone found a DSL in the wild that actually achieved its objectives? That is, it either simplified the code necessary to write a program that there was a net savings for the project compared with developers just writing it in a language they were familiar with, or it simplified the business logic aspect to such a degree that even non-devs were able to be productive with it? I admit this is partly determined by what we decide a DSL is, since it is, ultimately, at some level just an abstraction. But almost every instance I'd squarely say "that was an attempt at a DSL", it became just as complex as the underlying language, with additional gotchas, if it even 'worked' at all (i.e., allowed you to solve the domain specific problems it was intended to solve).
- agency 9y ago
  This may partly be Stockholm syndrome, but I'm actually somewhat fond of Gradle (a Groovy DSL for configuring builds/managing dependencies for Java projects). It's certainly not perfect and I think it would be overkill for simple builds, but I find the DSL well suited to writing custom build tasks. As an example, I wrote a Gradle plugin for a project I work on that lets you write: apply plugin: 'mycompany.service' mainClassName = 'mycompany.Main' service { id = 'serviceid' port = 1234 } This results in a `buildImage` task being added which builds a Docker image that runs `mycompany.Main` and exposes the specified port (and does some Docker tagging with the service id). Implementing this plugin was straightforward, less than 50 lines of Groovy, most of which is invoking another DSL[1] for creating a Dockerfile. [1] https://github.com/bmuschko/gradle-docker-plugin#creating-a-dockerfile-and-building-an-image https://github.com/bmuschko/gradle-docker-plugin#creating-a-...
  - vorg 9y ago
    > Gradle (a Groovy DSL Gradle is not a Groovy DSL. Gradle enables you to write its build files in either Kotlin or Apache Groovy, and perhaps there'll be more choices later on. > Implementing this plugin was straightforward, less than 50 lines of Groovy Did you know Gradleware now recommend using Kotlin for writing plugins for Gradle versions 3.0 and later?
didibus 9y ago
In CS, there are no good and bad things, there are only trade offs. DSLs are only good when the effort to learn to be proficient in the DSL repays the time it took you to learn it. Otherwise, the cognitive load of learning the semantics and the challenges often with debugging are not worth it. An example of a great DSL is linq in C#. List comprehensions in Clojure. JQuery in JavaScript. Etc.
Osmium 9y ago
> Unlike many other languages, it allows developers to create another language inside it. Is this true? e.g. Swift has extensions on generic types too, doesn't it, and could also be used to write DSLs? (Sincere question, asking because I genuinely don't know how rare this kind of functionality is)
robotpony 9y ago
DSLs can be great abstractions and interfaces, in that they can provide a way to state program knowledge in a form that non-programmers may be able to work with more easily. One example that stuck with me early was the Quake (and Doom) shader files, as well as the other game resource constructs from those old WAD-based games. The shader syntax wasn't much more than a rough wrapper on the program's `struct`s, but it allowed the graphic artists to twiddle with the resources manually (and later made a great program interface for the level editors). Quake map files were [pretty neat too](https://quakewiki.org/wiki/Quake_Map_Format https://quakewiki.org/wiki/Quake_Map_Format), compiling down to playable levels as BSPs (binary space partitioning files). XML (specific to document interchange), HTML (specific to hypertext interchange), and CSS (specific to DOM settings) were all DSLs that came about over the course of browser evolution. They offered ways to define content and configuration in a way that was both easy to understand and somewhat isolated from the underlying source code. I've developed several DSLs in various systems I've worked in. Mostly you're pushing stuff out of the source code that doesn't belong there: configuration, repeated definitions, and things that may change from installation to installation. One of the abstractions we built was a series of custom state machine scripting languages that mapped to serial and TCP/IP protocols in a way that reduced boilerplate, and made the guts of implementing specific communications scenarios easier. The amount of time spent in developing a DSL was generally many times smaller than the gain in flexibility, transparency, and exposure to who could interact with customizing the system.
ruleabidinguser 9y ago
The immutable data structures example was revolting

180 points by SerCe 9y ago

Anyone else dislike the immutability fad? Mutability has its place, and simply hiding it behind abstractions tacked on to the language (vars, refs, agents, and atoms in Clojure's case) isn't a productive way to deal with it. It has a lot of benefits, but the downsides are significant. Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way. It's the old debate of whether the programmer should be constrained by the language, or the language should serve the programmer. Maybe it's a matter of opinion. Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? What if you want to change it? The cost goes up significantly, and I'm not sure the purported ability to sidestep a few bugs is worth it. If safety is an issue, good test coverage can be a sufficient substitute for types. (The article is excellent, by the way.)

striking 9y ago
RE: Typed functions: Kotlin lives on top of a statically typed architecture, so I think you do have to make that function typed. Some languages get around this requirement by inferring types, but type inference itself can then become Turing complete... So it's not like there's a "healthy option". It's more that you're choosing the poison you prefer.
adrianN 9y ago
If you have sufficiently good test coverage to replace a type system you have to change tests whenever you'd have to change the type. No costs saved. Thinking more before coding is beneficial in my experience.
norswap 9y ago
Agreed that immutability seems overrated right now. The pendulum will swing back in full force, eventually. But of course, the art is in putting mutability and immutability in the right places. There are no fast rules, but in general mutability of local variables is harmless, and you probably need some mutability at the top level as well (as per: "functional core, imperative shell"). Mutability in other places can still be useful though. I don't agree or really follow your comment on function parameters. It seems an argument against typing in general.
- panic 9y ago
  I think type systems derived from linear/affine/uniqueness typing will help a lot here. Mutability is fine -- it's sharing that's the problem.
- sillysaurus3 9y ago
  Re: function parameters, here's a concrete example. Clojure's `reduce` takes a function `f` as an argument: https://clojuredocs.org/clojure.core/reduce https://clojuredocs.org/clojure.core/reduce When you pass an empty list, `f` is called with no arguments. That way, `(reduce + [])` can return 0, whereas `(reduce * [])` can return 1. It's very easy to write such a function in Clojure, thanks to a lack of types. What would that look like in Kotlin? A function that can either take zero arguments or takes two arguments of types specified by the input list?
  - Nullabillity 9y ago
    The usual implementation of fold/reduce takes a separate seed argument (as does clojure, optionally!), which IMO is far more sensible than having the same function do two completely separate things depending on the number of arguments.
    - deleted 9y ago
      [deleted]
    - sillysaurus3 9y ago
      Sure, let's roll with that. How would you write a function parameter that takes two arguments, whose type is determined by the seed argument? Also, with reduce, the return value of the function can determine what the type of the input arguments should be. For example if you pass in a `(fn (x y) (list x y))`, you end up with: > (reduce (fn (x y) (list x y)) (list "a" "b" "c" "d")) ("a" ("b" ("c" "d"))) Let's throw in a print statement to print out the `y` parameter: > (reduce (fn (x y) (print (str y)) (list x y)) (list "a" "b" "c" "d")) "d" ("c" "d") ("b" ("c" "d")) ("a" ("b" ("c" "d"))) So `y` starts out as a string, then a list of strings, then a list whose first element is a string and the second element is a list of strings, and so on. I'm not sure what the type of that function would even look like. And if there's no way to express something as simple as `reduce` without resorting to `f: (x: Any, y: Any) -> Any`, are we certain it's good design?
nicktelford 9y ago
Immutability is useful as a default, but languages should absolutely provide mutable alternatives for situations the programmer deems appropriate. The reason immutability has become popular so recently, and the reason why I say it's the best default assumption is the rise in multi-threaded programming. There are a growing number of strategies for dealing with mutability in a multi-threaded environment in a way that is both safe, and intuitive to reason about, the two that springs to mind are the rediscovery of actors (e.g. Akka), and go's goroutines. Mutability certainly has its place, but so does immutability. Previously, immutability has been largely ignored in favour of the convenience of mutablity. I wouldn't say immutability is being heavily favoured now; more that it's back where it belongs, as part of a binary choice.
- tdb7893 9y ago
  Even in single threaded programs immutability can make things easier to reason about. I've had to look at some old Java projects and the easiest time I've had reasoning about them was a project that used mostly immutable objects. Ever since then I've used immutable as the default for my Java objects and it's been better for me. Your mileage may vary obviously
- lostcolony 9y ago
  But even actors benefit from immutability. Imagine if the message you received and matched on was not the message you processed (because it was mutated in between). You either want immutability at least for message handling, or messages to be full copies, and there are tradeoffs to either choice.
  - nicktelford 9y ago
    Messages, of course, should be immutable for exactly those reasons. Actors encapsulate mutable state; they essentially serialize access to that state such that access is single-threaded.
    - lostcolony 9y ago
      Actors don't necessarily encapsulate mutable state; they may in fact encapsulate immutable state. An actor can be treated (as in Erlang) as essentially just a(n optionally) recursive function with a mailbox, where mutation can only occur across the function boundary (i.e., when you recurse, you pass in the new value). At this point the only benefit to allowing mutability is to allow you convenience things like for loops, and for performance reasons. From the perspective outside of the actor, the state behaves the same way whether it's mutable or immutable; you do not gain new behaviors by making it mutable (i.e., for sharing data or similar), you just make a more complex coding model (in that you're mixing mutable and immutable and have to know which is which).
zeptomu 9y ago
If your data is immutable or not is significant for program analysis and correctness, so I would say the distinction makes absolutely sense. > Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way. Languages that provide immutable data structures have often 2 variants, an immutable and mutable. Why can't you just take the mutable in that case? I think it's more about making immutability or mutability explicit.
pmontra 9y ago
You can have both mutability and immutability in the same language. Example from Ruby: $ irb 2.4.1 :001 > a = [1, 2, 3] => [1, 2, 3] 2.4.1 :002 > a.map {|x| x + 1} => [2, 3, 4] 2.4.1 :003 > a => [1, 2, 3] 2.4.1 :004 > a.map! {|x| x + 1} => [2, 3, 4] 2.4.1 :005 > a => [2, 3, 4] 2.4.1 :006 > a.freeze # make a immutable => [2, 3, 4] 2.4.1 :007 > a.map! {|x| x + 1} RuntimeError: can't modify frozen Array Ruby is fundamentally a language based on mutability, but it shows that it should be possible to have both ways. Questions: 1) Is really there a language which is fully agnostic about mutability and immutability? 2) If not, it's maybe because designers have strong opinions about this matter? Actually IMHO designing a fully agnostic language would demonstrate strong opinions too.
- moosingin3space 9y ago
  I'd argue most languages are agnostic about (im)mutability. It's libraries that aren't.
  - pmontra 9y ago
    Most but some don't. This fails in Erlang X = X + 1. because even variables are immutable. This is ok in Elixir even if it kind of fakes it [1] x = x + 1 and it's perfectly ok in Ruby and most other languages. [1] http://blog.plataformatec.com.br/2016/01/comparing-elixir-and-erlang-variables/ http://blog.plataformatec.com.br/2016/01/comparing-elixir-an...
typon 9y ago
> It's the old debate of whether the programmer should be constrained by the language, or the language should serve the programmer. Maybe it's a matter of opinion. I like to think of it as the language saving the programmer from herself
sporkenfang 9y ago
> Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? What if you want to change it? This is actually my favorite thing about languages with strong type systems where type inference is less emphasized -- if you're looking at a function definition, even if you're not terribly familiar with the language, you know what the thing is supposed to do. Sure, it's nice to let the compiler infer your types at compile or runtime, but it's really nice to just be able to read the code and be able to see what a function is at least meant to do, based on what it returns.
- mpweiher 9y ago
  ...and in fact this documentation effect is (so far) the only positive effect of static typing that has had at least some scientific validation.
  - reitanqild 9y ago
    ? This sounds really fishy to me. Do you have more details? Source: have programmed professionally in major static and non-static languages and as the years pass I appreciate static languages more and more for each time I waste time debugging what I have learned to think of as stupid, completely avoidable errors.
reitanqild 9y ago
Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? (Java programmer here.) Then return a sufficiently "broad" type up to, if necessary, Object. That said I almost never fall that far back, I'd see that as a sign that I should stop and think. Oh, and the advantage of a typed languages like Java: you can have extremely good tool support. Netbeans (or IntelliJ or Eclipse) will happily help you to change types if it later turns out you were wrong.
colanderman 9y ago
> What if you want to change it? If you need to change the argument/return types of a function, you have a LOT more work to do than just changing a one-line signature.
lomnakkus 9y ago
> What if you want to change it? Then you change it and let the compiler/IDE help you find all the other bits of code you need to change. I don't see the problem.
zintinio5 9y ago
Immutability is absolutely fantastic as a default (especially for typical business logic). I'm not sure I want to go back to mutability except in constrained environments or time-sensitive software. Mutability should be harder to reach for because it forces you to think hard about the implications to other parts of your program. I can think of a number of times we've had to parallelize a routine or share resources between workers, and it has been almost trivial _because_ of immutability. In contrast to that, I inherited a hairball of C code using pthreads, mutexes, global variables, etc which had to be completely gutted and rewritten as it was impossible to understand the dependencies in the software (we were witnessing segfaults, deadlock, etc).
kqr 9y ago
> Anyone else dislike the immutability fad? [...] Sometimes I just want to create a map and mutate its structure, and the language saying "no" is constraining me in an unpleasant way. Immutability does not prevent you from incrementally creating new versions of a map! At this point, you are conflating no less than three different and orthogonal concepts: A) A convenient way to write code that incrementally creates new versions of a value. This is trivial in most languages using immutable values, but it often requires unfamiliar idioms, giving people the false impression it cannot be done. B) A performance optimisation that performs such local incremental updates in place. This is a real concern. Any immutable-by-default language should support a safe (!!) way for the programmer to ensure updates happen in place. C) A mechanism whereby new versions of values can be distributed to other parts of a program. This one is tricky as all hell to do safely, and most good solutions work just as well with immutable values. "But kqr, why did you take something so simple and made it so complicated?" It was complicated from the start. Traditional languages have just ignored this complexity and solved it in an "undefined behaviour" sort of way: let's just do wha the metal happens to do, however unsafe!" You'll note tha such languages have no way to do safe in place updates, and value propagation is never sychronised or otherwise guaranteed not to cause partial views. It's a low-level hack that's far outstayed its welcome in the age of huge systems with all sorts of interaction effects.
- mpweiher 9y ago
  > It was complicated from the start. No it wasn't. It's only "complicated" from a certain very specific and constrained POV, which you seem to have adopted wholesale and now assume as the only valid reality. Mutability is not just how "the metal" works, it's also how the rest of the world works. If I eat a banana, the contents of my stomach (and by extension I) change. You don't get a new me that's no longer hungry + the old me that hasn't changed. If I put gas in my car's tank, the car changes. I don't get a new car. Etc. You are also conflating immutability with atomic updates, as far as I can tell.
  - inopinatus 9y ago
    You are perceiving yourself as having mutated after eating the banana, merely because you are in temporal lockstep with your stomach. Most of what you've just stated as "fact" and "how the world works" is a matter of perception. Time, after all, is (we believe) simply another dimension of spacetime.
emidln 9y ago
> Mutability has its place, and simply hiding it behind abstractions tacked on to the language (vars, refs, agents, and atoms in Clojure's case) isn't a productive way to deal with it. In Clojure if you want mutability, you don't use an PersistentHashMap i.e. `{}` inside an atom `(atom {})`, you import and instantiate a mutable java class `java.util.concurrent.ConcurrentHashMap` or `java.util.HashMap` `(let [m (doto (java.util.HashMap.) (.put "foo" "bar") (.put "spam "eggs"))] .... )` and bang on it just like you would in Java. Clojure doesn't attempt to solve mutability because the host platform has already done a good job at that. Clojure provides semantics around state managment of persistent data if you need that sort of thing, but that's not necessarily a good replacement for mutability if you actually need a mutable thing.
codecurve 9y ago
Few points on immutability and Clojure. The abstraction that hides mutable data structures is transient[1], not vars, refs, agents and atoms. Those abstractions are all to do with mutable references, not mutable data. They're there to make sure that you have safe ways to coordinate state in concurrent environments. I agree that immutability can be annoying in languages that weren't designed with it in mind, but the languages that were almost always make it easier, safer and performant to create copies rather than mutate data. If you're writing Clojure or Haskell and you think you're going to save yourself time by "just creating a map and mutating its structure" then you're misunderstanding the purpose of the language. These constraints are what enable the guarantees and contracts those languages make. [1]: https://clojure.org/reference/transients https://clojure.org/reference/transients
- sillysaurus3 9y ago
  But it does save time. JS is proof of that. There are no immutable data structures and the world rolls on. People will hate on JS, but it's effective. You can learn to get into the "immutability mindset" if you train yourself to, but are you certain it's worth the time investment? It seems like there's at least a chance that it's not.
  - codecurve 9y ago
    Sure, no arguments there. It does save time in JavaScript and a large part of that is because the language has been designed around mutability. Part of that trade-off is that JavaScript can't make the same guarantees about what happens when you pass an object into a function. It's harder to be confident that a given program is correct. Immutability is just a part of the "simple made easy"[1] ethos of Clojure and I think most Clojure programmers will argue that taking the time to understand that philosophy _is_ worth the investment. [1] https://www.youtube.com/watch?v=rI8tNMsozo0 https://www.youtube.com/watch?v=rI8tNMsozo0
    - ruleabidinguser 9y ago
      Ah, so you see why there are not many clojure programmers
borplk 9y ago
You pay for not specifying those parameters and so on in other ways later on. By leaving those things out you are broadening the problem model and denying the compiler information that it needs from you as the programmer so that it can do a good job of solving the problem. You have to remember that the computer can't synthesise knowledge about the problem domain that you deny it in the first place. So sure tightening things down by specifying them may be tedious but it is the reality of the problem that you are solving. Convenience of leaving it out will translate to instability such as run-time exceptions not to mention a massively reduced ability in tooling support (like IDE auto-complete).
dtech 9y ago
> Also, typed function parameters are painful. Declaring a parameter as a `f: () -> int` instead of `f` requires that you think about what the signature needs to be. What if you don't know? Why are you writing the code if you don't know yet what your input will be? A function transforms input into output, how can it do that correctly when you don't know what its input is? If you don't know the input type yet, you should first be writing the things that will deliver the input, or leave the function as an unimplemented placeholder and define only the output type. > What if you want to change it This is one of the best reasons to use types. Directly after the change you'll be told by the compiler which other code needs changes. Compare this to un-typed languages where you'll need a lot of tests to ensure the same or risk runtime errors.

norswap 9y ago

The crispness and terseness is definitely a big part of Kotlin's appeal. What I find suprising is that these ideas don't exactly seem breakthrough. Why aren't more language implementing them or taking them further?

pmontra 9y ago
Because they're not new. Check this XML DSL for Ruby, March 2008 http://www.codecommit.com/blog/ruby/xmlbuilder-a-ruby-dsl-case-study http://www.codecommit.com/blog/ruby/xmlbuilder-a-ruby-dsl-ca... What's happened here is that it's being done in a statically typed language which is harder than in a dynamically typed on. So, good job done.
- tormeh 9y ago
  It's not new in statically typed languages either. There are already too many DSLs in Scala...

athenot 9y ago

(For those who were wondering, the article is about DSLs in Kotlin). DSLs can be very elegant ways to improve code readability (as long as the assumptions and language they use is meaningful to the team). This is an area that I wish node.js had invested in. Perl6 has "slangs" which are extremely powerful, see [1] for some examples that just flow elegantly. Combine that with custom operators to write code like "4.7kΩ ± 5%" that would make sense to electronicians [2] and you can get some really user-friendly syntaxes. (Again, it bears repeating that a DSL improves readability by sacrificing generality, so a DSL is for audience that's usually a subset of that language's users. That electronics code would be useless to the average web developer.) [1] https://github.com/tony-o/perl6-slang-sql https://github.com/tony-o/perl6-slang-sql [2] https://perl6advent.wordpress.com/tag/dsl/ https://perl6advent.wordpress.com/tag/dsl/

kqr 9y ago
> a DSL improves readability by sacrificing generality, so a DSL is for audience that's usually a subset of that language's users. You'd think that'd be self-evident from the "domain specific" in the name, but I guess a lot of things are obvious once they're pointed out to you...
wahern 9y ago
Wow. The Perl6 slang feature is awesome. I can't believe I missed it. I haven't had a chance to use Perl 6 but I would really like an excuse. I love Lua's LPeg module. Perl 6 has deeply embedded PEGs (i.e. Grammars, aka parsing combinators) into it's design. Knowing first-hand how powerful PEGs can be if the engine and language integration is implemented well, I've never doubted the awesomeness and elegance of Perl 6. It's unclear to me if Perl 6 Grammars' Action feature is as powerful as LPeg's capturing and inline transformation primitives[1], but the fact that you can plug a grammar into Perl 6's parser is crazy awesome. I'm not surprised Slang exists--I've known it was possible; just surprised that it's a module and idiomatic pattern and I hadn't read about it before. [1] See http://www.inf.puc-rio.br/~roberto/lpeg/#captures http://www.inf.puc-rio.br/~roberto/lpeg/#captures Most PEG engines just return your raw parse tree as a complete data structure and require you to fix it up manually. LPeg has really well thought-out extensions that allow you to express capture and tree transformations much more naturally; specifically, inline (both syntactically and runtime execution) with the pattern definition(s).

agentgt 9y ago

I used to be fervent embedded DSL fan many years ago (particularly in college with Scheme/Racket) but since then I have grown to hate them particularly Scala, Groovy and Ruby DSLs. External DSLs (that is a DSL independent of the host language)... I love them. An external DSL is like a protocol or the ultimate formal API (an example would be HTTP). Why do I dislike embedded DSLs: I guess the crux could be that you are essentially mixing two languages when the reality it is just one so you are doing non canonical things in the host to make the guest language look pretty (eg going to town with implicits, extension functions, and operator overloading (scala)). Basically a whole bunch of confusing stuff to normal developers just to make the DSL look pretty. With Scheme/Lisp/Clojure it is OK because embedded DSLs using sexp look like lisp. In Haskell it is sort of OK because Monads and various other things Haskell provides (laziness). With Scala its the absolute worse with operator overloading, implicits, and various other gotchas. In my experience it also gets really confusing trying to figure what is going on when a embedded DSL breakk as well as extremely difficult to secure an embedded DSL. `ie System.exit(1)` is just one call away. There are exceptions of course. DSLs that mimic external DSLs one-to-one. One example being jOOQ as it aims to be one to one with an external DSL ... SQL.

pluma 9y ago
Embedded DSLs often seem like the typical "bike shed coding styles" taken to the extreme. In my opinion they often value form over function, creating an entirely new (subset of the) language just because. I can obsess about code formatting as much as anybody else but creating my own subset of the language just for a specific domain just seems excessive. It puts additional burden on anyone else trying to work with it because they now not only have to worry about the "host" language but also the idiosyncrasies of the embedded DSL.
- swsieber 9y ago
  Counter argument for statically enforced dsl (e.g. Kotlin): I have an idea for a DSL - parse the templates for a site to generate type information about what buttons, inputs, interpolated values, etc can be found on each page. Combine that with parsed route information to allow one to generate dsl binding for a all the pages / routes. If we're doing that with Kotlin, then afterwards we can get a really nice, statically enforced testing api (for Selenium, etc), with IDE completion. And you don't even have to run them to make sure that the element selectors work (barring of course any bugs in your parsing setup). I had this idea several months ago but haven't had the opportunity to use it. I'm itching to because it seems like a feasible, if not slightly insane, way to making writing automated end to end tests easier to write and maintain. tl;dr if you're going to do some code gen for code generation, why not make the interface a DSL?

lasermike026 9y ago

I like ruby's rspec and Rake. Very powerful.

lostcolony 9y ago

Out of curiosity (because this is what I was hoping from from the article), has anyone found a DSL in the wild that actually achieved its objectives? That is, it either simplified the code necessary to write a program that there was a net savings for the project compared with developers just writing it in a language they were familiar with, or it simplified the business logic aspect to such a degree that even non-devs were able to be productive with it? I admit this is partly determined by what we decide a DSL is, since it is, ultimately, at some level just an abstraction. But almost every instance I'd squarely say "that was an attempt at a DSL", it became just as complex as the underlying language, with additional gotchas, if it even 'worked' at all (i.e., allowed you to solve the domain specific problems it was intended to solve).

agency 9y ago
This may partly be Stockholm syndrome, but I'm actually somewhat fond of Gradle (a Groovy DSL for configuring builds/managing dependencies for Java projects). It's certainly not perfect and I think it would be overkill for simple builds, but I find the DSL well suited to writing custom build tasks. As an example, I wrote a Gradle plugin for a project I work on that lets you write: apply plugin: 'mycompany.service' mainClassName = 'mycompany.Main' service { id = 'serviceid' port = 1234 } This results in a `buildImage` task being added which builds a Docker image that runs `mycompany.Main` and exposes the specified port (and does some Docker tagging with the service id). Implementing this plugin was straightforward, less than 50 lines of Groovy, most of which is invoking another DSL[1] for creating a Dockerfile. [1] https://github.com/bmuschko/gradle-docker-plugin#creating-a-dockerfile-and-building-an-image https://github.com/bmuschko/gradle-docker-plugin#creating-a-...
- vorg 9y ago
  > Gradle (a Groovy DSL Gradle is not a Groovy DSL. Gradle enables you to write its build files in either Kotlin or Apache Groovy, and perhaps there'll be more choices later on. > Implementing this plugin was straightforward, less than 50 lines of Groovy Did you know Gradleware now recommend using Kotlin for writing plugins for Gradle versions 3.0 and later?

didibus 9y ago

In CS, there are no good and bad things, there are only trade offs. DSLs are only good when the effort to learn to be proficient in the DSL repays the time it took you to learn it. Otherwise, the cognitive load of learning the semantics and the challenges often with debugging are not worth it. An example of a great DSL is linq in C#. List comprehensions in Clojure. JQuery in JavaScript. Etc.

Osmium 9y ago

> Unlike many other languages, it allows developers to create another language inside it. Is this true? e.g. Swift has extensions on generic types too, doesn't it, and could also be used to write DSLs? (Sincere question, asking because I genuinely don't know how rare this kind of functionality is)

robotpony 9y ago

DSLs can be great abstractions and interfaces, in that they can provide a way to state program knowledge in a form that non-programmers may be able to work with more easily. One example that stuck with me early was the Quake (and Doom) shader files, as well as the other game resource constructs from those old WAD-based games. The shader syntax wasn't much more than a rough wrapper on the program's `struct`s, but it allowed the graphic artists to twiddle with the resources manually (and later made a great program interface for the level editors). Quake map files were [pretty neat too](https://quakewiki.org/wiki/Quake_Map_Format https://quakewiki.org/wiki/Quake_Map_Format), compiling down to playable levels as BSPs (binary space partitioning files). XML (specific to document interchange), HTML (specific to hypertext interchange), and CSS (specific to DOM settings) were all DSLs that came about over the course of browser evolution. They offered ways to define content and configuration in a way that was both easy to understand and somewhat isolated from the underlying source code. I've developed several DSLs in various systems I've worked in. Mostly you're pushing stuff out of the source code that doesn't belong there: configuration, repeated definitions, and things that may change from installation to installation. One of the abstractions we built was a series of custom state machine scripting languages that mapped to serial and TCP/IP protocols in a way that reduced boilerplate, and made the guts of implementing specific communications scenarios easier. The amount of time spent in developing a DSL was generally many times smaller than the gain in flexibility, transparency, and exposure to who could interact with customizing the system.

ruleabidinguser 9y ago

The immutable data structures example was revolting