15 ms·

And? DSLs can have any syntax you like. And any type system you want. And DSLs done the right way, via macros, are much better in integrating with tools than a

• lmm 11y ago
  > And DSLs done the right way, via macros, are much better in integrating with tools than any ad hoc interpreted DSLs would ever be able to. You can easily have syntax and semantic highlighting infered, with auto indentation, intellisense and all the bells and whistles. For no extra cost. No you can't. If the macro is arbitrary code then no tool can offer those things - there's no way to offer intellisense if you don't know what strings are meaningful in the language, and an unconstrained macro could use anything to mean anything.
  • sklogic 11y ago
    Yes you can - as soon as you start wrapping your arbitrarily complex macros into a custom syntax. I am easily doing this stuff with any language I am adding macros and extensible syntax to.
    • lomnakkus 11y ago
      > Yes you can - as soon as you start wrapping your arbitrarily complex macros into a custom syntax. Well, by that definition you get exactly the same if the host language of your DSL is statically typed and doesn't use macros. Custom syntax is custom syntax and whether tools/IDEs understand it has nothing to do with the host language.
      • sklogic 11y ago
        Sorry, I did not quite get what you mean. Of course macro+syntax extension got absolutely nothing to do with what you can achieve in a language without macros. And, no, you did not understand. Any custom syntax you're adding (if the right tools are used, like mine, for example) would automatically become available for your IDE and all the other tools, because they're reusing the same compiler front-end.
        • lmm 11y ago
          > And, no, you did not understand. Any custom syntax you're adding (if the right tools are used, like mine, for example) would automatically become available for your IDE and all the other tools, because they're reusing the same compiler front-end. Just being able to execute the macro isn't enough for the IDE though. E.g. if a macro is "rot13 all identifiers in this block" then sure the IDE can run it, but it can't offer sensible autocompletion inside the block without understanding more about the structure of the macro.
          • sklogic 11y ago
            IDE does not execute the macro - it knows the result of its expansion from the compiler. And compiler keeps a track of all the identifiers and their origins.
            • lmm 11y ago
              The IDE can autocomplete the rot13ed identifiers from outside, perhaps. But it can't possibly suggest rot13ed identifiers inside the macro block for autocomplete, because it can't possibly know that that's what the macro does.
              • sklogic 11y ago
                Why? You know which macro made the identifiers. You know what this macro consumed. In most practically important cases this is sufficient. But, yes, you cannot do it with the Common Lisp approach, where macros operate on bare lists, not the scheme-like syntax objects. The problem here is that the lists had been stripped from the important location metadata. For this reason I had to depart from the simple list-based macros and using custom syntax extension with rich ASTs underneath. Still, on top of a Lisp.
                • lmm 11y ago
                  Even with location information, if the IDE's going to offer autocomplete inside the macro it would need to be able to invert the way the macro transforms identifiers, which is not possible to do to arbitrary code. I agree that this is very rarely practically important - but if you think about it that's precisely the fact that a more restricted alternative to macros should be adequate.
    • lmm 11y ago
      But if the syntax is arbitrarily customizable, you can't possibly have the tools understand how to highlight it / intellisense / autoindent / etc.
      • sklogic 11y ago
        Of course I can. If my tools are communicating with my compiler, they know everything it knows. I have a single tiny generic Emacs mode (and a similar Visual Studio extension) that handles all the languages designed on top of my extensibility framework. It's trivial. Any PEG parser I add on top automatically communicates all the highlighting, indentation data and all that to the tools (and it's inferred from the declarative spec, no additional user input is required). Underlying typing engines do the same, for the nice tooltips and code completion. The very compiler core doest the same with all the symbol definitions, dependencies, etc. Easy.
        • lmm 11y ago
          Exactly! You end up defining your macros in a particular restricted subset of lisp, and your tooling for Emacs and Visual Studio has to know about that particular subset. Other people writing similar macros will no doubt have their own, subtly different subset, and their own integrations for their subset. But since your way of writing declarative specs for language customization isn't standardized, you can't use each other's tool integrations. The way you express DSLs is something that needs to be understood by language tooling, so it belongs in the language spec.
          • sklogic 11y ago
            No. Tools do not know anything about the restrictions. In fact they work with a wide range of languages, not just lisp. The only "restriction" is a protocol, built into the macro expander, syntax frontend and compiler core. So, in your rot13 example compiler would rat all the new identifiers with their origins to the tools.
            • lmm 11y ago
  • kazinator 11y ago
    The tools could have hooks for this. It doesn't take much imagination. You know how GNU Bash is customizeable with custom completion for any command, so that when you're, say, in the middle of a git command, it will complete on a branch name or whatever? Similarly, we can teach a syntax highlighter, completer or whatever in some IDE how to work with our custom macro.
    • lmm 11y ago
      Sure - but at that point we've lost a lot of the value of having a standardized language at all. The whole point of a language standard is that multiple independent tools can be written to work with it - that your profiler and your linter and your compiler can be written independently, because they'll be written to the spec. If everyone has to customize all their tools to work with their own code, that's a lot of duplicated effort. Better to have a common standard for how you embed DSLs in the language, so that all the tools already understand how to work with them.
      • sklogic 11y ago
        It is a broken approach. A much better way is to have a standard protocol (see slime for example, or IPython, or whatever else), and use the same tools as your compiler does, instead of reimplementing all the crap over and over again from the language standard. I expect that not that many C++ tools that do not use libclang will remain.
        • lmm 11y ago
          At that point you're essentially advocating treating libclang as the standard. All the usual problems of "the implementation is the spec" apply.
          • sklogic 11y ago
            libclang is just an example, maybe not an ideal one. But, yes, I'm advocating for an executable language spec, one that you'd use as a (probably suboptimal, but canonical) implementation. A good example of such a thing would be something like https://github.com/kframework/c-semantics https://github.com/kframework/c-semantics
• lomnakkus 11y ago
  > And? DSLs can have any syntax you like. And any type system you want. While you're technically correct, as a practical matter you won't implement a type system/type checker for all the DSLs you create. (Nor do I believe you'll even do it for anything approaching a majority.). Obviously, I'm using the impersonal "you" here. Implementing real type systems is hard and mostly rather tedious work. syntax-parse only gets you so far[1]. [1] https://docs.racket-lang.org/syntax/Parsing_Syntax.html https://docs.racket-lang.org/syntax/Parsing_Syntax.html
  • sklogic 11y ago
    I have a nice DSL which makes building complex type systems (including dependent) trivial and fun. So I do not mind adding typing to pretty much any kind of DSLs, including the smallest of them.
    • nickpsecurity 11y ago
      I still think you should have some links bookmarked to drop in conversations like this. Bookmarked so you can just type a name rather than waste time looking. Many in these discussions might think you're speculating or have some toy project rather than the interesting one I found digging through your old posts that backs up your claims. Maybe even have a series of examples that illustrate solutions you keep mentioning so others can learn and apply them. Just a thought. :)
      • sklogic 11y ago
        I am already slow-banned here (or whatever it is called). If I start spamming the links, they will suspend this account.
        • deleted 11y ago
          [deleted]
    • lomnakkus 11y ago
      Just trying to understand here: Are you saying that you have a meta-framework[1] for developing DSLs? If so, then I suppose I misunderstood you original claim, but I won't apologize because your claim was very opaque to anyone who doesn't know you. [1] An example of what I mean would be xtext.