4 ms·

My experience is that you can have powerful runtime metaprogramming, or you can have good static type checking, but you can't have both. In most of the larger

My experience is that you can have powerful runtime metaprogramming, or you can have good static type checking, but you can't have both. In most of the larger programs I've worked in the value of static typing is immense. In terms of my productivity, the only programming language feature that's possibly had a bigger positive impact is garbage collection. Runtime metaprogramming is very cool, but I've yet to work in a codebase where I felt that the value of it was greater than the value I get from good static analysis. One way to think of static types is that they're a debugger that simultaneously debugs every possible run of the program. And on top of that, you also get code navigation and better performance. It's hard for me to imagine a sufficiently great metaprogramming feature that would be worth giving that up.

grncdr 4y ago
One approach that I feel is underexplored would be building a language server that is extensible from within the analyzed program. Somewhere adjacent to procedural macros and compiler plugins, there's probably a sweet spot for dynamic languages where your program can simply (and simply would be key to pulling it off) tell the language server what some bit of metaprogramming actually _does_ at runtime. I've played with this idea a bit in Solargraph (especially in regards to Rails) and while the results are promising, the barrier to entry and amount of labour involved is pretty high.
bobbylarrybobby 4y ago
I feel that oftentimes runtime metaprogramming is to make up for code that can't be written down statically. But if you have the right macros then you don't need the metaprogramming to happen at runtime. For instance, Rust has a pretty solid type system (not quite Haskell, but more intricate than Java) and lets you do things like generate serialization and deserialization code for arbitrary structs at compile time, which most languages would do at runtime by inspecting fields or something similar. All this to say, runtime metaprogramming doesn't seem inherently more capable compared to what's possible in the more “static compiler” languages; rather, it seems like an escape hatch for when you can't generate the code you want at compile time.
- kaba0 4y ago
  What I find absolutely intriguing is the existence of multiple compile/runtimes, like a Matryoshka doll. LISPs are most famous for attempting to solve this problem with their macro systems. The JVM ecosystem also has a very oft used, but more advanced case for this: runtime code generation. Depending on how dynamic your problem is this might be required and then “traditional compile-time” macros would not be enough.
- LAC-Tech 4y ago
  All this to say, runtime metaprogramming doesn't seem inherently more capable compared to what's possible in the more “static compiler” languages; rather, it seems like an escape hatch for when you can't generate the code you want at compile time. It's by definition more powerful though, in that the runtime language is (hopefully) more powerful than the type description language. I'm not arguing that it isn't preferable to produce (de)serialisation code statically. Just that by definition it's more capable as it can do everything the runtime can.
- Fire-Dragon-DoL 4y ago
  What this person said. 15 years of ruby and all that metaprogramming is literally the most painful part. Funny, because I used to love it.
- munificent 4y ago
  Yes, compile-time metaprogramming is a different approach that I'm personally really interested in. There are trade-offs as always, but it can give you much of the power of metaprogramming without totally killing static analysis.
- bradrn 4y ago
  > For instance, Rust has a pretty solid type system (not quite Haskell, but more intricate than Java) and lets you do things like generate serialization and deserialization code for arbitrary structs at compile time, which most languages would do at runtime by inspecting fields or something similar. For that matter, Haskell can do exactly the same thing, and as you note its type system is even more sophisticated than Rust’s.
syspec 4y ago
Java honestly provides both, and it does it extremely well! I personally prefer to use Kotlin, but same idea.
- paulddraper 4y ago
  > Java honestly provides both I think there is a serious disconnect about what is "metaprogramming."
  - kccqzy 4y ago
    Agreed. I was going to say Haskell does both very well, but that's impossible to agree or disagree without first agreeing on a definition for metaprogramming.
- munificent 4y ago
  Java can do some fairly powerful stuff with reflection and classloaders, but isn't anywhere near the level of runtime flexibility you can get with Smalltalk or Ruby where the entire running program is a mutable data structure you can programmatically introspect over and modify.
  - kaba0 4y ago
    Well, groovy runs on top of the JVM and can change every method even on an object-level, or even implement how non-defined functions are to be run.
valcron1000 4y ago
I would only give up static typing to have at least the amount of power that I get using Common Lisp
EdwardDiego 4y ago
Working with a large Django codebase, I agree. Django leans on metaprogramming hard, and it's hard to express the type "It's a Model, but doesn't have an inner class Meta right now, but it will at runtime".
MrJohz 4y ago
I quite like Typescript's approach here, which is (a) to model that metaprogramming as much as possible with metaprogramming of types, and (b) to always give you an escape hatch so you can wrap type unsafe metaprogramming in a type safe layer. For example, I recently wrote a function that takes an arbitration nested object and a list of paths through that object (e.g. "path.to.object"), and does things with the data found at those paths. For (1), I could write a type that, given an object type, would return all the possible allowed paths as types, basically preventing the function from being used correctly (and it already has helped me catch a few typos here and there). So now func({foo:...}, "bar") is disallowed directly at the type system. But working with those exact types inside the function would have been too complicated, so I also did (2): inside the function, I just treated as the strings as normal strings (which, to the runtime, they are!) and did normal JavaScript metaprogramming to get arbitrary keys from arbitrary objects. I then have less type safety inside the function, but I can test that part more, while still leaving the caller of the function with the exact correct types. In fairness, there are other ways to solve the same problem in other languages, with similar type safety. And JavaScript natively is not that high up on the metaprogramming scale. But I'm always impressed in Typescript how much a powerful type system and metaprogramming can dovetail into each other.

grncdr 4y ago

One approach that I feel is underexplored would be building a language server that is extensible from within the analyzed program. Somewhere adjacent to procedural macros and compiler plugins, there's probably a sweet spot for dynamic languages where your program can simply (and simply would be key to pulling it off) tell the language server what some bit of metaprogramming actually _does_ at runtime. I've played with this idea a bit in Solargraph (especially in regards to Rails) and while the results are promising, the barrier to entry and amount of labour involved is pretty high.

bobbylarrybobby 4y ago

I feel that oftentimes runtime metaprogramming is to make up for code that can't be written down statically. But if you have the right macros then you don't need the metaprogramming to happen at runtime. For instance, Rust has a pretty solid type system (not quite Haskell, but more intricate than Java) and lets you do things like generate serialization and deserialization code for arbitrary structs at compile time, which most languages would do at runtime by inspecting fields or something similar. All this to say, runtime metaprogramming doesn't seem inherently more capable compared to what's possible in the more “static compiler” languages; rather, it seems like an escape hatch for when you can't generate the code you want at compile time.

kaba0 4y ago
What I find absolutely intriguing is the existence of multiple compile/runtimes, like a Matryoshka doll. LISPs are most famous for attempting to solve this problem with their macro systems. The JVM ecosystem also has a very oft used, but more advanced case for this: runtime code generation. Depending on how dynamic your problem is this might be required and then “traditional compile-time” macros would not be enough.
LAC-Tech 4y ago
All this to say, runtime metaprogramming doesn't seem inherently more capable compared to what's possible in the more “static compiler” languages; rather, it seems like an escape hatch for when you can't generate the code you want at compile time. It's by definition more powerful though, in that the runtime language is (hopefully) more powerful than the type description language. I'm not arguing that it isn't preferable to produce (de)serialisation code statically. Just that by definition it's more capable as it can do everything the runtime can.
Fire-Dragon-DoL 4y ago
What this person said. 15 years of ruby and all that metaprogramming is literally the most painful part. Funny, because I used to love it.
munificent 4y ago
Yes, compile-time metaprogramming is a different approach that I'm personally really interested in. There are trade-offs as always, but it can give you much of the power of metaprogramming without totally killing static analysis.
bradrn 4y ago
> For instance, Rust has a pretty solid type system (not quite Haskell, but more intricate than Java) and lets you do things like generate serialization and deserialization code for arbitrary structs at compile time, which most languages would do at runtime by inspecting fields or something similar. For that matter, Haskell can do exactly the same thing, and as you note its type system is even more sophisticated than Rust’s.

syspec 4y ago

Java honestly provides both, and it does it extremely well! I personally prefer to use Kotlin, but same idea.

paulddraper 4y ago
> Java honestly provides both I think there is a serious disconnect about what is "metaprogramming."
- kccqzy 4y ago
  Agreed. I was going to say Haskell does both very well, but that's impossible to agree or disagree without first agreeing on a definition for metaprogramming.
munificent 4y ago
Java can do some fairly powerful stuff with reflection and classloaders, but isn't anywhere near the level of runtime flexibility you can get with Smalltalk or Ruby where the entire running program is a mutable data structure you can programmatically introspect over and modify.
- kaba0 4y ago
  Well, groovy runs on top of the JVM and can change every method even on an object-level, or even implement how non-defined functions are to be run.

valcron1000 4y ago

I would only give up static typing to have at least the amount of power that I get using Common Lisp

EdwardDiego 4y ago

Working with a large Django codebase, I agree. Django leans on metaprogramming hard, and it's hard to express the type "It's a Model, but doesn't have an inner class Meta right now, but it will at runtime".

MrJohz 4y ago

I quite like Typescript's approach here, which is (a) to model that metaprogramming as much as possible with metaprogramming of types, and (b) to always give you an escape hatch so you can wrap type unsafe metaprogramming in a type safe layer. For example, I recently wrote a function that takes an arbitration nested object and a list of paths through that object (e.g. "path.to.object"), and does things with the data found at those paths. For (1), I could write a type that, given an object type, would return all the possible allowed paths as types, basically preventing the function from being used correctly (and it already has helped me catch a few typos here and there). So now func({foo:...}, "bar") is disallowed directly at the type system. But working with those exact types inside the function would have been too complicated, so I also did (2): inside the function, I just treated as the strings as normal strings (which, to the runtime, they are!) and did normal JavaScript metaprogramming to get arbitrary keys from arbitrary objects. I then have less type safety inside the function, but I can test that part more, while still leaving the caller of the function with the exact correct types. In fairness, there are other ways to solve the same problem in other languages, with similar type safety. And JavaScript natively is not that high up on the metaprogramming scale. But I'm always impressed in Typescript how much a powerful type system and metaprogramming can dovetail into each other.