3 ms·
Fragility of the more clever dynamic typing stuff like the "Replacer" is a big problem. An even bigger problem is the increasing difficulty of composing those t
by jerf 4y ago
Fragility of the more clever dynamic typing stuff like the "Replacer" is a big problem. An even bigger problem is the increasing difficulty of composing those things together when you want more than one at a time.
And that difficulty scales up fast. Consider the Python function decorator. It was quickly discovered that the decorators need to preserve the metadata for the function's name, etc., or the decorated functions broke in places that looked at that metadata. But if you have a "function name" parameter that the decorators can modify, now there's a mismatch between the fact that the function claims to be "the original GetUser function" but it's actually the decorated function. So now you ought to have "the visible function name" and "the real function name", so for instance, the former works with metaprogramming correctly but the latter allows for correct debugging output on crashes.
But what if you decorate a decorator and then want to do metaprogramming on it? Now you're got the apparent function name, the real name of the function beneath it, and the real name of the function beneath that.
And this is just one dimension of problem. There are many others when you get to metaprogramming like this.
The core problem is really this style of metaprogramming, in my opinion. It almost inevitably slips into becoming Deception-Based Programming, in which increasing swathes of the program are trying to figure out what lies to tell to other chunks of the program to get the desired effect. Dynamic typing makes this harder because it makes it very difficult to get a manifest that lists all the behaviors this particular chunk of code or code object has, but even if you were doing this in a static language that didn't make that hard, a language would still have a problem exposing all the correct properties and assisting programmers in getting them completely correct.
It's fun at first, and very powerful at first. But deception-based programming is one of the quintessential ways a code base fails to scale and becomes an incomprehensible tangled mass, code bases that use heuristics to figure out what lies to tell to other bits of the programming to cover up the effects of the lies the other bits of the code are telling. This leads to your programming becoming an embodiment of the phrase "What a tangled web we weave, when first we practice to deceive".
That said, before someone pops up with "but we could do that without deception"... I agree! Or at least, I agree that it should be explored better. I would be interested in a language that somehow enables such metaprogramming, but with a philosophy that says that it will stay "honest" throughout, and basically deliberately eschew and exclude this sort of "transparency". I don't think it's necessarily intrinsic to the style. I just think that it's sooooo tempting that if you don't build a language that excludes it from the get go that the temptation to just shim this little behavior in just here is too tempting, plus the languages tend to end up requiring it, be it accidentally or deliberately, because they afford that behavior and the language ends up being built around how it is at least supposed to be possible. I don't know entirely what this would look like.
Edit: I say I don't know what this would look like, but I have had a couple of ideas. One I've talked about before is to have a clean separation between an "initialization" phase in which metaprogramming is legal, and a "sealing" phase in which it is all done and the metaprogramming becomes invalid/illegal. I wrote this for the runtime not to have to be super flexible all the time, but it works for programmers using the system to know that things won't change out from under them.
The other is some convenient and powerful way to dump things out in their final state post-manipulation. I've gotten some good mileage out of this in my own code that is using a lot of composition. It is very helpful to be able to take the top level of the composed object and say "Tell me what exactly you are made out of". Even if I'm not doing metaprogramming based on the response, it's a huge debugging help. I see traces and fragments of this idea here and there, but no system that has it coherently integrated in a way I'd like to see.
- mjr00 4y agoAgreed with all of this. It's the programming version of being penny wise and pound foolish. This style of metaprogramming reduces a few lines of boilerplate code at the expensive of making a system as a whole extremely difficult to understand or maintain.
- quickthrower2 4y agoYes. I suspect this taming of complexity and severe uniqueness of each codebase with it’s clever “patches” applied through meta programming is why we have the meme of the lone Lisp programmer writing their dependencies from scratch. Having restrictions like types and no meta programming is less “fun” but the new person on the team will get up to speed more quickly. The one job where in C# actually they used a tonne of meta programming I could barely get anything done. Because of course the documentation is non existent and the codebase so big would take you years to comprehend, or trying to get some help from never-free team members. tldr KISS!
- skybrian 4y agoIt seems like to make this work, we need clarity around what's the public interface and what's the implementation. Some implementation details need to be visible to debuggers and for analysis, but hidden in "regular" code so that you can substitute a different implementation without invalidating constraints. For example, functions normally have names for debugging purposes, but the names can't be relied on in "regular" code or you wouldn't be able to rename them without breaking things. It may be better if the names aren't visible at runtime at all. The reason Python function decorators have these issues is due to a lack of structure; a lot of metadata is just there for anybody to use, with no constraints. It's also possible to do this by convention - for example, many things in Go's reflection package will break abstraction boundaries if you use them the wrong way. It's less of an issue if you have strong language conventions against doing that. In many scripting languages, the conventions for when you can use metadata are weaker.
- rileyphone 4y agoThe "Mirrors" system by Bracha and Ungar [0] is intended to address this IIUC. It's obviously a big, very messy problem, but without a good meta system, languages are forced to grow large and messy on their own in order to be appropriate for a wide variety of programming tasks, or to overuse compile-time macros/templating to similarly hairy effect. 0. https://bracha.org/mirrors.pdf https://bracha.org/mirrors.pdf