6 ms·

> But it's just so damned useful to be able to reach into a library and get functionality the library's writer didn't think I'd need, that I'll personally never

by pyrale 9y ago

> But it's just so damned useful to be able to reach into a library and get functionality the library's writer didn't think I'd need, that I'll personally never consider this to be bad. I was ready to accept this as a matter of opinion, but then with this : > Metaprogramming is the source of exponentially more technical debt (and bugs) than is reasonable, frankly. Once you've jumped the shark and play with introspection and internals, are you really going to make the point that metaprogramming (or almost anything else, for that matter) should be banned ?

falcolas 9y ago
> are you really going to make the point that metaprogramming should be banned ? As a fan of the concept of the "Catfish" developer (even though I'm a "corporate drone"), absolutely. Or, if not banned, kept to an absolute minimum. It's like the preprocessor in C. You can do anything and everything in it... but should you? You can be exceedingly clever with metaprogramming, but clever code is exceedingly hard to read and maintain. Give me boring, explicit code any day of the week.
- flavio81 9y ago
  Metaprogramming goes way beyond what the C preprocessor does. Metaprogramming also cannot necessarily be "hard to read and mantain". Take a look at macros in Common Lisp. They are almost the same as a normal function. Yet they do metaprogramming. > Give me boring, explicit code any day of the week. With macro metaprogramming you can eliminate boilerplate code / unnecessary copy-paste repetition of code, and this does improve maintainability of the code. At the same time, replacing 10 repetitions of the same big boilerplate code (similar code, written with slight variations each of the 10 times) with 10 very simple (one line each) calls to a macro, improves readability: because you will easily read what is the difference between those 10 calls each.
  - joshuamorton 9y ago
    Macro metaprogramming can (and in practice often does) break ctrl+f search. If I want to find the definition of PREFIX_varname, I'd expect to grep for "PREFIX_varname", but often, the correct solution is to search solely for varname, because PREFIX_varname is defined by a call to MACRO(varname).
    - KirinDave 9y ago
      Maybe don't use such a primitive editor integration? I'm not trying to dunk on your editor, but people have had this problem solved since the early 90s. If your programmer's editor doesn't have some affordances for your language, then it's not a programmer's editor at all, it's a text editor.
      - joshuamorton 9y ago
        I'm curious: what editor can do this? Consider that the codebase I work on is large enough that it cannot fit in a project file or ctags or similar. (Which is to say that VS, Clion, and others don't work, nor will vim (for this specific feature))
        KirinDave 9y ago
        For which language? In the case of systems like Clojure, if you're introducing new toplevel bindings you're not going to get completion without direct interactive evaluation (naturally). But everything short of that should be well-formed. In many cases with common macros I'd add decls for Cursive Clojure + Idea to let me autocomplete introduced bindings. For example, I helped my editor understand unique structure, e.g., (defauthedendpoint get-frobnaz [request user] ...re<tab>quest)
        joshuamorton 9y ago
        C/C++. In the general case, I don't believe its possible to resolve macros without linking.
  - falcolas 9y ago
    Lisp macros go far beyond what is possible in Python/Ruby, and oddly enough are less of a problem because of that. In Python and Ruby, the inability to change the underlying syntax makes alot of developers go to absurd lengths to create new syntax around the existing syntax. And that's what makes Python/Ruby metaprogramming really painful to understand. That said, my (admittedly few) days spent debugging bad macros were pretty damned bad. Having to inspect the results of the macro expansion and figure out what went wrong is not my idea of a good time. And that doesn't even account for reader macros.
    - kazinator 9y ago
      If a macro is correctly expanding, but you have to go into the expansion to find out what is going wrong when the macro is misused, that means the macro lacks robustness; it is not checking arguments, or not sufficiently or in the right way for the cases you're running into. Bad arguments make their way into the expansion and then trigger errors which are conveyed in terms of the expansion, not necessarily making sense to the macro user. It takes extra time and effort to make macros that are suitable for broader use (people other than the macro writer).
      - falcolas 9y ago
        Agreed on all points. But ultimately, bugs in software happen, and bugs in macros can be a pain in the butt to debug. The extra steps to go in and expand the macro, and then translate the fix in that expansion into a fix for the macro itself gets complicated, fast.
        kazinator 9y ago
        The job of the macro is to convert a form into an expansion. If it's doing the job wrong, I don't see how you can avoid looking at the output. With respect to what is that "extra". It's no different from any other "input -> process -> output" computing situation where the output is wrong, the right output is obvious, and you have work back into the process to make that output come out. You shouldn't be fixing macro output. If the macro is wrong, the first step is to find the simplest macro invocation which reproduces the problem. Use a dummy argument for anything irrelevant, and use an atom for any argument that allows one. For any argument position which accepts a list of items, see if the problem reproduces with an empty list, or a list of just one. If you can see the problem in (mymac (a b) c d), you have it made. Hey look, b is being wrapped in an extra list, and d should be quoted."
- tome 9y ago
  > You can be exceedingly clever with metaprogramming, but clever code is exceedingly hard to read and maintain. Give me boring, explicit code any day of the week. Boring, explicit code which depends directly on the unspecified, unsupported, implementation details of the libraries you're using ... ?
  - throwaway91111 9y ago
    I mean that's the point of tooling. Metaprogramming removes any advantage and IDE can give you to make sense of unknown code.
    - KirinDave 9y ago
      This is strictly false though. IDE integration can give insight into metaprogramming in many languages.
      - falcolas 9y ago
        Python, the language in question, does not easily offer this support, since all of the metaprogramming occurs at runtime. You have to run a debugger while creating the object to see what is actually happening. Lisps, C Preprocessor (within certain limits), and so forth are a bit different, since you can expand the macros without having to run the code itself. Does that indicate a strength of Python or a weakness? A complete value judgement there. In either case, metaprogramming in Python (and many other languages, even those with support for metaprogramming resolution in their IDEs) is still simply harder to understand and debug.
        KirinDave 9y ago
        I feel no real problem shaming the entire design of Python as a long bet on failed premises. But other languages with similar properties can and do report their metaprogramming results to language servers.
        9y ago
HelloNurse 9y ago
"Reaching into a library" means depending on something not meant to be public, with the understanding that it is undocumented and subject to change without notice. Giving a reliable but soft warning (through quasi-encapsulation naming conventions and explicit module exports) and assuming developer maturity is the typical Python approach to double-edged features. Hacking something with private library parts doesn't involve any more "introspection" and "internals" than reading the library's code and finding useful loot to steal; metaprogramming is a completely different and orthogonal way to compromise code maintainability.
kafkaesq 9y ago
Once you've jumped the shark and play with introspection and internals, are you really going to make the point that metaprogramming should be banned ? "Banned", no. Just that's not simply this big trove of goodies that it's often cracked up to be. And in particular, it isn't strictly speaking necessary or essential for a language to be successful -- or even among the top 5 considerations that make a language successful or not.

by pyrale 9y ago

falcolas 9y ago

> are you really going to make the point that metaprogramming should be banned ? As a fan of the concept of the "Catfish" developer (even though I'm a "corporate drone"), absolutely. Or, if not banned, kept to an absolute minimum. It's like the preprocessor in C. You can do anything and everything in it... but should you? You can be exceedingly clever with metaprogramming, but clever code is exceedingly hard to read and maintain. Give me boring, explicit code any day of the week.

flavio81 9y ago
Metaprogramming goes way beyond what the C preprocessor does. Metaprogramming also cannot necessarily be "hard to read and mantain". Take a look at macros in Common Lisp. They are almost the same as a normal function. Yet they do metaprogramming. > Give me boring, explicit code any day of the week. With macro metaprogramming you can eliminate boilerplate code / unnecessary copy-paste repetition of code, and this does improve maintainability of the code. At the same time, replacing 10 repetitions of the same big boilerplate code (similar code, written with slight variations each of the 10 times) with 10 very simple (one line each) calls to a macro, improves readability: because you will easily read what is the difference between those 10 calls each.
- joshuamorton 9y ago
  Macro metaprogramming can (and in practice often does) break ctrl+f search. If I want to find the definition of PREFIX_varname, I'd expect to grep for "PREFIX_varname", but often, the correct solution is to search solely for varname, because PREFIX_varname is defined by a call to MACRO(varname).
  - KirinDave 9y ago
    Maybe don't use such a primitive editor integration? I'm not trying to dunk on your editor, but people have had this problem solved since the early 90s. If your programmer's editor doesn't have some affordances for your language, then it's not a programmer's editor at all, it's a text editor.
    - joshuamorton 9y ago
      I'm curious: what editor can do this? Consider that the codebase I work on is large enough that it cannot fit in a project file or ctags or similar. (Which is to say that VS, Clion, and others don't work, nor will vim (for this specific feature))
      - KirinDave 9y ago
        For which language? In the case of systems like Clojure, if you're introducing new toplevel bindings you're not going to get completion without direct interactive evaluation (naturally). But everything short of that should be well-formed. In many cases with common macros I'd add decls for Cursive Clojure + Idea to let me autocomplete introduced bindings. For example, I helped my editor understand unique structure, e.g., (defauthedendpoint get-frobnaz [request user] ...re<tab>quest)
        joshuamorton 9y ago
        C/C++. In the general case, I don't believe its possible to resolve macros without linking.
- falcolas 9y ago
  Lisp macros go far beyond what is possible in Python/Ruby, and oddly enough are less of a problem because of that. In Python and Ruby, the inability to change the underlying syntax makes alot of developers go to absurd lengths to create new syntax around the existing syntax. And that's what makes Python/Ruby metaprogramming really painful to understand. That said, my (admittedly few) days spent debugging bad macros were pretty damned bad. Having to inspect the results of the macro expansion and figure out what went wrong is not my idea of a good time. And that doesn't even account for reader macros.
  - kazinator 9y ago
    If a macro is correctly expanding, but you have to go into the expansion to find out what is going wrong when the macro is misused, that means the macro lacks robustness; it is not checking arguments, or not sufficiently or in the right way for the cases you're running into. Bad arguments make their way into the expansion and then trigger errors which are conveyed in terms of the expansion, not necessarily making sense to the macro user. It takes extra time and effort to make macros that are suitable for broader use (people other than the macro writer).
    - falcolas 9y ago
      Agreed on all points. But ultimately, bugs in software happen, and bugs in macros can be a pain in the butt to debug. The extra steps to go in and expand the macro, and then translate the fix in that expansion into a fix for the macro itself gets complicated, fast.
      - kazinator 9y ago
        The job of the macro is to convert a form into an expansion. If it's doing the job wrong, I don't see how you can avoid looking at the output. With respect to what is that "extra". It's no different from any other "input -> process -> output" computing situation where the output is wrong, the right output is obvious, and you have work back into the process to make that output come out. You shouldn't be fixing macro output. If the macro is wrong, the first step is to find the simplest macro invocation which reproduces the problem. Use a dummy argument for anything irrelevant, and use an atom for any argument that allows one. For any argument position which accepts a list of items, see if the problem reproduces with an empty list, or a list of just one. If you can see the problem in (mymac (a b) c d), you have it made. Hey look, b is being wrapped in an extra list, and d should be quoted."
tome 9y ago
> You can be exceedingly clever with metaprogramming, but clever code is exceedingly hard to read and maintain. Give me boring, explicit code any day of the week. Boring, explicit code which depends directly on the unspecified, unsupported, implementation details of the libraries you're using ... ?
- throwaway91111 9y ago
  I mean that's the point of tooling. Metaprogramming removes any advantage and IDE can give you to make sense of unknown code.
  - KirinDave 9y ago
    This is strictly false though. IDE integration can give insight into metaprogramming in many languages.
    - falcolas 9y ago
      Python, the language in question, does not easily offer this support, since all of the metaprogramming occurs at runtime. You have to run a debugger while creating the object to see what is actually happening. Lisps, C Preprocessor (within certain limits), and so forth are a bit different, since you can expand the macros without having to run the code itself. Does that indicate a strength of Python or a weakness? A complete value judgement there. In either case, metaprogramming in Python (and many other languages, even those with support for metaprogramming resolution in their IDEs) is still simply harder to understand and debug.
      - KirinDave 9y ago
        I feel no real problem shaming the entire design of Python as a long bet on failed premises. But other languages with similar properties can and do report their metaprogramming results to language servers.
        9y ago

HelloNurse 9y ago

"Reaching into a library" means depending on something not meant to be public, with the understanding that it is undocumented and subject to change without notice. Giving a reliable but soft warning (through quasi-encapsulation naming conventions and explicit module exports) and assuming developer maturity is the typical Python approach to double-edged features. Hacking something with private library parts doesn't involve any more "introspection" and "internals" than reading the library's code and finding useful loot to steal; metaprogramming is a completely different and orthogonal way to compromise code maintainability.

kafkaesq 9y ago

Once you've jumped the shark and play with introspection and internals, are you really going to make the point that metaprogramming should be banned ? "Banned", no. Just that's not simply this big trove of goodies that it's often cracked up to be. And in particular, it isn't strictly speaking necessary or essential for a language to be successful -- or even among the top 5 considerations that make a language successful or not.