5 ms·

>Python on the other hand has no macros I challenge this. I won't pretend that Python has full macro equivalence, but decorators are damned close.

kenjackson 16y ago
Can you expand on this. What is something that macros can do that decorators can't? And likewise, what are some common uses of macros that decorators can do?
- jerf 16y ago
  One thing that answers both your questions at once are macros that create new control structures. Macros can take a chunk of syntax tree but not evaluate it, and decorators can only manipulate functions, they can't be used in a function. That said, the usual answers to the question of "Why do I need macros" are slowly but surely being chewed through by Python. The relatively-recent (albeit years old) addition of "with" took another previously-macro-only use case away. I'm not sure what massive win for macros is left. If your Python is "downright verbose" you're probably doing something wrong. It may not always be the Absolute Shortest (TM) but "downright verbose" shouldn't come up often.
  - jfm3 16y ago
    >I'm not sure what massive win for macros is left. From my point of view, not being able to define new syntax is as serious a limitation as not being able to define new functions, or data structures. There are a number of specific examples usually cited to explain why the ability to define new syntax is important. Answering these specific examples by adding more static syntax to the language seems to be severely missing the point, to me. Imagine it was functions instead. "Well your language already has a sort() function, but say you want to use an in-place sort instead of what you get with the standard library? You need the ability to define your own functions." "No we don't, our BDFL added an in_place_sort() routine to the standard library for the next version of the language." That doesn't make sense to me at all.
    - jerf 16y ago
      "From my point of view, not being able to define new syntax is as serious a limitation as not being able to define new functions, or data structures." Which is exactly my point. We've gone from "Well, I can use it to define a ternary operator" or "I can use it to define a 'with' block" or "I can create continuations" or a number of other specific examples to "Uh, well, I just like them, here I do this with them" and in my experience there's always an easy "Well, the Pythonic way to do that is this and it's not all that much harder, may even be easier, and is a standard part of a relatively simple language rather than your own hand-rolled thing". It's a very different argument than it was ten years ago when I first got into Python. It's gone from really strong pro-macro points to personal opinion. I respect and acknowledge that opinion, no sarcasm. But it's not the same. Your last paragraph isn't relevant to me because the specificity of your example is misleading. It's really hard to come up with an actual macro use case that is not covered by modern Python and is still actually a good idea. (It's trivial to come up with bad uses of macros not covered by Python, but I'm hardly crying about that.)
      - jfm3 16y ago
        > It's really hard to come up with an actual macro use case that is not covered by modern Python and is still actually a good idea. sqlalchemy (and now mongoalchemy) is begging for macros. See CLSQL for an answer. Regular expressions are infinitely more readable as expressions in a dynamic syntax. See the rx package in Emacs for starters. What if you want shell/Ruby style back quotes, which in essence create a closure that runs a batch job? See my SHELLSHOCK library (jfm3.org) for the Common Lisp extension. It's not much code at all. What if you want large literal strings which respect the indentation of their surrounding code, without uglying things up with <<<EOF ? See my BOXEN library (also jfm3.org) for that Common Lisp extension. What if you want something like Python's r"string", but you want more control over which things are escaped and which are not? See CL-INTERPOL for that. What if you want anaphora in your control structures? What if you want Lex/Yacc style parser generator specifications where the type of productions is extensible, and the productions themselves can be generated dynamically at compile time? (This will make no sense to you unless you've ever had no choice but to run m4 on your .l and .y files.) I could go on. It's not hard for me.
- softbuilder 16y ago
  As jfm3 points out, you're not going to define new language syntax with decorators. You're also not going to apply them on the spot, inline with other code, since they are defined on a function, and aren't used in the same way as macros are in Lisp. What you can do though is define behavior that varies based on the context. For example in Python you don't have function overloading or types, but with a decorator you can effectively create that type of behavior. So while you're not changing the language, you are changing the semantics of the functions you decorate.
jfm3 16y ago
Decorators are ways to wrap function definitions with code that gets executed at the time the function is defined. Commonly they add stuff that happens every time the function is called. This is certainly something you can do with macros, but it's not even really close to the full semantics. Macros allow you to add new special forms to the language. If Python had macros, for example, the new `with` statements would have been a five line addition to the standard libraries. I really don't see it as "damned close" at all.
- euccastro 16y ago
  Definitely. Indeed decorators are a patch to a shortcoming in Python's ability to define anonymous functions or classes. If you could just say something like: f = memoized(lambda x, y: ... some multiline function) or C = coords_or_vectors(class: ... some class definition) you wouldn't need the kludge: @memoized def f(x, y): ... @coords_or_vectors class C: ... Note that I'm not arguing for the syntax in the first two examples, just the ability to do the equivalent without special cases in the normal syntax of the language. In Scheme you wouldn't need to learn the extra syntax, learn and remember the order in which decorators apply (top to bottom?, inner to outer?). I was actually very happy when decorators got added to Python. But let's face it, they just give you a way out of a corner Python painted itself into. They're nothing to be too proud of.
- softbuilder 16y ago
  "Damned close" perhaps should be qualified as from the point of view of a Pythonista, given the ideals of that approach to programming. I took issue with the specific point that Python "has no macros", and then I qualified it, knowing full well that Lisp macros do more. Decorators ultimately result in a function that is a substitute for another function. That function can have all sorts of wonderful runtime behavior, including access to arguments, the call stack, and other functions. Dismissing them as "adding stuff that happens every time the function is called" misses an entire dimension of their utility.
  - euccastro 16y ago

kenjackson 16y ago

Can you expand on this. What is something that macros can do that decorators can't? And likewise, what are some common uses of macros that decorators can do?

jerf 16y ago
One thing that answers both your questions at once are macros that create new control structures. Macros can take a chunk of syntax tree but not evaluate it, and decorators can only manipulate functions, they can't be used in a function. That said, the usual answers to the question of "Why do I need macros" are slowly but surely being chewed through by Python. The relatively-recent (albeit years old) addition of "with" took another previously-macro-only use case away. I'm not sure what massive win for macros is left. If your Python is "downright verbose" you're probably doing something wrong. It may not always be the Absolute Shortest (TM) but "downright verbose" shouldn't come up often.
- jfm3 16y ago
  >I'm not sure what massive win for macros is left. From my point of view, not being able to define new syntax is as serious a limitation as not being able to define new functions, or data structures. There are a number of specific examples usually cited to explain why the ability to define new syntax is important. Answering these specific examples by adding more static syntax to the language seems to be severely missing the point, to me. Imagine it was functions instead. "Well your language already has a sort() function, but say you want to use an in-place sort instead of what you get with the standard library? You need the ability to define your own functions." "No we don't, our BDFL added an in_place_sort() routine to the standard library for the next version of the language." That doesn't make sense to me at all.
  - jerf 16y ago
    "From my point of view, not being able to define new syntax is as serious a limitation as not being able to define new functions, or data structures." Which is exactly my point. We've gone from "Well, I can use it to define a ternary operator" or "I can use it to define a 'with' block" or "I can create continuations" or a number of other specific examples to "Uh, well, I just like them, here I do this with them" and in my experience there's always an easy "Well, the Pythonic way to do that is this and it's not all that much harder, may even be easier, and is a standard part of a relatively simple language rather than your own hand-rolled thing". It's a very different argument than it was ten years ago when I first got into Python. It's gone from really strong pro-macro points to personal opinion. I respect and acknowledge that opinion, no sarcasm. But it's not the same. Your last paragraph isn't relevant to me because the specificity of your example is misleading. It's really hard to come up with an actual macro use case that is not covered by modern Python and is still actually a good idea. (It's trivial to come up with bad uses of macros not covered by Python, but I'm hardly crying about that.)
    - jfm3 16y ago
      > It's really hard to come up with an actual macro use case that is not covered by modern Python and is still actually a good idea. sqlalchemy (and now mongoalchemy) is begging for macros. See CLSQL for an answer. Regular expressions are infinitely more readable as expressions in a dynamic syntax. See the rx package in Emacs for starters. What if you want shell/Ruby style back quotes, which in essence create a closure that runs a batch job? See my SHELLSHOCK library (jfm3.org) for the Common Lisp extension. It's not much code at all. What if you want large literal strings which respect the indentation of their surrounding code, without uglying things up with <<<EOF ? See my BOXEN library (also jfm3.org) for that Common Lisp extension. What if you want something like Python's r"string", but you want more control over which things are escaped and which are not? See CL-INTERPOL for that. What if you want anaphora in your control structures? What if you want Lex/Yacc style parser generator specifications where the type of productions is extensible, and the productions themselves can be generated dynamically at compile time? (This will make no sense to you unless you've ever had no choice but to run m4 on your .l and .y files.) I could go on. It's not hard for me.
softbuilder 16y ago
As jfm3 points out, you're not going to define new language syntax with decorators. You're also not going to apply them on the spot, inline with other code, since they are defined on a function, and aren't used in the same way as macros are in Lisp. What you can do though is define behavior that varies based on the context. For example in Python you don't have function overloading or types, but with a decorator you can effectively create that type of behavior. So while you're not changing the language, you are changing the semantics of the functions you decorate.

jfm3 16y ago

Decorators are ways to wrap function definitions with code that gets executed at the time the function is defined. Commonly they add stuff that happens every time the function is called. This is certainly something you can do with macros, but it's not even really close to the full semantics. Macros allow you to add new special forms to the language. If Python had macros, for example, the new `with` statements would have been a five line addition to the standard libraries. I really don't see it as "damned close" at all.

euccastro 16y ago
Definitely. Indeed decorators are a patch to a shortcoming in Python's ability to define anonymous functions or classes. If you could just say something like: f = memoized(lambda x, y: ... some multiline function) or C = coords_or_vectors(class: ... some class definition) you wouldn't need the kludge: @memoized def f(x, y): ... @coords_or_vectors class C: ... Note that I'm not arguing for the syntax in the first two examples, just the ability to do the equivalent without special cases in the normal syntax of the language. In Scheme you wouldn't need to learn the extra syntax, learn and remember the order in which decorators apply (top to bottom?, inner to outer?). I was actually very happy when decorators got added to Python. But let's face it, they just give you a way out of a corner Python painted itself into. They're nothing to be too proud of.
softbuilder 16y ago
"Damned close" perhaps should be qualified as from the point of view of a Pythonista, given the ideals of that approach to programming. I took issue with the specific point that Python "has no macros", and then I qualified it, knowing full well that Lisp macros do more. Decorators ultimately result in a function that is a substitute for another function. That function can have all sorts of wonderful runtime behavior, including access to arguments, the call stack, and other functions. Dismissing them as "adding stuff that happens every time the function is called" misses an entire dimension of their utility.
- euccastro 16y ago