5 ms·

I used Racket for a couple projects, but meta programming hasn't turned out to be super useful. When I did use meta programming, I got pretty confused when ret

by yters 7y ago

I used Racket for a couple projects, but meta programming hasn't turned out to be super useful. When I did use meta programming, I got pretty confused when returning to my code after leaving it for awhile. I know PG claims meta programming was the secret to ViaWeb's success, but I'm not sure how exactly that was the case, and even if true whether that is still relevant today. I have found Python to hit the sweet spot much better in terms of rapid prototyping, and providing a runway to making the prototype more rigorous and maintainable.

skohan 7y ago
> I have found Python to hit the sweet spot much better in terms of rapid prototyping, and providing a runway to making the prototype more rigorous and maintainable. I've found Swift to be extremely good for this. It might be hair slower than python to get a project up and running in that very early stage, but it has a similar feeling of "writing the intent of the program not the syntax of the language" and the type system and compile time checking help alleviate a lot of the growing pains you might run into when a project starts getting medium-sized.
wry_discontent 7y ago
I've found Python to inhibit me a lot from moving quickly past the very initial stages. I'm fond of Clojure for that, but it tends to de-emphasize macros as a good way of doing things. Most things are just regular fns.
- yters 7y ago
  Do you find it is easy to come back to your Clojure code and understand what is going on?
  - fctorial 7y ago
    Yeah, tried it on a year old code, looks fine.
  - wry_discontent 7y ago
    The issue with that, I've found, is that you do actually have to read the code. I do JS professionally, and you can definitely tell some of what's going on just from the shape of the code, but I've found Clojure doesn't do that. That said, after a couple minutes looking at something, it's usually clear.
Ididntdothis 7y ago
“I know PG claims meta programming was the secret to ViaWeb's success, but I'm not sure how exactly that was the case, and even if true whether that is still relevant today” I can totally see this if you are alone or a very small tight team is working on the code. You will know what’s going on but once you start adding more people it will be very hard for them to work with such a codebase. I have done some really elegant stuff that made me super productive but looking at the code three years later I myself can barely figure out how it works or how to make changes. other people have almost no chance. Once you grow it’s probably good to stick to paradigms a lot of people in the Industry are familiar with.
- phlakaton 7y ago
  > I have done some really elegant stuff that made me super productive but looking at the code base three years later I myself can barely figure out how it works or how to make changes. Could well apply to me and some of my old C++ projects. Well, except for the super-productive part. PG would say that it's not that the abstractions afforded by metaprogramming are incomprehensible. It's just that, if you are a habitual Blub programmer, everything up the power curve from you is always going to appear weird and hairy. This sort of argument is a terrible one to have with generalizations, though. You should look at specific cases where metaprogramming has been used and analyze them on a case-by-case basis. Sure, a bad, leaky, poorly-documented abstraction is going to lead to grief. On the other hand, umpteen thousands of programmers use the metamagic of Rails on a daily basis and barely bat an eyelash. (Or so it seems to me.)
  - Ididntdothis 7y ago
    Rails is a great example. For me it’s one level further up from simple meta programming. Writing stuff for yourself is one thing but writing stuff that’s universally usable for a lot of people is much harder.
jlarocco 7y ago
After years of using Common Lisp, my opinion on "meta-programming" is that it's wonderful when used sparingly and in just the right way, but a bad idea 99% of the time. If it's not completely obvious what (not how) a macro does just by looking at the code that calls it, then it's almost certainly a bad macro. For example, there's a CL library, iterate, that introduces several new iteration constructs. There are a bunch of complicated macros under the hood, but as an end user I never have to think about them, and the provided interface is obvious, even without knowing anything about the iterate library. For example: (iterate (for (key . item) in alist) (collect key into keys) (collect item into items) (finally (return (values keys items)))) I would definitely consider, "Just because you can doesn't mean you should" to be the number one rule of metaprogramming.
ralphc 7y ago
I think of Lisp vs. Python similarly to PostScript vs PDF. In PostScript you have a full language, can do anything but you often have to do it yourself. PDF's aren't extensible but have _just about_ all the functionality you need to produce a document. An example of macros you often see is surrounding a block of code with open/close pairs to make sure you don't exit the code leaving dangling resources. File open/close, database connections, etc. Python has the __enter__ and __exit__ protocols to handle that. You can't do every bit of metaprogramming in Python, but over the years they've covered close to every contingency a developer comes across.
- lispm 7y ago
  Lisp uses advising for that. In the modern form these are :before, :after and :around methods in CLOS. The difference between some Lisps and CPython is this: Lisp is its own implementation language and provides lower level control flow constructs, which can be compiled to efficient code.
pdonis 7y ago
> I know PG claims meta programming was the secret to ViaWeb's success, but I'm not sure how exactly that was the case IIRC, PG said the advantage of Lisp was being able to add features much more quickly than the competition, so that (a) they could be adding features almost daily that the competition would take weeks or months to match, and (b) whenever the competition added a new feature that looked like a good idea, they could implement it within a day or two, so by the time the press came around to ask them if they had it too, they could say yes. I also remember him saying that Lisp macros were about 25% of Viaweb's code, so a significant part of the advantage of Lisp was metaprogamming. > even if true whether that is still relevant today. It's probably not that much of an advantage any more if your competition is other startups, since they probably have access to basically the same metaprogramming tools that you do. It might still be a significant advantage if your competition is large, mature businesses whose business model you are trying to disrupt. > I have found Python to hit the sweet spot much better in terms of rapid prototyping, and providing a runway to making the prototype more rigorous and maintainable. I have found this to be true as well. But I also find that I do a significant amount of metaprogramming in Python, and one of the key things to transitioning to more rigorous and maintainable code is to make sure that whatever metaprogramming needs to stay there for the long term is packaged into a library or a tool whose internals won't need to change.
- alwalo 7y ago
  Could you write something more on metaprogramming in Python ? And how does this compare with Lisp, what do you use it for ? Any good resources to learn this ?
  - pdonis 7y ago
    Metaprogramming is a fairly broad term, but some of the kinds of things I often find myself doing in Python are: - Writing functions that make functions: often I'll find that I want a family of functions that all follow a the same general pattern. In Lisp this would generally be done with a macro; in Python it can usually be done by writing a function that returns a function. The "maker" function will generally have some parameters that fill in specific parts of the general pattern; the function that gets returned is therefore a closure over those parameters (i.e., it has specific values of them filled in). Doing this with a "maker" function is less error prone (particularly if there are a large number of variations on the general pattern) and follows the DRY (don't repeat yourself) principle, which I think is a very good one. - Writing decorators that add a particular common behavior (such as logging or debug output) to a variety of different functions or classes or methods. This is a common case. (In fact one could argue that decorators in general are examples of metaprogramming.) - Using data to construct code. Database programming is a common application of this: for example, I have a database library I commonly use (it's on GitLab here: https://gitlab.com/pdonis/plib3-dbtools https://gitlab.com/pdonis/plib3-dbtools) which constructs typed namedtuples specific to a database table's fields, and wraps rows returned from queries in them for easier field access. More complicated database ORMs use similar techniques. Unfortunately I don't know of any good resources on metaprogramming in general.

by yters 7y ago

skohan 7y ago

> I have found Python to hit the sweet spot much better in terms of rapid prototyping, and providing a runway to making the prototype more rigorous and maintainable. I've found Swift to be extremely good for this. It might be hair slower than python to get a project up and running in that very early stage, but it has a similar feeling of "writing the intent of the program not the syntax of the language" and the type system and compile time checking help alleviate a lot of the growing pains you might run into when a project starts getting medium-sized.

wry_discontent 7y ago

I've found Python to inhibit me a lot from moving quickly past the very initial stages. I'm fond of Clojure for that, but it tends to de-emphasize macros as a good way of doing things. Most things are just regular fns.

yters 7y ago
Do you find it is easy to come back to your Clojure code and understand what is going on?
- fctorial 7y ago
  Yeah, tried it on a year old code, looks fine.
- wry_discontent 7y ago
  The issue with that, I've found, is that you do actually have to read the code. I do JS professionally, and you can definitely tell some of what's going on just from the shape of the code, but I've found Clojure doesn't do that. That said, after a couple minutes looking at something, it's usually clear.

Ididntdothis 7y ago

“I know PG claims meta programming was the secret to ViaWeb's success, but I'm not sure how exactly that was the case, and even if true whether that is still relevant today” I can totally see this if you are alone or a very small tight team is working on the code. You will know what’s going on but once you start adding more people it will be very hard for them to work with such a codebase. I have done some really elegant stuff that made me super productive but looking at the code three years later I myself can barely figure out how it works or how to make changes. other people have almost no chance. Once you grow it’s probably good to stick to paradigms a lot of people in the Industry are familiar with.

phlakaton 7y ago
> I have done some really elegant stuff that made me super productive but looking at the code base three years later I myself can barely figure out how it works or how to make changes. Could well apply to me and some of my old C++ projects. Well, except for the super-productive part. PG would say that it's not that the abstractions afforded by metaprogramming are incomprehensible. It's just that, if you are a habitual Blub programmer, everything up the power curve from you is always going to appear weird and hairy. This sort of argument is a terrible one to have with generalizations, though. You should look at specific cases where metaprogramming has been used and analyze them on a case-by-case basis. Sure, a bad, leaky, poorly-documented abstraction is going to lead to grief. On the other hand, umpteen thousands of programmers use the metamagic of Rails on a daily basis and barely bat an eyelash. (Or so it seems to me.)
- Ididntdothis 7y ago
  Rails is a great example. For me it’s one level further up from simple meta programming. Writing stuff for yourself is one thing but writing stuff that’s universally usable for a lot of people is much harder.

jlarocco 7y ago

After years of using Common Lisp, my opinion on "meta-programming" is that it's wonderful when used sparingly and in just the right way, but a bad idea 99% of the time. If it's not completely obvious what (not how) a macro does just by looking at the code that calls it, then it's almost certainly a bad macro. For example, there's a CL library, iterate, that introduces several new iteration constructs. There are a bunch of complicated macros under the hood, but as an end user I never have to think about them, and the provided interface is obvious, even without knowing anything about the iterate library. For example: (iterate (for (key . item) in alist) (collect key into keys) (collect item into items) (finally (return (values keys items)))) I would definitely consider, "Just because you can doesn't mean you should" to be the number one rule of metaprogramming.

ralphc 7y ago

I think of Lisp vs. Python similarly to PostScript vs PDF. In PostScript you have a full language, can do anything but you often have to do it yourself. PDF's aren't extensible but have _just about_ all the functionality you need to produce a document. An example of macros you often see is surrounding a block of code with open/close pairs to make sure you don't exit the code leaving dangling resources. File open/close, database connections, etc. Python has the __enter__ and __exit__ protocols to handle that. You can't do every bit of metaprogramming in Python, but over the years they've covered close to every contingency a developer comes across.

lispm 7y ago
Lisp uses advising for that. In the modern form these are :before, :after and :around methods in CLOS. The difference between some Lisps and CPython is this: Lisp is its own implementation language and provides lower level control flow constructs, which can be compiled to efficient code.

pdonis 7y ago

> I know PG claims meta programming was the secret to ViaWeb's success, but I'm not sure how exactly that was the case IIRC, PG said the advantage of Lisp was being able to add features much more quickly than the competition, so that (a) they could be adding features almost daily that the competition would take weeks or months to match, and (b) whenever the competition added a new feature that looked like a good idea, they could implement it within a day or two, so by the time the press came around to ask them if they had it too, they could say yes. I also remember him saying that Lisp macros were about 25% of Viaweb's code, so a significant part of the advantage of Lisp was metaprogamming. > even if true whether that is still relevant today. It's probably not that much of an advantage any more if your competition is other startups, since they probably have access to basically the same metaprogramming tools that you do. It might still be a significant advantage if your competition is large, mature businesses whose business model you are trying to disrupt. > I have found Python to hit the sweet spot much better in terms of rapid prototyping, and providing a runway to making the prototype more rigorous and maintainable. I have found this to be true as well. But I also find that I do a significant amount of metaprogramming in Python, and one of the key things to transitioning to more rigorous and maintainable code is to make sure that whatever metaprogramming needs to stay there for the long term is packaged into a library or a tool whose internals won't need to change.

alwalo 7y ago
Could you write something more on metaprogramming in Python ? And how does this compare with Lisp, what do you use it for ? Any good resources to learn this ?
- pdonis 7y ago
  Metaprogramming is a fairly broad term, but some of the kinds of things I often find myself doing in Python are: - Writing functions that make functions: often I'll find that I want a family of functions that all follow a the same general pattern. In Lisp this would generally be done with a macro; in Python it can usually be done by writing a function that returns a function. The "maker" function will generally have some parameters that fill in specific parts of the general pattern; the function that gets returned is therefore a closure over those parameters (i.e., it has specific values of them filled in). Doing this with a "maker" function is less error prone (particularly if there are a large number of variations on the general pattern) and follows the DRY (don't repeat yourself) principle, which I think is a very good one. - Writing decorators that add a particular common behavior (such as logging or debug output) to a variety of different functions or classes or methods. This is a common case. (In fact one could argue that decorators in general are examples of metaprogramming.) - Using data to construct code. Database programming is a common application of this: for example, I have a database library I commonly use (it's on GitLab here: https://gitlab.com/pdonis/plib3-dbtools https://gitlab.com/pdonis/plib3-dbtools) which constructs typed namedtuples specific to a database table's fields, and wraps rows returned from queries in them for easier field access. More complicated database ORMs use similar techniques. Unfortunately I don't know of any good resources on metaprogramming in general.