7 ms·

I've been learning lisp this past week. I think I get the main idea of "code is data and data is code", but I struggle to understand how it's more powerful than

I've been learning lisp this past week. I think I get the main idea of "code is data and data is code", but I struggle to understand how it's more powerful than, say, making a function factory in Python.

agumonkey 4y ago
maybe a function factory is just an OO pattern for return closures, it's difficult to not perceive things from 'now', but i'll be frank i 1) never liked DP 2) couldn't spend more than 5 minutes reading about them as it seemed more verbose and redundant
majormunky 4y ago
I'm also learning lisp, mainly as a part of learning emacs, so I may not be super accurate here. In python, when you have a reference to a function, its a bit opaque, you can tell the thing is a function, but I don't think you can really go into the function object, and alter it lets say. You may be able to using the AST or something, but its not a "first-class" thing you can do. In lisp (and code is data, data is code), the function is also a list, so you would be able to use all the normal programming tools to iterate over the list, add new items to the list, etc, which can change how the function runs.
- altruios 4y ago
  Such a concept could use an example to solidify that point. If I have a function Foo that adds two numbers together. How is that represented in lisp (as a list you say?)? How would one alter that function/lisp-list: to multiply numbers instead?
  - majormunky 4y ago
    I've not really encountered any situations where I've needed to do anything like this yet (still learning), but, I found this example, where there's an existing function, and someone alters the body of the function to create a new function. My guess is that there's various ways of doing this type of stuff: https://stackoverflow.com/a/1220198 https://stackoverflow.com/a/1220198 Edit: a macro example, which I think may be the more common route of doing this type of stuff? https://lisp-journey.gitlab.io/blog/common-lisp-macros-by-example-tutorial/ https://lisp-journey.gitlab.io/blog/common-lisp-macros-by-ex...
  - drekipus 4y ago
    > How is that represented in lisp (as a list you say?) I'm only a beginner in lisp too, but it as simple as (defun add-numbers (a b) (+ a b)) if you look at it as "everything between () is a list" then you'll find the above code is actually: [word word [word word] [word word word]] to get it to multiply numbers instead, you'd just get the last item of the list `(+ a b)` -- replace the first item `+` with `*` (using other list functions, etc..) and then re-evaluate the entire list again, So now your function will multiply values instead of adding them.. The "power" aspect comes from the fact that we use "other list functions" to change the code, ie: code is data that we can manipulate. note, probable errors, like I said, I'm new
- kazinator 4y ago
  This is not what "code is data" is supposed to mean. It means "source code is data": the source code of Lisp is a data structure. You shouldn't do it in any Lisp production code. E.g. in ANSI Lisp, self-modifying code is undefined behavior. Even if you get some expected results, it may not work portably, and won't work if the code is compiled. Compiled Lisp code doesn't have a list representation any more. Code-is-data lets us have a transformation pass on the code with application-defined rules, before it is interpreted or compiled. This is a recursive step that returns a new version of the code based on the old
cess11 4y ago
Not sure what you mean by "function factory", could you elaborate? Isn't it metaclasses and decorators that are used for metaprogramming in Python? In my opinion the 'everything is an expression' is the really tasty part of the lisps, which goes well with quoting.
- halkony 4y ago
  My naive interpretation of `defmacro` is passing data into a function to return another function. From my python POV that would be the same as returning a callable from a function (decorators being one example). I think top comment does a good job explaining that while I'd get the same functional result, Python objects have tons of abstraction behind them that make them harder to work with than lisp combinations.
  - Jtsummers 4y ago
    Macros don't (necessarily) return functions. They take a set of values and then produce an s-expression, which in turn is read by the reader and evaluated. For instance a user-created `when` macro would be something like this (not tested): (defmacro when (condition &body body) `(if ,condition (progn ,@body))) That's not a new function that's been returned, just a new expression which can then be read (and macro expansion can be a repeated process as its read so you can use nested macros in macros in macros, even recursive macros but take care there). You can use macros as you describe, in a manner similar to a Python decorator, but that's only one of many possible uses.
kfoley 4y ago
I used to have the same issue understanding the benefit. One simple example that helped it clicked for me was considering the `when` or `unless` macro. They're just very simple macros based around the `if` special form. Because they're macros, they can just treat the body as data, i.e. it's not executed as part of argument resolution. If you wanted to make a `when` function in Python, the "body" would have to be a callable like a lambda or named function. I know this probably isn't the most compelling use case but I found it to be a really simple way to understand some of the benefits of macros and how they could be used elsewhere.
- halkony 4y ago
  So let's say i have a lisp macro along the lines of (analyze 1 2 expensiveComputation) In python, to prevent evaluating all arguments when running analyze, I'd have to pass expensiveComputation as a callable or a function partial and remember to call it if I need it. Whereas in lisp, I can pass whatever expensive variable I want into the macro because it will only get evaluated when using (expensiveComputation ...) and that's baked into the language.
twawaaay 4y ago
There is a number of reasons that make Lisp macros so much more powerful than functions in any other language. Consider you want to implement printf that is compatible with C printf. As the function implementor, you want something like printf("%d", 5+2, anExpensiveFunctionThatProducesAString()) do the right thing and print "7" on the terminal, but you get many more options and mechanisms you can use when implementing it in Lisp vs other languages. Every other language: * every time printf is called, all of the arguments have already been evaluated. There is no way to avoid calling the expensive function when the format indicates the result is not needed. * every time the function runs you have to parse the format string. You could try and be smart and cache parsing results but this will quickly make it very expensive and complex and might cause problems when format strings are generated dynamically. * your function cannot make use of the fact that both the format string and the argument never change Lisp macro: * you can decide to only evaluate arguments when they are needed, * you can notice that the format string is static and output code IN PLACE of the macro based on this information. This means you have the option to make parsing happen only at compile time if it is possible. * you can analyse the arguments further and decide to output code that does princ('7') IN PLACE of the macro. The resulting code will have no operations to parse or analyse anything and will always execute quickly and efficiently. In general this is the way I think about Lisp macros vs functions in other languages: "Lisp macros allow you to run logic at compile time that can create code dynamically that does exactly what you need it to do, no more, no less, based on macro arguments and/or unevaluated macro arguments and/or the environment." Lisp macros are ultimate in building abstractions. Where in other languages you have to contend with the language always "sticking out" of your abstractions -- there is only so much you can do to hide the underlying language, in Common Lisp you can always hide the language completely. Implementing your abstraction is taking a data structure (can be for example an XML file) and running an arbitrary logic to generate another data structure (a function) that is then outputted in place of the macro. If the XML describes your binary file structure, the macro can take those descriptions and generate perfect parsing code that does exactly what is needed without any extra stack frames, conditionals, syntax artifacts, etc.

agumonkey 4y ago

maybe a function factory is just an OO pattern for return closures, it's difficult to not perceive things from 'now', but i'll be frank i 1) never liked DP 2) couldn't spend more than 5 minutes reading about them as it seemed more verbose and redundant

majormunky 4y ago

I'm also learning lisp, mainly as a part of learning emacs, so I may not be super accurate here. In python, when you have a reference to a function, its a bit opaque, you can tell the thing is a function, but I don't think you can really go into the function object, and alter it lets say. You may be able to using the AST or something, but its not a "first-class" thing you can do. In lisp (and code is data, data is code), the function is also a list, so you would be able to use all the normal programming tools to iterate over the list, add new items to the list, etc, which can change how the function runs.

altruios 4y ago
Such a concept could use an example to solidify that point. If I have a function Foo that adds two numbers together. How is that represented in lisp (as a list you say?)? How would one alter that function/lisp-list: to multiply numbers instead?
- majormunky 4y ago
  I've not really encountered any situations where I've needed to do anything like this yet (still learning), but, I found this example, where there's an existing function, and someone alters the body of the function to create a new function. My guess is that there's various ways of doing this type of stuff: https://stackoverflow.com/a/1220198 https://stackoverflow.com/a/1220198 Edit: a macro example, which I think may be the more common route of doing this type of stuff? https://lisp-journey.gitlab.io/blog/common-lisp-macros-by-example-tutorial/ https://lisp-journey.gitlab.io/blog/common-lisp-macros-by-ex...
- drekipus 4y ago
  > How is that represented in lisp (as a list you say?) I'm only a beginner in lisp too, but it as simple as (defun add-numbers (a b) (+ a b)) if you look at it as "everything between () is a list" then you'll find the above code is actually: [word word [word word] [word word word]] to get it to multiply numbers instead, you'd just get the last item of the list `(+ a b)` -- replace the first item `+` with `*` (using other list functions, etc..) and then re-evaluate the entire list again, So now your function will multiply values instead of adding them.. The "power" aspect comes from the fact that we use "other list functions" to change the code, ie: code is data that we can manipulate. note, probable errors, like I said, I'm new
kazinator 4y ago
This is not what "code is data" is supposed to mean. It means "source code is data": the source code of Lisp is a data structure. You shouldn't do it in any Lisp production code. E.g. in ANSI Lisp, self-modifying code is undefined behavior. Even if you get some expected results, it may not work portably, and won't work if the code is compiled. Compiled Lisp code doesn't have a list representation any more. Code-is-data lets us have a transformation pass on the code with application-defined rules, before it is interpreted or compiled. This is a recursive step that returns a new version of the code based on the old

cess11 4y ago

Not sure what you mean by "function factory", could you elaborate? Isn't it metaclasses and decorators that are used for metaprogramming in Python? In my opinion the 'everything is an expression' is the really tasty part of the lisps, which goes well with quoting.

halkony 4y ago
My naive interpretation of `defmacro` is passing data into a function to return another function. From my python POV that would be the same as returning a callable from a function (decorators being one example). I think top comment does a good job explaining that while I'd get the same functional result, Python objects have tons of abstraction behind them that make them harder to work with than lisp combinations.
- Jtsummers 4y ago
  Macros don't (necessarily) return functions. They take a set of values and then produce an s-expression, which in turn is read by the reader and evaluated. For instance a user-created `when` macro would be something like this (not tested): (defmacro when (condition &body body) `(if ,condition (progn ,@body))) That's not a new function that's been returned, just a new expression which can then be read (and macro expansion can be a repeated process as its read so you can use nested macros in macros in macros, even recursive macros but take care there). You can use macros as you describe, in a manner similar to a Python decorator, but that's only one of many possible uses.

kfoley 4y ago

I used to have the same issue understanding the benefit. One simple example that helped it clicked for me was considering the `when` or `unless` macro. They're just very simple macros based around the `if` special form. Because they're macros, they can just treat the body as data, i.e. it's not executed as part of argument resolution. If you wanted to make a `when` function in Python, the "body" would have to be a callable like a lambda or named function. I know this probably isn't the most compelling use case but I found it to be a really simple way to understand some of the benefits of macros and how they could be used elsewhere.

halkony 4y ago
So let's say i have a lisp macro along the lines of (analyze 1 2 expensiveComputation) In python, to prevent evaluating all arguments when running analyze, I'd have to pass expensiveComputation as a callable or a function partial and remember to call it if I need it. Whereas in lisp, I can pass whatever expensive variable I want into the macro because it will only get evaluated when using (expensiveComputation ...) and that's baked into the language.

twawaaay 4y ago

There is a number of reasons that make Lisp macros so much more powerful than functions in any other language. Consider you want to implement printf that is compatible with C printf. As the function implementor, you want something like printf("%d", 5+2, anExpensiveFunctionThatProducesAString()) do the right thing and print "7" on the terminal, but you get many more options and mechanisms you can use when implementing it in Lisp vs other languages. Every other language: * every time printf is called, all of the arguments have already been evaluated. There is no way to avoid calling the expensive function when the format indicates the result is not needed. * every time the function runs you have to parse the format string. You could try and be smart and cache parsing results but this will quickly make it very expensive and complex and might cause problems when format strings are generated dynamically. * your function cannot make use of the fact that both the format string and the argument never change Lisp macro: * you can decide to only evaluate arguments when they are needed, * you can notice that the format string is static and output code IN PLACE of the macro based on this information. This means you have the option to make parsing happen only at compile time if it is possible. * you can analyse the arguments further and decide to output code that does princ('7') IN PLACE of the macro. The resulting code will have no operations to parse or analyse anything and will always execute quickly and efficiently. In general this is the way I think about Lisp macros vs functions in other languages: "Lisp macros allow you to run logic at compile time that can create code dynamically that does exactly what you need it to do, no more, no less, based on macro arguments and/or unevaluated macro arguments and/or the environment." Lisp macros are ultimate in building abstractions. Where in other languages you have to contend with the language always "sticking out" of your abstractions -- there is only so much you can do to hide the underlying language, in Common Lisp you can always hide the language completely. Implementing your abstraction is taking a data structure (can be for example an XML file) and running an arbitrary logic to generate another data structure (a function) that is then outputted in place of the macro. If the XML describes your binary file structure, the macro can take those descriptions and generate perfect parsing code that does exactly what is needed without any extra stack frames, conditionals, syntax artifacts, etc.