5 ms·

Author here. Thanks for your comment. I appreciate your feedback! Let me first comment on your C example: > "a + b; // Oh gosh, what is the + operator?" Ther

Author here. Thanks for your comment. I appreciate your feedback! Let me first comment on your C example: > "a + b; // Oh gosh, what is the + operator?" There is no problem with the "+" operator. It's a language builtin. You can look that up in the manual. In the Lisp case, I did not pick on the "+" operator, at all. The problem is a line below: > "(sum x) ;; ??? Where does "sum" come from?" There is no function "sum" at all! It just looks like a function call, which is it not. You have to know what the enclosing special form "let" does to it's arguments. You could have those problems with C macros as well, but not in this example. > Can you obfuscate the meaning? Hell, yes! > Is that a reason not to allow it and to force obfuscated code by using unsuitable evaluation orders everywhere? Hell, no! I am not picking sides here. I am pinpointing a source of confusion. The Python/Ruby authors took another path and made the language less powerful, so they don't give developers the power to obfuscate the code in this way.

heinrichhartman 8y ago
Reading this again, I realize that the variable name "sum" was chosen badly. I'll change that in the post.
- lispm 8y ago
  Neither Ruby nor Python support the idea of 'code is data'. Sure they are simpler. The concept of code is data is not about simplicity, it's there to support code translations. Most languages have constructs which don't behave like function calls and which have special evaluation rules. A simple example is the IF statement. It takes a condition and two expressions. Depending on the condition only one of the expressions will be evaluated. If you look around, you'll find a bunch of such constructs in most languages. Lisp also has such an IF operator. Lisp additionally allows us to write any amount of additional operators which have evaluation rules, which are not the evalation rule of a function call. Is the IF statement irregular? It isn't. It's just one of many operators which does not follow the evaluation rule of a function call. Something like Ruby has a lot of weirdness in its calling mechanism. Python has strange non-local transfers going on. There are lots of ways we can perceive 'irregularities'.
  - heinrichhartman 8y ago
    > Neither Ruby nor Python support the idea of 'code is data'. Well, Python and Ruby have an "eval" statement. So "code as data" is supported, but discouraged. In contrast Lisp macros / "code as data" are commonly used. Why is there a difference in culture? Is the only reason, that strings are a lot harder to manipulate than lists? Is there more to it? Maybe translated code is harder to maintain and reason about? Maybe debugging and profiling is a lot harder (this is true for C macros at least). I have just shared one confusion that I was struggling with. I found python and ruby much easier to read and pick up. One reason for this is, that programmers are not allowed to define new irregularities. Maybe this insight is helpful for others, who are getting Lisp as well. At least this was my motivation for writing this post.
    - lispm 8y ago
      Python and Ruby use flat strings as input to eval. That's an entirely different thing. To do ANY useful code transformations one needs to parse it. Lisp code already comes in a hierarchical token tree. Ruby and Python represent code as text. That's what Lisp does, too. One has a file of code. But Lisp has something which neither Python and Ruby do: the text is actually a data structure: lists, strings, numbers, vectors, symbols. Lisp provides a function READ. You can read code: CL-USER 73 > (with-input-from-string (stream "(1 + 2)") (print (read stream)) (values)) (1 + 2) It returns a list of three items. This then makes it easy to compute valid code and execute it: CL-USER 74 > (with-input-from-string (stream "(1 + 2)") (let ((code (print (read stream)))) (rotatef (first code) (second code)) (print code) (print (eval code))) (values)) (1 + 2) (+ 1 2) 3 Then put it into a macro: CL-USER 75 > (defmacro infix (expression) (setf expression (copy-tree expression)) (rotatef (first expression) (second expression)) expression) INFIX CL-USER 76 > (infix (1 + 3)) 4 CL-USER 77 > (macroexpand '(infix (1 + 3))) (+ 1 3) T Better, and complete infix macros exist. Or use infix conversion during reading. Write your code with sections of a more conventional syntax: CL-USER 80 > (defun hypot (a b) "Compute the length of the hypotenuse of a right triangle with sides A and B." #I( sqrt(a^^2 + b^^2) )) HYPOT CL-USER 81 > (hypot 1 2) 2.236068 Just put a quote in front of the expression and enter it at the repl -> the result is the transformed code. CL-USER 84 > '#I( sqrt(a^^2 + b^^2) ) (SQRT (+ (EXPT A 2) (EXPT B 2))) There is nothing like this in Ruby or Python. If you look at a Lisp interpreter, it actually executes Lisp data as code. Macros transform Lisp data as code. This is far far away from what Ruby or Python do. Python provides some access to its AST - that's also far away from what Lisp does. Lisp macros don't define irregularities. They do code transformations. Stuff where the language designer/implementor of, say, Python refuses to provide a control structure, you can do it yourself in Lisp in a few minutes or you can spend years to implement really cool stuff. The Common Lisp object system started as a large portable library (with only very little system dependent code), which code be loaded into Lisp and which the provided a very elaborate object system. This also included some clever macro code. I think titles like 'The problem of Lisp' may need a bit more reflection and patience. What is new for you is old news for the Lisp community. The exact same problem report can be already read in Lisp books from the 1960s. It's a bit like blogging 'The world isn't flat!.;-)
kazinator 8y ago
Your article is confused between code walking (recognition of forms) and evaluation order. Code is generally walked top down for the purposes of expansion of macros, compilation or interpretation. Evaluation of nested expressions is generally bottom-up, and in well-behaved Lisp dialects, left-to-right. You cannot simply let your eyes randomly land on some (a b c) in Lisp code and assume that a is a function being called with arguments b c. You have to read the whole sentences and phrases, so to speak. No language is immune to this. Can you assume that * X is a pointer dereference? No, it could be a declarator. Or a fragment of a multiplicative expression. Or the interior of a comment as in /* X. (a, b) could be a comma expression, or the argument list of a function call. I can't think of any language in which I can extract any substring of a sentence that could be a syntactic unit on its own, and correctly understand its meaning without knowing the rest of the sentence. Lisp has to be scanned from the outside in: we have a defun, which encloses a let, and so on. When you're familiar with the piece of code you're working on, you can then take considerable shortcuts. You know you're in the middle of a function, even though the defun is off the screen. You know you're in the middle of the progn-like body of some construct (a loop, or let or whatever), so all the lines of code you see are evaluated forms.
- heinrichhartman 8y ago
  > Your article is confused between code walking (recognition of forms) and evaluation order. Yes, to some extend. > Can you assume that * X is a pointer dereference? [Aside: The C declaration syntax is particularly terrible. In a declaration 'star' means "address of", elsewhere it means "dereference". https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=frontcover&dq=21st+century+C+pointer+declaration+syntax&hl=en&sa=X&ved=0ahUKEwjil_ShpYTeAhWNiKYKHfZ5CAYQ6AEIJzAA#v=onepage&q=21st%20century%20C%20pointer%20declaration%20syntax&f=false https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=front...] > I can't think of any language in which I can extract any substring of a sentence that could be a syntactic unit on its own, and correctly understand its meaning without knowing the rest of the sentence. Well, most languages have expressions/statements that roughly correspond to source lines. When I read a line of python like this: self.raw_requestline = self.rfile.readline(65537) https://github.com/django/django/blob/master/django/core/servers/basehttp.py#L139 https://github.com/django/django/blob/master/django/core/ser... I can be 100% sure that: - `self` is an object with an attribute rfile - `=` is the assignment operator - there is a method `readline` that get's called with an integer argument 65537. All this without checking any enclosing socpes. A random line of lisp: (setcdr current (list :args (cdr current))) https://github.com/emacs-mirror/emacs/blob/master/lisp/wid-edit.el#L742 https://github.com/emacs-mirror/emacs/blob/master/lisp/wid-e... could mean anything depending on the enclosing scopes. Pretty sure setcdr,list,cdr are function calls but who knows. You don't have enough information to decide this from the snippet provided.

heinrichhartman 8y ago

Reading this again, I realize that the variable name "sum" was chosen badly. I'll change that in the post.

lispm 8y ago
Neither Ruby nor Python support the idea of 'code is data'. Sure they are simpler. The concept of code is data is not about simplicity, it's there to support code translations. Most languages have constructs which don't behave like function calls and which have special evaluation rules. A simple example is the IF statement. It takes a condition and two expressions. Depending on the condition only one of the expressions will be evaluated. If you look around, you'll find a bunch of such constructs in most languages. Lisp also has such an IF operator. Lisp additionally allows us to write any amount of additional operators which have evaluation rules, which are not the evalation rule of a function call. Is the IF statement irregular? It isn't. It's just one of many operators which does not follow the evaluation rule of a function call. Something like Ruby has a lot of weirdness in its calling mechanism. Python has strange non-local transfers going on. There are lots of ways we can perceive 'irregularities'.
- heinrichhartman 8y ago
  > Neither Ruby nor Python support the idea of 'code is data'. Well, Python and Ruby have an "eval" statement. So "code as data" is supported, but discouraged. In contrast Lisp macros / "code as data" are commonly used. Why is there a difference in culture? Is the only reason, that strings are a lot harder to manipulate than lists? Is there more to it? Maybe translated code is harder to maintain and reason about? Maybe debugging and profiling is a lot harder (this is true for C macros at least). I have just shared one confusion that I was struggling with. I found python and ruby much easier to read and pick up. One reason for this is, that programmers are not allowed to define new irregularities. Maybe this insight is helpful for others, who are getting Lisp as well. At least this was my motivation for writing this post.
  - lispm 8y ago
    Python and Ruby use flat strings as input to eval. That's an entirely different thing. To do ANY useful code transformations one needs to parse it. Lisp code already comes in a hierarchical token tree. Ruby and Python represent code as text. That's what Lisp does, too. One has a file of code. But Lisp has something which neither Python and Ruby do: the text is actually a data structure: lists, strings, numbers, vectors, symbols. Lisp provides a function READ. You can read code: CL-USER 73 > (with-input-from-string (stream "(1 + 2)") (print (read stream)) (values)) (1 + 2) It returns a list of three items. This then makes it easy to compute valid code and execute it: CL-USER 74 > (with-input-from-string (stream "(1 + 2)") (let ((code (print (read stream)))) (rotatef (first code) (second code)) (print code) (print (eval code))) (values)) (1 + 2) (+ 1 2) 3 Then put it into a macro: CL-USER 75 > (defmacro infix (expression) (setf expression (copy-tree expression)) (rotatef (first expression) (second expression)) expression) INFIX CL-USER 76 > (infix (1 + 3)) 4 CL-USER 77 > (macroexpand '(infix (1 + 3))) (+ 1 3) T Better, and complete infix macros exist. Or use infix conversion during reading. Write your code with sections of a more conventional syntax: CL-USER 80 > (defun hypot (a b) "Compute the length of the hypotenuse of a right triangle with sides A and B." #I( sqrt(a^^2 + b^^2) )) HYPOT CL-USER 81 > (hypot 1 2) 2.236068 Just put a quote in front of the expression and enter it at the repl -> the result is the transformed code. CL-USER 84 > '#I( sqrt(a^^2 + b^^2) ) (SQRT (+ (EXPT A 2) (EXPT B 2))) There is nothing like this in Ruby or Python. If you look at a Lisp interpreter, it actually executes Lisp data as code. Macros transform Lisp data as code. This is far far away from what Ruby or Python do. Python provides some access to its AST - that's also far away from what Lisp does. Lisp macros don't define irregularities. They do code transformations. Stuff where the language designer/implementor of, say, Python refuses to provide a control structure, you can do it yourself in Lisp in a few minutes or you can spend years to implement really cool stuff. The Common Lisp object system started as a large portable library (with only very little system dependent code), which code be loaded into Lisp and which the provided a very elaborate object system. This also included some clever macro code. I think titles like 'The problem of Lisp' may need a bit more reflection and patience. What is new for you is old news for the Lisp community. The exact same problem report can be already read in Lisp books from the 1960s. It's a bit like blogging 'The world isn't flat!.;-)

kazinator 8y ago

Your article is confused between code walking (recognition of forms) and evaluation order. Code is generally walked top down for the purposes of expansion of macros, compilation or interpretation. Evaluation of nested expressions is generally bottom-up, and in well-behaved Lisp dialects, left-to-right. You cannot simply let your eyes randomly land on some (a b c) in Lisp code and assume that a is a function being called with arguments b c. You have to read the whole sentences and phrases, so to speak. No language is immune to this. Can you assume that * X is a pointer dereference? No, it could be a declarator. Or a fragment of a multiplicative expression. Or the interior of a comment as in /* X. (a, b) could be a comma expression, or the argument list of a function call. I can't think of any language in which I can extract any substring of a sentence that could be a syntactic unit on its own, and correctly understand its meaning without knowing the rest of the sentence. Lisp has to be scanned from the outside in: we have a defun, which encloses a let, and so on. When you're familiar with the piece of code you're working on, you can then take considerable shortcuts. You know you're in the middle of a function, even though the defun is off the screen. You know you're in the middle of the progn-like body of some construct (a loop, or let or whatever), so all the lines of code you see are evaluated forms.

heinrichhartman 8y ago
> Your article is confused between code walking (recognition of forms) and evaluation order. Yes, to some extend. > Can you assume that * X is a pointer dereference? [Aside: The C declaration syntax is particularly terrible. In a declaration 'star' means "address of", elsewhere it means "dereference". https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=frontcover&dq=21st+century+C+pointer+declaration+syntax&hl=en&sa=X&ved=0ahUKEwjil_ShpYTeAhWNiKYKHfZ5CAYQ6AEIJzAA#v=onepage&q=21st%20century%20C%20pointer%20declaration%20syntax&f=false https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=front...] > I can't think of any language in which I can extract any substring of a sentence that could be a syntactic unit on its own, and correctly understand its meaning without knowing the rest of the sentence. Well, most languages have expressions/statements that roughly correspond to source lines. When I read a line of python like this: self.raw_requestline = self.rfile.readline(65537) https://github.com/django/django/blob/master/django/core/servers/basehttp.py#L139 https://github.com/django/django/blob/master/django/core/ser... I can be 100% sure that: - `self` is an object with an attribute rfile - `=` is the assignment operator - there is a method `readline` that get's called with an integer argument 65537. All this without checking any enclosing socpes. A random line of lisp: (setcdr current (list :args (cdr current))) https://github.com/emacs-mirror/emacs/blob/master/lisp/wid-edit.el#L742 https://github.com/emacs-mirror/emacs/blob/master/lisp/wid-e... could mean anything depending on the enclosing scopes. Pretty sure setcdr,list,cdr are function calls but who knows. You don't have enough information to decide this from the snippet provided.