3 ms·

(before I begin: I am familiar with macros & DSLs, code generation, building a custom language layer and then write your app in that custom language and so on,

(before I begin: I am familiar with macros & DSLs, code generation, building a custom language layer and then write your app in that custom language and so on, and I buy into almost all of it) I am sure the code above is poetry for those deep into it, but for a person who is a bit rusty (or god forbid, new to the code base), this is basically indecipherable. Look at how many tokens are there in a line like (each c (dedup (map downcase (trues [uvar _ topcolor] (keys profs*)))) How many people can answer the question of which ones are macros, functions, simple identifiers and so on? In that particular example I am stumped at [uvar _ topcolor] and can't get past it. Yes, I have read Paul's thesis on Lisp code density (quick summary: each line is dense, but there is a lot less overall code so it evens out or better), but my counter would be that there is a density beyond which most mortal brains can't penetrate. Without intending to cause offense, I am going to say that this is tending to write-only code. Sure, the equivalent Ruby or Python (leave alone blubs like Java or C#) would be pages long, but I suspect that lower density leads to better understandability.

pg 19y ago
Look at how many tokens are there in a line like There's not really a concept of lines in Lisp; you can break expressions wherever you like. E.g. you could also write: (defop topcolors req (minipage "Custom Colors" (tab (each c (dedup (map downcase (trues [uvar _ topcolor] (keys profs*)))) (tr (td c) (tdcolor (hex>color c) (hspace 30)))))))
sridharvembu 19y ago
Actually, the equivalent Ruby/Python would use a simple SQL query. I have long noticed how SQL is the poor-man's functional "DSL" for such uses ...
nostrademons 19y ago
I don't think that's a problem with macros so much as obscure naming conventions and an obsession with brevity. In the original example, I had a pretty good idea what minipage, tab, each, dedup, map, keys, downcase, tr, td, and tdcolor mean. I can figure out c (color?) and req (request?) from context. I have no clue about defopa, uvar, trues, _, [], profs, topcolor, hex>color or hspace (or I have guesses, but wouldn't trust my guesses enough to pin down the semantics). From PG's post above, it seems like those are fairly evenly distributed across macros and functions. I don't need to know whether td is a macro or function to guess that it outputs a table data cell. I do need to know that it outputs a table data cell, and I only know that because I'm familiar with HTML.
- sridharvembu 19y ago
  Agreed on the obsession with brevity. Reminds me of mathematical proofs, where you work out everything offline, and then make the proof elegantly brief. There is a similar process going on here - writing macros really does boil down to that kind of offline work. Just as with such elegant proofs, though, you do need a pencil and paper to trace through the steps. And if the programmer were anything but of the highest caliber, it is fairly easy to get tripped up. The way blub languages handle (the absence of) macros for td, tr etc. is to invent templating languages (JSP etc) where you embed Java inside HTML. You basically have a DSL living in a separate syntactic space. That gets the job done, for the specific problem space at hand (web apps). I have come to like that syntactic space separation: HTML lives as HTML, SQL lives as SQL and so on. With macros, you are getting everything to live in the same syntactic space, so you get the proliferation of tokens, without that structural separation that aids in comprehension.
shiro 19y ago
I wonder what makes the difference in person's perception of code. I think I have a good idea of what the pg's code fragment does at a first grance. (I do know what [ _ ] means in Arc, though; pg mentioned it somewhere.) I doubt it's the density that's the problem. I also doubt macros are. It's more like whether you recognize "the way" in which you're supposed to write, in the given language. Any language, verbose or terse, is indecipherable until you get familiar with the way even you know the syntax and semantics of the language.
- sridharvembu 19y ago
  Granted. To be fair to Paul, I got about 60% of it based on a (rusty) idea of lisp. The _ got me (should've thought of Perl)... but now it is clear. My broader point still holds though - there is a maximal density ... And the reason I commented on it was that in posting that code fragment, the message was that it would be "kind of obvious" to any reasonably competent programmers how macros really help. I agree macros really do help, so no quarrel there. In this case, we understand the "application" perfectly, and then we get to see the code. It still got (some of us) stumped, and (some of us again) actually know about macros and are moderately familiar with Lisp idioms. That is why I asked if density can go too far. Compare that to something like LINQ, which I am coincidentally studying. It is a DSL for querying object collections (a similar use case to the code here). With no special background, you can go in, read code, and get a reasonable idea of what is going on. LINQ actually lets you get rid of a lot of code (something I agree with in general), so it lives much higher on the density scale than a typical blub.
  - shiro 19y ago
    I see your point. So the question is actually whether a language designer can only care his "types"---those who have similar thinking habit and thus has less difficulty to understand the code---or should target at larger audiences. Ultimately that's the choice of the designer; not all languages need to address to the broad range of programmers.
  - pchristensen 19y ago
    I don't think the point was that everyone was supposed to be able to understand every detail of the code. I think he was showing that by using a bunch of macros, he could add a new command to the web server that would create a new type of page with a very small amount of code. How would the code look to create the top color page if it was written in another language? Less code = faster development, easier maintenance, fewer bugs
mnemonicsloth 19y ago
You're just confused by some syntactic sugar. [uvar _ topcolor] is shorthand for (lambda (x) (uvar x topcolor)), according to "Arc at 3 Weeks" each = mutant foreach dedup = remove duplicates (trues [] (keys profs*)) would mean "map the lambda (predicate) over the list of keys to the hash of profiles, returning a list of elements for which it returned #t." A few more specifics about the application and language would make this comprehensible.
- pg 19y ago
  Close, except trues returns the values returned by the function, not the elements for which the function returned true. (def keep (f seq) (rem [no (f _)] seq)) (def trues (f seq) (rem nil (map f seq)))
tel 19y ago
With dense code, you have to think about every word. With sparse code, you have to think along with every line/block/idiom. Both require the (same?) thought process, but with dense code you get two advantages: - You can look ahead easier and get a bird's-eye-view - Once you learn an operator, recognizing it (and writing it correctly) becomes a single word instead of said line/block/idiom. My guess is that [uvar _ topcolor] is a function similar to (lambda (x) (uvar x topcolor)). That's just a guess, but once I learn whatever it is, I'd wage that it's easier to recognize and debug the bracket syntax.
Zak 19y ago
Let me see if I can translate that in to Ruby. Note that the expression isn't closed, so I'm changing it to: (each c (dedup (map downcase (trues [uvar _ topcolor] (keys profs*)))) (do-stuff-with c)) which I think translates roughly to: keys(profs*).find_all{|prof| uvar(prof, topcolor)}.map(lambda {|str| str.downcase}).uniq.each do |c| do_stuff_with(c) end This could be slightly off, as I'm having to guess at what some of this stuff does. It's almost as dense and no more readable.
- deleted 19y ago
  [deleted]
- paul 19y ago
  For me, your ruby version is more readable because the order is more sequential (like a pipeline). The arc version has to be read from the inside-out, vs left-to-right in ruby.
  - brlewis 19y ago
    I always read s-expressions left-to-right. The outer expression tells you what you're getting. I usually don't have to go very far in before I know what the code is intended to do.

pg 19y ago

Look at how many tokens are there in a line like There's not really a concept of lines in Lisp; you can break expressions wherever you like. E.g. you could also write: (defop topcolors req (minipage "Custom Colors" (tab (each c (dedup (map downcase (trues [uvar _ topcolor] (keys profs*)))) (tr (td c) (tdcolor (hex>color c) (hspace 30)))))))

sridharvembu 19y ago

Actually, the equivalent Ruby/Python would use a simple SQL query. I have long noticed how SQL is the poor-man's functional "DSL" for such uses ...

nostrademons 19y ago

I don't think that's a problem with macros so much as obscure naming conventions and an obsession with brevity. In the original example, I had a pretty good idea what minipage, tab, each, dedup, map, keys, downcase, tr, td, and tdcolor mean. I can figure out c (color?) and req (request?) from context. I have no clue about defopa, uvar, trues, _, [], profs, topcolor, hex>color or hspace (or I have guesses, but wouldn't trust my guesses enough to pin down the semantics). From PG's post above, it seems like those are fairly evenly distributed across macros and functions. I don't need to know whether td is a macro or function to guess that it outputs a table data cell. I do need to know that it outputs a table data cell, and I only know that because I'm familiar with HTML.

sridharvembu 19y ago
Agreed on the obsession with brevity. Reminds me of mathematical proofs, where you work out everything offline, and then make the proof elegantly brief. There is a similar process going on here - writing macros really does boil down to that kind of offline work. Just as with such elegant proofs, though, you do need a pencil and paper to trace through the steps. And if the programmer were anything but of the highest caliber, it is fairly easy to get tripped up. The way blub languages handle (the absence of) macros for td, tr etc. is to invent templating languages (JSP etc) where you embed Java inside HTML. You basically have a DSL living in a separate syntactic space. That gets the job done, for the specific problem space at hand (web apps). I have come to like that syntactic space separation: HTML lives as HTML, SQL lives as SQL and so on. With macros, you are getting everything to live in the same syntactic space, so you get the proliferation of tokens, without that structural separation that aids in comprehension.

shiro 19y ago

I wonder what makes the difference in person's perception of code. I think I have a good idea of what the pg's code fragment does at a first grance. (I do know what [ _ ] means in Arc, though; pg mentioned it somewhere.) I doubt it's the density that's the problem. I also doubt macros are. It's more like whether you recognize "the way" in which you're supposed to write, in the given language. Any language, verbose or terse, is indecipherable until you get familiar with the way even you know the syntax and semantics of the language.

sridharvembu 19y ago
Granted. To be fair to Paul, I got about 60% of it based on a (rusty) idea of lisp. The _ got me (should've thought of Perl)... but now it is clear. My broader point still holds though - there is a maximal density ... And the reason I commented on it was that in posting that code fragment, the message was that it would be "kind of obvious" to any reasonably competent programmers how macros really help. I agree macros really do help, so no quarrel there. In this case, we understand the "application" perfectly, and then we get to see the code. It still got (some of us) stumped, and (some of us again) actually know about macros and are moderately familiar with Lisp idioms. That is why I asked if density can go too far. Compare that to something like LINQ, which I am coincidentally studying. It is a DSL for querying object collections (a similar use case to the code here). With no special background, you can go in, read code, and get a reasonable idea of what is going on. LINQ actually lets you get rid of a lot of code (something I agree with in general), so it lives much higher on the density scale than a typical blub.
- shiro 19y ago
  I see your point. So the question is actually whether a language designer can only care his "types"---those who have similar thinking habit and thus has less difficulty to understand the code---or should target at larger audiences. Ultimately that's the choice of the designer; not all languages need to address to the broad range of programmers.
- pchristensen 19y ago
  I don't think the point was that everyone was supposed to be able to understand every detail of the code. I think he was showing that by using a bunch of macros, he could add a new command to the web server that would create a new type of page with a very small amount of code. How would the code look to create the top color page if it was written in another language? Less code = faster development, easier maintenance, fewer bugs

mnemonicsloth 19y ago

You're just confused by some syntactic sugar. [uvar _ topcolor] is shorthand for (lambda (x) (uvar x topcolor)), according to "Arc at 3 Weeks" each = mutant foreach dedup = remove duplicates (trues [] (keys profs*)) would mean "map the lambda (predicate) over the list of keys to the hash of profiles, returning a list of elements for which it returned #t." A few more specifics about the application and language would make this comprehensible.

pg 19y ago
Close, except trues returns the values returned by the function, not the elements for which the function returned true. (def keep (f seq) (rem [no (f _)] seq)) (def trues (f seq) (rem nil (map f seq)))

tel 19y ago

With dense code, you have to think about every word. With sparse code, you have to think along with every line/block/idiom. Both require the (same?) thought process, but with dense code you get two advantages: - You can look ahead easier and get a bird's-eye-view - Once you learn an operator, recognizing it (and writing it correctly) becomes a single word instead of said line/block/idiom. My guess is that [uvar _ topcolor] is a function similar to (lambda (x) (uvar x topcolor)). That's just a guess, but once I learn whatever it is, I'd wage that it's easier to recognize and debug the bracket syntax.

Zak 19y ago

Let me see if I can translate that in to Ruby. Note that the expression isn't closed, so I'm changing it to: (each c (dedup (map downcase (trues [uvar _ topcolor] (keys profs*)))) (do-stuff-with c)) which I think translates roughly to: keys(profs*).find_all{|prof| uvar(prof, topcolor)}.map(lambda {|str| str.downcase}).uniq.each do |c| do_stuff_with(c) end This could be slightly off, as I'm having to guess at what some of this stuff does. It's almost as dense and no more readable.

deleted 19y ago
[deleted]
paul 19y ago
For me, your ruby version is more readable because the order is more sequential (like a pipeline). The arc version has to be read from the inside-out, vs left-to-right in ruby.
- brlewis 19y ago
  I always read s-expressions left-to-right. The outer expression tells you what you're getting. I usually don't have to go very far in before I know what the code is intended to do.