3 ms·

I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. How is writing special operators in Lisp different from writing sp

I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. How is writing special operators in Lisp different from writing special functions in any other language?

marcosdumay 3y ago
PG has other essays where he claims it's because the lisp version of operators can just do more things. And that's true, lisp macros are way more powerful than what most languages support. The newer ones (and some "progressive" old ones) got template systems that are almost as powerful, but the most used languages simply don't have anything comparable. That said, nowadays using a lot of that kind of feature is normally frowned upon.
- leetrout 3y ago
  I have a theory it is frowned upon because there are not a lot of small shops left. Everyone wants be a cog at a big tech co where its leetcode interviews and code that large groups can (somewhat) easily maintain. In these cases the languages used are generally uniform so your skills easily transfer and you aren't suffering as many 200% problems by learning both the language and the authors peculiar add-ons. Of course it still happens just not as easy with rigid languages such as Go. It's all tradeoffs.
  - marcosdumay 3y ago
    As a rule, when something starts deserving being called by "shop", it is already too large for heavy usage of macros. In fact, I'm not sure it scales for more than a single developer. That is, unless you treat them as another language, with tooling, documentation, and stability. Then they scale again, but it requires quite a lot of work.
    - leetrout 3y ago
      Is this website an example? Dang has been talking about a set of performance changes for a long time and I always wonder if it is slow because it is so bespoke.
jksmith 3y ago
Extreme reinforcement of your point: Read Leo Brodie's classics "Starting Forth" and "Thinking Forth."
- MaxBarraclough 3y ago
  Chuck Moore, inventor of Forth, is also an advocate of bottom-up development. > You mentioned that a lot of Forth programs you’ve seen look like C programs. How do you design a better Forth program? > Chuck: Bottom-up. > First, you presumably have some I/O signals that you have to generate, so you generate them. Then you write some code that controls the generation of those signals. Then you work your way up until finally you have the highest-level word, and you call it go and you type go and everything happens. > I have very little faith in systems analysts who work top-down. They decide what the problem is and then they factor it in such a way that it can be very difficult to implement. https://www.oreilly.com/library/view/masterminds-of-programming/9780596801670/ch04.html https://www.oreilly.com/library/view/masterminds-of-programm...
Jtsummers 3y ago
> I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. It can be, and he even says as much: >> Bottom-up design is possible to a certain degree in languages other than Lisp. Whenever you see library functions, bottom-up design is happening. I'll also point to kazinator's quote of Ken Thompson in the previous discussion linked by dang: >> It is the way I think. I am a very bottom-up thinker. If you give me the right kind of Tinker Toys, I can imagine the building. I can sit there and see primitives and recognize their power to build structures a half mile high, if only I had just one more to make it functionally complete. I can see those kinds of things. The converse is true, too, I think. I can’t— from the building—imagine the Tinker Toys. When I see a top-down description of a system or language that has infinite libraries described by layers and layers, all I just see is a morass. I can’t get a feel for it. I can’t understand how the pieces fit; I can’t understand something presented to me that’s very complex. Maybe I do what I do because if I built anything more complicated, I couldn’t understand it. I really must break it down into little pieces. - Ken Thompson, Unix and Beyond: An Interview with Ken Thompson, 1999 And if you use a TDD style of development (in particular, but test-heavy in general with frequent use of unit and integration tests below the full end-to-end-test level) you'll also likely stumble onto a similar bottom-up style of development. I think, circa 1993, his emphasis on Lisp and bottom-up development made a lot more sense than it does today with the increasing availability of interactive development environments (essentially every dynamically typed language, pretty much; but even many others like Java and C#) and increased emphasis on test-heavy development methodologies.
norir 3y ago
A function call has runtime overhead that a macro may not. There are also things that can be difficult to express as a function in the language type system but that are straightforward to implement as a macro. Lisp isn't the only language with a strong macro system but it disproportionately attracts programmers who value the power and expressivity that a strong macro system provides. There are a number of reasons that bottom up programming in the style PG seems to be advocating for is not popular in industry. Industrial programming favors the median developer while bottom up is a technique that works best with small teams of skilled programmers. Big cos obviously prefer "safe" languages, but even startups are generally conservative due to the vc induced pressure towards hypergrowth, which also favors the median developer since hypergrowth requires hiring. Writing good programs from the bottom up requires more skill[1], but can lead to substantially better programs by certain criteria. But it may also be considered inscrutable by the median developer. The median developer can also misuse the power afforded by the bottom up language and write something much worse than they would have been able to in a more conservative language. This isn't a knock on the median developer, btw, just my perspective on why this style is rarely found in industry. [1] neither LLMs nor stack overflow are of much help when you have essentially written a new language to solve your problem.
- TerrifiedMouse 3y ago
  > There are a number of reasons that bottom up programming in the style PG seems to be advocating for is not popular in industry. I think another reason is that it’s just more work for another programmer to grok a bottom-up written program written with its own DSL then a program that’s bent to fit the existing (more rigid) language since they already know the existing language.
narag 3y ago
I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. My no. 1 suspect is OOP or, more specifically, languages like Java that won't let you write a function. You can bypass that limitation creating a class whose only reason is to hold stand-alone functions. I don't do that, lately I don't much Java at all, but I've seen huge Util.java classes. In sane languages I used to start exploratory programs with two files: one for the main flow, the other for utility functions. Later the former would become different windows, the latter different libraries. Classes grew slowly, one functionality aspect at a time. Forced OOP nudge you into a priori design the class structure before understanding the domain. Bottom-up advantage is that, having the primitives earlier, you can test with real data from the start and show the results to the domain experts.

I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. How is writing special operators in Lisp different from writing special functions in any other language?

marcosdumay 3y ago

PG has other essays where he claims it's because the lisp version of operators can just do more things. And that's true, lisp macros are way more powerful than what most languages support. The newer ones (and some "progressive" old ones) got template systems that are almost as powerful, but the most used languages simply don't have anything comparable. That said, nowadays using a lot of that kind of feature is normally frowned upon.

leetrout 3y ago
I have a theory it is frowned upon because there are not a lot of small shops left. Everyone wants be a cog at a big tech co where its leetcode interviews and code that large groups can (somewhat) easily maintain. In these cases the languages used are generally uniform so your skills easily transfer and you aren't suffering as many 200% problems by learning both the language and the authors peculiar add-ons. Of course it still happens just not as easy with rigid languages such as Go. It's all tradeoffs.
- marcosdumay 3y ago
  As a rule, when something starts deserving being called by "shop", it is already too large for heavy usage of macros. In fact, I'm not sure it scales for more than a single developer. That is, unless you treat them as another language, with tooling, documentation, and stability. Then they scale again, but it requires quite a lot of work.
  - leetrout 3y ago
    Is this website an example? Dang has been talking about a set of performance changes for a long time and I always wonder if it is slow because it is so bespoke.

jksmith 3y ago

Extreme reinforcement of your point: Read Leo Brodie's classics "Starting Forth" and "Thinking Forth."

MaxBarraclough 3y ago
Chuck Moore, inventor of Forth, is also an advocate of bottom-up development. > You mentioned that a lot of Forth programs you’ve seen look like C programs. How do you design a better Forth program? > Chuck: Bottom-up. > First, you presumably have some I/O signals that you have to generate, so you generate them. Then you write some code that controls the generation of those signals. Then you work your way up until finally you have the highest-level word, and you call it go and you type go and everything happens. > I have very little faith in systems analysts who work top-down. They decide what the problem is and then they factor it in such a way that it can be very difficult to implement. https://www.oreilly.com/library/view/masterminds-of-programming/9780596801670/ch04.html https://www.oreilly.com/library/view/masterminds-of-programm...

Jtsummers 3y ago

> I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. It can be, and he even says as much: >> Bottom-up design is possible to a certain degree in languages other than Lisp. Whenever you see library functions, bottom-up design is happening. I'll also point to kazinator's quote of Ken Thompson in the previous discussion linked by dang: >> It is the way I think. I am a very bottom-up thinker. If you give me the right kind of Tinker Toys, I can imagine the building. I can sit there and see primitives and recognize their power to build structures a half mile high, if only I had just one more to make it functionally complete. I can see those kinds of things. The converse is true, too, I think. I can’t— from the building—imagine the Tinker Toys. When I see a top-down description of a system or language that has infinite libraries described by layers and layers, all I just see is a morass. I can’t get a feel for it. I can’t understand how the pieces fit; I can’t understand something presented to me that’s very complex. Maybe I do what I do because if I built anything more complicated, I couldn’t understand it. I really must break it down into little pieces. - Ken Thompson, Unix and Beyond: An Interview with Ken Thompson, 1999 And if you use a TDD style of development (in particular, but test-heavy in general with frequent use of unit and integration tests below the full end-to-end-test level) you'll also likely stumble onto a similar bottom-up style of development. I think, circa 1993, his emphasis on Lisp and bottom-up development made a lot more sense than it does today with the increasing availability of interactive development environments (essentially every dynamically typed language, pretty much; but even many others like Java and C#) and increased emphasis on test-heavy development methodologies.

norir 3y ago

A function call has runtime overhead that a macro may not. There are also things that can be difficult to express as a function in the language type system but that are straightforward to implement as a macro. Lisp isn't the only language with a strong macro system but it disproportionately attracts programmers who value the power and expressivity that a strong macro system provides. There are a number of reasons that bottom up programming in the style PG seems to be advocating for is not popular in industry. Industrial programming favors the median developer while bottom up is a technique that works best with small teams of skilled programmers. Big cos obviously prefer "safe" languages, but even startups are generally conservative due to the vc induced pressure towards hypergrowth, which also favors the median developer since hypergrowth requires hiring. Writing good programs from the bottom up requires more skill[1], but can lead to substantially better programs by certain criteria. But it may also be considered inscrutable by the median developer. The median developer can also misuse the power afforded by the bottom up language and write something much worse than they would have been able to in a more conservative language. This isn't a knock on the median developer, btw, just my perspective on why this style is rarely found in industry. [1] neither LLMs nor stack overflow are of much help when you have essentially written a new language to solve your problem.

TerrifiedMouse 3y ago
> There are a number of reasons that bottom up programming in the style PG seems to be advocating for is not popular in industry. I think another reason is that it’s just more work for another programmer to grok a bottom-up written program written with its own DSL then a program that’s bent to fit the existing (more rigid) language since they already know the existing language.

narag 3y ago

I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. My no. 1 suspect is OOP or, more specifically, languages like Java that won't let you write a function. You can bypass that limitation creating a class whose only reason is to hold stand-alone functions. I don't do that, lately I don't much Java at all, but I've seen huge Util.java classes. In sane languages I used to start exploratory programs with two files: one for the main flow, the other for utility functions. Later the former would become different windows, the latter different libraries. Classes grew slowly, one functionality aspect at a time. Forced OOP nudge you into a priori design the class structure before understanding the domain. Bottom-up advantage is that, having the primitives earlier, you can test with real data from the start and show the results to the domain experts.