4 ms·

> Scheme isn’t special because it has closures, it is special because you can write functions that write functions. I swear this informational parasite will ne

by Spivak 2y ago

> Scheme isn’t special because it has closures, it is special because you can write functions that write functions. I swear this informational parasite will never die, any language with eval can write functions that write functions. What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure and it works.

hollerith 2y ago
>What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure I think I know Scheme well, and I don't catch your meaning. Most Scheme implementations are compilers that output a machine-language executable. How can this executable be "manipulated live" in a way that differs from an executable produced by any other language? What am I not seeing?
- Spivak 2y ago
  At the level where you're executing assembly every language is the same. Which is a way of saying that assembly is homoiconic. And even in C you're more than capable of W&Xing your binary to change behavior at runtime but you're doing it at a level below the language. It's the (not completely unrestricted) self-modifying bit that's cool and unique. You can go into an existing function and change what it does by manipulating or outright replacing its code. Other languages you end up having to evaluate and re-point/redefine the reference and it gets you where you need to go but not nearly as cool. For folks that liked Dijkstra's whole structured programming thing but didn't want to give up the power entirely.
  - PaulHoule 2y ago
    How about monkey patching in languages like Javascript and Python? I'd also say "homoiconic" is a less important concept than people would think. A while back I was writing a lodash-like library for Java because I don't like the Streams API (though in the end I just used the collectors from the Stream API because they were easy to fit into my system) One thing I found frustrating about that was that I didn't want to write 8 different functions for the 8 primitive types (never mind 64 functions for the direct product of the primitive types) and conceived of a code generator that would let you write Java-in-Java with a somewhat Lispy looking API that builds up something like S-expressions with typed static imports, here is some sample code https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/test/java/com/ontology2/ferocity/TestUrFierce.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... The plan was to write a ferocity0 sufficient to write a code generator that could generate stubs for the standard library plus a persistent collection library, then write most of ferocity1 in ferocity0 and maybe write a ferocity2 in ferocity1 and ferocity2 would be the master tool for code generation in Java. Look how nice the stub generator is https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/main/java/com/ontology2/ferocity/WrapperGenerator.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... Now many people would think this combines all the drawbacks of Common Lisp and Java and they might be right. On the other hand, I think if you're aggressive in organizing your code and using code generation you might cut your code volume enough to make up for the bulking that happens when you write Java-in-Java. The stubs to write Java-in-Java are a huge amount of code but they're 99% generated and once you have Java-in-Java you can keep recursing and messing around with quote and eval and do all the stuff you'd do in a homoiconic language except you are having to write code in a DSL.
    - kazinator 2y ago
      Now suppose three of you are working on the same project, and each brings their own code generator like this. You can take Common Lisp macros from multiple sources and use them in the same expression, in any pattern of nesting.
      - PaulHoule 2y ago
        In principle. Most people find somebody else's metaprogramming-heavy Common Lisp difficult to work with. When you look at classic programs from the golden age of AI you find they either rewrote their Lisp prototype in C++ for speed or the Lisp program (like an expert system shell) is much smaller than you imagine possible but it looks like the engine of a UFO.
        aeonik 2y ago
        Do you have a link to one of these UFO engines that I can study up on?
        PaulHoule 2y ago
        See https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html and the more modern http://www.clara-rules.org/ http://www.clara-rules.org/
        deleted 2y ago
        [deleted]
        kazinator 2y ago
        That code looks readable to me. It is well formatted and the functions are small, with simple control paths. No variable assignments or loops to unravel. It defines no macros, but refers to some defined elsewhere; the instructions say you are supposed to use a certain other source file related to Chapter 14 of a book. I could probably ramp up on this in a day, if I had reason to.
  - hollerith 2y ago
    >You can go into an existing function and change what it does by manipulating or outright replacing its code. I still don't see what this might mean. It doesn't sound like anything I've ever done in Scheme or I've ever heard of anyone else doing. You probably don't mean editing Scheme source code and recompiling.
- trealira 2y ago
  > the code that is currently executing can be manipulated live Typically when people say this about Lisp, they mean that they can redefine functions and objects live, having all the code adapt to use the new definitions, or when their code hits an error, they can not only redefine functions and objects live, but also examine the state of the call stack and change the values of variables live, and then let the code run again. This[0] is a clear explanation that I found. There's also a video[1] that shows the same thing. However, I don't know if those apply to Scheme. Also, I am pretty sure it is unrelated to Lisp being homoiconic, and that it's more about it being dynamically typed and interpretable. [0]: https://malisper.me/debugging-lisp-part-1-recompilation/ https://malisper.me/debugging-lisp-part-1-recompilation/ [1]: https://youtu.be/jBBS4FeY7XM https://youtu.be/jBBS4FeY7XM
  - hollerith 2y ago
    But that can't be "what makes Scheme and other homoiconic languages special".
    - trealira 2y ago
      I suppose that, if I'm asked "What is something that makes homoiconic languages special?" and the answer can't be that it makes macros easy, then I would say it's that it's easy to write an interpreter for that language in the language itself, because the data structures that form the syntactic structure of the language are themselves first-class objects. That's why the interpreter from the appendix of the LISP 1.5 manual can be so short. That's also why the EVAL function takes as a parameter a list of atoms, and not a string, as in non-Lisp dynamically typed languages; since a Lisp program is also a list of atoms, that's the form it takes in the interpreter. The first Lisp compiler [0] isn't so long, either. The equivalent eval function in non-homoiconic languages requires a lexer, parser, and special definitions of data structures to represent the syntax, as well as functions to manipulate those data structures. [0]: https://texdraft.github.io/lisp-compiler/internals.html https://texdraft.github.io/lisp-compiler/internals.html
      - hollerith 2y ago
PaulHoule 2y ago
Back in the 1980s I found a book at the public library where somebody had a BASIC program that would read a BASIC program without line numbers and with structured programming loops and rewrite it an ordinary BASIC. It was oriented towards CP/M so I typed the program in and modified it to run on my TRS-80 Coco. It was kinda like https://en.wikipedia.org/wiki/Ratfor https://en.wikipedia.org/wiki/Ratfor Similarly I've written awk programs that write shell scripts (guilty pleasure), Python programs that write AVR-8 assembly, toolkits to write Java ASTs with static imports like @Test public void buildAList() { var stream = callBuild(callAdd(callAdd(callAdd(callBuilder(), Literal.of(7)), Literal.of(11)),Literal.of(15))); var expr = callToList(stream); var l = expr.evaluateRT(); assertEquals(3,l.size()); assertEquals(7, l.get(0)); assertEquals(11, l.get(1)); assertEquals(15, l.get(2)); } and evaluate with a tree-walking interpreter or use as a code generator in maven, do transformations on the ASTs, etc. There are many paths to metaprogramming but the Lisp family leads in putting it on your fingertips. Once you learn those methods you can apply them in the trashiest programming languages, but I think often people miss the point.
- touisteur 2y ago
  Very common for me to write shell scripts that generate shell scripts to run. Instead of creating gnuplot scripts to loaded, sometimes a good sequence of echo "set terminal png 800,600; plot..." | gnuplot > my.png or a very long imagemagick drawing command or a simple svg generated then converted to png then assembled (thank montage), can do the job. The whole advantage of generating intermediate shell scripts, is to replay one part specifically to debug a small part of a script pipeline.
andai 2y ago
>code that is currently executing can be manipulated live like any other data structure and it works. I heard from someone who did this with x86(-64?) assembly in production code, but I don't know the details.
- deleted 2y ago
  [deleted]
lispm 2y ago
> What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure and it works. That would be true for source level interpreters, but not for compiled implementations.
kazinator 2y ago
No, Scheme does not make self-modifying code defined. See this example in R7RS for the set-car! function: (define (f) (list 'not-a-constant-list)) (define (g) '(constant-list)) (set-car! (f) 3) ⟹ unspecified (set-car! (g) 3) ⟹ error It is an error in Scheme to modify a constant list. The implementation is not required to diagnose it (an exception is not required), which basically means that it's undefined behavior. In Common Lisp, it is likewise undefined behavior. Running Lisp code is not necessarily still made of nested lists. That may be the case if it is interpreted, but Lisp code is often compiled. Modifying the original nested lists that comprise the source code will do nothing to the object code, except if those parts are modified that serve as literals, which the compiled code may be sharing with the source code. In short, your understanding is wrong. When Lisp people talking about modifying the running program, that just refers to redefining functions and other entities. Common Lisp has strong support for redefining classes of which there are existing instances. Existing objects can be migrated to new representations in the redefined classes. None of that is related to homoiconicity or related topics.

by Spivak 2y ago

hollerith 2y ago

>What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure I think I know Scheme well, and I don't catch your meaning. Most Scheme implementations are compilers that output a machine-language executable. How can this executable be "manipulated live" in a way that differs from an executable produced by any other language? What am I not seeing?

Spivak 2y ago
At the level where you're executing assembly every language is the same. Which is a way of saying that assembly is homoiconic. And even in C you're more than capable of W&Xing your binary to change behavior at runtime but you're doing it at a level below the language. It's the (not completely unrestricted) self-modifying bit that's cool and unique. You can go into an existing function and change what it does by manipulating or outright replacing its code. Other languages you end up having to evaluate and re-point/redefine the reference and it gets you where you need to go but not nearly as cool. For folks that liked Dijkstra's whole structured programming thing but didn't want to give up the power entirely.
- PaulHoule 2y ago
  How about monkey patching in languages like Javascript and Python? I'd also say "homoiconic" is a less important concept than people would think. A while back I was writing a lodash-like library for Java because I don't like the Streams API (though in the end I just used the collectors from the Stream API because they were easy to fit into my system) One thing I found frustrating about that was that I didn't want to write 8 different functions for the 8 primitive types (never mind 64 functions for the direct product of the primitive types) and conceived of a code generator that would let you write Java-in-Java with a somewhat Lispy looking API that builds up something like S-expressions with typed static imports, here is some sample code https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/test/java/com/ontology2/ferocity/TestUrFierce.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... The plan was to write a ferocity0 sufficient to write a code generator that could generate stubs for the standard library plus a persistent collection library, then write most of ferocity1 in ferocity0 and maybe write a ferocity2 in ferocity1 and ferocity2 would be the master tool for code generation in Java. Look how nice the stub generator is https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/main/java/com/ontology2/ferocity/WrapperGenerator.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... Now many people would think this combines all the drawbacks of Common Lisp and Java and they might be right. On the other hand, I think if you're aggressive in organizing your code and using code generation you might cut your code volume enough to make up for the bulking that happens when you write Java-in-Java. The stubs to write Java-in-Java are a huge amount of code but they're 99% generated and once you have Java-in-Java you can keep recursing and messing around with quote and eval and do all the stuff you'd do in a homoiconic language except you are having to write code in a DSL.
  - kazinator 2y ago
    Now suppose three of you are working on the same project, and each brings their own code generator like this. You can take Common Lisp macros from multiple sources and use them in the same expression, in any pattern of nesting.
    - PaulHoule 2y ago
      In principle. Most people find somebody else's metaprogramming-heavy Common Lisp difficult to work with. When you look at classic programs from the golden age of AI you find they either rewrote their Lisp prototype in C++ for speed or the Lisp program (like an expert system shell) is much smaller than you imagine possible but it looks like the engine of a UFO.
      - aeonik 2y ago
        Do you have a link to one of these UFO engines that I can study up on?
        PaulHoule 2y ago
        See https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html and the more modern http://www.clara-rules.org/ http://www.clara-rules.org/
        deleted 2y ago
        [deleted]
        kazinator 2y ago
        That code looks readable to me. It is well formatted and the functions are small, with simple control paths. No variable assignments or loops to unravel. It defines no macros, but refers to some defined elsewhere; the instructions say you are supposed to use a certain other source file related to Chapter 14 of a book. I could probably ramp up on this in a day, if I had reason to.
- hollerith 2y ago
  >You can go into an existing function and change what it does by manipulating or outright replacing its code. I still don't see what this might mean. It doesn't sound like anything I've ever done in Scheme or I've ever heard of anyone else doing. You probably don't mean editing Scheme source code and recompiling.
trealira 2y ago
> the code that is currently executing can be manipulated live Typically when people say this about Lisp, they mean that they can redefine functions and objects live, having all the code adapt to use the new definitions, or when their code hits an error, they can not only redefine functions and objects live, but also examine the state of the call stack and change the values of variables live, and then let the code run again. This[0] is a clear explanation that I found. There's also a video[1] that shows the same thing. However, I don't know if those apply to Scheme. Also, I am pretty sure it is unrelated to Lisp being homoiconic, and that it's more about it being dynamically typed and interpretable. [0]: https://malisper.me/debugging-lisp-part-1-recompilation/ https://malisper.me/debugging-lisp-part-1-recompilation/ [1]: https://youtu.be/jBBS4FeY7XM https://youtu.be/jBBS4FeY7XM
- hollerith 2y ago
  But that can't be "what makes Scheme and other homoiconic languages special".
  - trealira 2y ago
    I suppose that, if I'm asked "What is something that makes homoiconic languages special?" and the answer can't be that it makes macros easy, then I would say it's that it's easy to write an interpreter for that language in the language itself, because the data structures that form the syntactic structure of the language are themselves first-class objects. That's why the interpreter from the appendix of the LISP 1.5 manual can be so short. That's also why the EVAL function takes as a parameter a list of atoms, and not a string, as in non-Lisp dynamically typed languages; since a Lisp program is also a list of atoms, that's the form it takes in the interpreter. The first Lisp compiler [0] isn't so long, either. The equivalent eval function in non-homoiconic languages requires a lexer, parser, and special definitions of data structures to represent the syntax, as well as functions to manipulate those data structures. [0]: https://texdraft.github.io/lisp-compiler/internals.html https://texdraft.github.io/lisp-compiler/internals.html
    - hollerith 2y ago

PaulHoule 2y ago

Back in the 1980s I found a book at the public library where somebody had a BASIC program that would read a BASIC program without line numbers and with structured programming loops and rewrite it an ordinary BASIC. It was oriented towards CP/M so I typed the program in and modified it to run on my TRS-80 Coco. It was kinda like https://en.wikipedia.org/wiki/Ratfor https://en.wikipedia.org/wiki/Ratfor Similarly I've written awk programs that write shell scripts (guilty pleasure), Python programs that write AVR-8 assembly, toolkits to write Java ASTs with static imports like @Test public void buildAList() { var stream = callBuild(callAdd(callAdd(callAdd(callBuilder(), Literal.of(7)), Literal.of(11)),Literal.of(15))); var expr = callToList(stream); var l = expr.evaluateRT(); assertEquals(3,l.size()); assertEquals(7, l.get(0)); assertEquals(11, l.get(1)); assertEquals(15, l.get(2)); } and evaluate with a tree-walking interpreter or use as a code generator in maven, do transformations on the ASTs, etc. There are many paths to metaprogramming but the Lisp family leads in putting it on your fingertips. Once you learn those methods you can apply them in the trashiest programming languages, but I think often people miss the point.

touisteur 2y ago
Very common for me to write shell scripts that generate shell scripts to run. Instead of creating gnuplot scripts to loaded, sometimes a good sequence of echo "set terminal png 800,600; plot..." | gnuplot > my.png or a very long imagemagick drawing command or a simple svg generated then converted to png then assembled (thank montage), can do the job. The whole advantage of generating intermediate shell scripts, is to replay one part specifically to debug a small part of a script pipeline.

andai 2y ago

>code that is currently executing can be manipulated live like any other data structure and it works. I heard from someone who did this with x86(-64?) assembly in production code, but I don't know the details.

deleted 2y ago
[deleted]

lispm 2y ago

> What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure and it works. That would be true for source level interpreters, but not for compiled implementations.

kazinator 2y ago

No, Scheme does not make self-modifying code defined. See this example in R7RS for the set-car! function: (define (f) (list 'not-a-constant-list)) (define (g) '(constant-list)) (set-car! (f) 3) ⟹ unspecified (set-car! (g) 3) ⟹ error It is an error in Scheme to modify a constant list. The implementation is not required to diagnose it (an exception is not required), which basically means that it's undefined behavior. In Common Lisp, it is likewise undefined behavior. Running Lisp code is not necessarily still made of nested lists. That may be the case if it is interpreted, but Lisp code is often compiled. Modifying the original nested lists that comprise the source code will do nothing to the object code, except if those parts are modified that serve as literals, which the compiled code may be sharing with the source code. In short, your understanding is wrong. When Lisp people talking about modifying the running program, that just refers to redefining functions and other entities. Common Lisp has strong support for redefining classes of which there are existing instances. Existing objects can be migrated to new representations in the redefined classes. None of that is related to homoiconicity or related topics.