5 ms·

>What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure I

>What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure I think I know Scheme well, and I don't catch your meaning. Most Scheme implementations are compilers that output a machine-language executable. How can this executable be "manipulated live" in a way that differs from an executable produced by any other language? What am I not seeing?

Spivak 2y ago
At the level where you're executing assembly every language is the same. Which is a way of saying that assembly is homoiconic. And even in C you're more than capable of W&Xing your binary to change behavior at runtime but you're doing it at a level below the language. It's the (not completely unrestricted) self-modifying bit that's cool and unique. You can go into an existing function and change what it does by manipulating or outright replacing its code. Other languages you end up having to evaluate and re-point/redefine the reference and it gets you where you need to go but not nearly as cool. For folks that liked Dijkstra's whole structured programming thing but didn't want to give up the power entirely.
- PaulHoule 2y ago
  How about monkey patching in languages like Javascript and Python? I'd also say "homoiconic" is a less important concept than people would think. A while back I was writing a lodash-like library for Java because I don't like the Streams API (though in the end I just used the collectors from the Stream API because they were easy to fit into my system) One thing I found frustrating about that was that I didn't want to write 8 different functions for the 8 primitive types (never mind 64 functions for the direct product of the primitive types) and conceived of a code generator that would let you write Java-in-Java with a somewhat Lispy looking API that builds up something like S-expressions with typed static imports, here is some sample code https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/test/java/com/ontology2/ferocity/TestUrFierce.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... The plan was to write a ferocity0 sufficient to write a code generator that could generate stubs for the standard library plus a persistent collection library, then write most of ferocity1 in ferocity0 and maybe write a ferocity2 in ferocity1 and ferocity2 would be the master tool for code generation in Java. Look how nice the stub generator is https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/main/java/com/ontology2/ferocity/WrapperGenerator.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... Now many people would think this combines all the drawbacks of Common Lisp and Java and they might be right. On the other hand, I think if you're aggressive in organizing your code and using code generation you might cut your code volume enough to make up for the bulking that happens when you write Java-in-Java. The stubs to write Java-in-Java are a huge amount of code but they're 99% generated and once you have Java-in-Java you can keep recursing and messing around with quote and eval and do all the stuff you'd do in a homoiconic language except you are having to write code in a DSL.
  - kazinator 2y ago
    Now suppose three of you are working on the same project, and each brings their own code generator like this. You can take Common Lisp macros from multiple sources and use them in the same expression, in any pattern of nesting.
    - PaulHoule 2y ago
      In principle. Most people find somebody else's metaprogramming-heavy Common Lisp difficult to work with. When you look at classic programs from the golden age of AI you find they either rewrote their Lisp prototype in C++ for speed or the Lisp program (like an expert system shell) is much smaller than you imagine possible but it looks like the engine of a UFO.
      - aeonik 2y ago
        Do you have a link to one of these UFO engines that I can study up on?
        PaulHoule 2y ago
        See https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html and the more modern http://www.clara-rules.org/ http://www.clara-rules.org/
        deleted 2y ago
        [deleted]
        kazinator 2y ago
        That code looks readable to me. It is well formatted and the functions are small, with simple control paths. No variable assignments or loops to unravel. It defines no macros, but refers to some defined elsewhere; the instructions say you are supposed to use a certain other source file related to Chapter 14 of a book. I could probably ramp up on this in a day, if I had reason to.
- hollerith 2y ago
  >You can go into an existing function and change what it does by manipulating or outright replacing its code. I still don't see what this might mean. It doesn't sound like anything I've ever done in Scheme or I've ever heard of anyone else doing. You probably don't mean editing Scheme source code and recompiling.
trealira 2y ago
> the code that is currently executing can be manipulated live Typically when people say this about Lisp, they mean that they can redefine functions and objects live, having all the code adapt to use the new definitions, or when their code hits an error, they can not only redefine functions and objects live, but also examine the state of the call stack and change the values of variables live, and then let the code run again. This[0] is a clear explanation that I found. There's also a video[1] that shows the same thing. However, I don't know if those apply to Scheme. Also, I am pretty sure it is unrelated to Lisp being homoiconic, and that it's more about it being dynamically typed and interpretable. [0]: https://malisper.me/debugging-lisp-part-1-recompilation/ https://malisper.me/debugging-lisp-part-1-recompilation/ [1]: https://youtu.be/jBBS4FeY7XM https://youtu.be/jBBS4FeY7XM
- hollerith 2y ago
  But that can't be "what makes Scheme and other homoiconic languages special".
  - trealira 2y ago
    I suppose that, if I'm asked "What is something that makes homoiconic languages special?" and the answer can't be that it makes macros easy, then I would say it's that it's easy to write an interpreter for that language in the language itself, because the data structures that form the syntactic structure of the language are themselves first-class objects. That's why the interpreter from the appendix of the LISP 1.5 manual can be so short. That's also why the EVAL function takes as a parameter a list of atoms, and not a string, as in non-Lisp dynamically typed languages; since a Lisp program is also a list of atoms, that's the form it takes in the interpreter. The first Lisp compiler [0] isn't so long, either. The equivalent eval function in non-homoiconic languages requires a lexer, parser, and special definitions of data structures to represent the syntax, as well as functions to manipulate those data structures. [0]: https://texdraft.github.io/lisp-compiler/internals.html https://texdraft.github.io/lisp-compiler/internals.html
    - hollerith 2y ago

Spivak 2y ago

At the level where you're executing assembly every language is the same. Which is a way of saying that assembly is homoiconic. And even in C you're more than capable of W&Xing your binary to change behavior at runtime but you're doing it at a level below the language. It's the (not completely unrestricted) self-modifying bit that's cool and unique. You can go into an existing function and change what it does by manipulating or outright replacing its code. Other languages you end up having to evaluate and re-point/redefine the reference and it gets you where you need to go but not nearly as cool. For folks that liked Dijkstra's whole structured programming thing but didn't want to give up the power entirely.

PaulHoule 2y ago
How about monkey patching in languages like Javascript and Python? I'd also say "homoiconic" is a less important concept than people would think. A while back I was writing a lodash-like library for Java because I don't like the Streams API (though in the end I just used the collectors from the Stream API because they were easy to fit into my system) One thing I found frustrating about that was that I didn't want to write 8 different functions for the 8 primitive types (never mind 64 functions for the direct product of the primitive types) and conceived of a code generator that would let you write Java-in-Java with a somewhat Lispy looking API that builds up something like S-expressions with typed static imports, here is some sample code https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/test/java/com/ontology2/ferocity/TestUrFierce.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... The plan was to write a ferocity0 sufficient to write a code generator that could generate stubs for the standard library plus a persistent collection library, then write most of ferocity1 in ferocity0 and maybe write a ferocity2 in ferocity1 and ferocity2 would be the master tool for code generation in Java. Look how nice the stub generator is https://github.com/paulhoule/ferocity/blob/main/ferocity0/src/main/java/com/ontology2/ferocity/WrapperGenerator.java https://github.com/paulhoule/ferocity/blob/main/ferocity0/sr... Now many people would think this combines all the drawbacks of Common Lisp and Java and they might be right. On the other hand, I think if you're aggressive in organizing your code and using code generation you might cut your code volume enough to make up for the bulking that happens when you write Java-in-Java. The stubs to write Java-in-Java are a huge amount of code but they're 99% generated and once you have Java-in-Java you can keep recursing and messing around with quote and eval and do all the stuff you'd do in a homoiconic language except you are having to write code in a DSL.
- kazinator 2y ago
  Now suppose three of you are working on the same project, and each brings their own code generator like this. You can take Common Lisp macros from multiple sources and use them in the same expression, in any pattern of nesting.
  - PaulHoule 2y ago
    In principle. Most people find somebody else's metaprogramming-heavy Common Lisp difficult to work with. When you look at classic programs from the golden age of AI you find they either rewrote their Lisp prototype in C++ for speed or the Lisp program (like an expert system shell) is much smaller than you imagine possible but it looks like the engine of a UFO.
    - aeonik 2y ago
      Do you have a link to one of these UFO engines that I can study up on?
      - PaulHoule 2y ago
        See https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html https://www.cs.unm.edu/~luger/ai-final/code/LISP.ess.html and the more modern http://www.clara-rules.org/ http://www.clara-rules.org/
        deleted 2y ago
        [deleted]
        kazinator 2y ago
        That code looks readable to me. It is well formatted and the functions are small, with simple control paths. No variable assignments or loops to unravel. It defines no macros, but refers to some defined elsewhere; the instructions say you are supposed to use a certain other source file related to Chapter 14 of a book. I could probably ramp up on this in a day, if I had reason to.
hollerith 2y ago
>You can go into an existing function and change what it does by manipulating or outright replacing its code. I still don't see what this might mean. It doesn't sound like anything I've ever done in Scheme or I've ever heard of anyone else doing. You probably don't mean editing Scheme source code and recompiling.

trealira 2y ago

> the code that is currently executing can be manipulated live Typically when people say this about Lisp, they mean that they can redefine functions and objects live, having all the code adapt to use the new definitions, or when their code hits an error, they can not only redefine functions and objects live, but also examine the state of the call stack and change the values of variables live, and then let the code run again. This[0] is a clear explanation that I found. There's also a video[1] that shows the same thing. However, I don't know if those apply to Scheme. Also, I am pretty sure it is unrelated to Lisp being homoiconic, and that it's more about it being dynamically typed and interpretable. [0]: https://malisper.me/debugging-lisp-part-1-recompilation/ https://malisper.me/debugging-lisp-part-1-recompilation/ [1]: https://youtu.be/jBBS4FeY7XM https://youtu.be/jBBS4FeY7XM

hollerith 2y ago
But that can't be "what makes Scheme and other homoiconic languages special".
- trealira 2y ago
  I suppose that, if I'm asked "What is something that makes homoiconic languages special?" and the answer can't be that it makes macros easy, then I would say it's that it's easy to write an interpreter for that language in the language itself, because the data structures that form the syntactic structure of the language are themselves first-class objects. That's why the interpreter from the appendix of the LISP 1.5 manual can be so short. That's also why the EVAL function takes as a parameter a list of atoms, and not a string, as in non-Lisp dynamically typed languages; since a Lisp program is also a list of atoms, that's the form it takes in the interpreter. The first Lisp compiler [0] isn't so long, either. The equivalent eval function in non-homoiconic languages requires a lexer, parser, and special definitions of data structures to represent the syntax, as well as functions to manipulate those data structures. [0]: https://texdraft.github.io/lisp-compiler/internals.html https://texdraft.github.io/lisp-compiler/internals.html
  - hollerith 2y ago