3 ms·

This seems similar to the technique popularised by Jai (jblow) and adopted by Zig -- making the compiler programmable in the host language, at compile-time. He

This seems similar to the technique popularised by Jai (jblow) and adopted by Zig -- making the compiler programmable in the host language, at compile-time. Here it seems this is happening at run-time by using an interpreted language. Nevertheless the reasoning they give is similar to why you'd want compile-time to be programmable, that the distinction between compiler and program is somewhat artificial and generally severely hobbles the programmer from being able to introspect their program (etc.). Given how Jai/Zig/etc. work, it now seems crazy that compiler authors have gone to extreme lengths to provide really-bad compile-time languages (called "type systems") that do 5% of what's useful with incomprehensible syntax, rather than just make the compiler itself programmable. This technique makes much of programming language design seem to be fumbling around in the dark.

Ygg2 3y ago
> really-bad compile-time languages (called "type systems") that do 5% of what's useful with incomprehensible syntax, rather than just make the compiler itself programmable. Because some things aren't knowable at compile time. E.g. what is the value I enter at runtime, and all values derived from it, but also state of environment, and so on. Pretending they are the same is a weak point of Zig. By making `const` a property determined by the compiler, you can accidentally introduce breaking changes, just by changing function contents. https://typesanitizer.com/blog/zig-generics.html https://typesanitizer.com/blog/zig-generics.html
- mjburgess 3y ago
  I wasn't talking just about Zig's implementation of comptime, I was talking about collapsing the distinction between compiler and program -- if done well then the problems mentioned in that article need not occur. I think the key feature here is making compilation a program that you write wherein you can fully introspect your program. Worries here about IDE support fadeaway, as the full state of the compiler is available -- if some annotations on code are required to guide IDEs, so be it. I wasnt saying that type systems should be replaced with compile-time programs, only that they are compile time programs, and often really bad ones. (My claim: Programmers shouldnt only have the type system to program the compiler.) Better that the type system be extremely simple and let the programer transform their program as they wish.
  - Ygg2 3y ago
    Sure. But I think my point still applies. Separation of compile vs runtime is as useful as separations between pure and non-pure functions. Pretending they are the same will lose you information. Sounds to me you just want Lisp or Smalltalk (self modifications galore, no difference between compile and runtime, small syntax).
    - manwe150 3y ago
      I think you are making more of a distinction between optimizations and runtime, not compilation. Doing compilation is about taking a lot of arbitrary user-input (in the form of urls and cmake files and code edits) and executing that. Then optimization occurs which is a lot of pure transforms. Then runtime occurs, which is when the program takes a lot of user-input again. But step one usually is in a different language than step three, which is extra effort to learn. And step 2 doesn’t really need to visible to users, since it is pure the user doesn’t really need to directly see the side effects from that.
      - Ygg2 3y ago
        That wasn't my intention if it came out that way. Compilation is something that happens (generally) once and returns an executable artifact. Runtime is execution of said artifact, and can happen many times, with many different environment setups. Compile time functions and types capture stuff that can be known during compilation.
        mjburgess 3y ago
        runtime_result = f(static state, dynamic state)
pjc50 3y ago
Type systems aren't programs because sensible type systems are decidable. They're very specifically not a programming language but a constraint language. https://3fx.ch/typing-is-hard.html https://3fx.ch/typing-is-hard.html ; more people should understand Hindley-Milner. Now, there's also a case for compile-time programmability, but that is a feature that needs to be used carefully because it can make the question of what code is actually running very opaque, as well as arbitrarily increasing compile time. (C++, the bad bits) > making compilation a program that you write wherein you can fully introspect your program I'm experimenting with C# source generators at the moment, and wondering if there are any breakthrough features we could add this way by making it fully generalizable. (They have the limitation that they are additive-only, you can't mutate existing source)
- SigmundA 3y ago
  >They have the limitation that they are additive-only, you can't mutate existing source .Net has System.CodeDom which can load existing source code and mutate it.

Ygg2 3y ago

> really-bad compile-time languages (called "type systems") that do 5% of what's useful with incomprehensible syntax, rather than just make the compiler itself programmable. Because some things aren't knowable at compile time. E.g. what is the value I enter at runtime, and all values derived from it, but also state of environment, and so on. Pretending they are the same is a weak point of Zig. By making `const` a property determined by the compiler, you can accidentally introduce breaking changes, just by changing function contents. https://typesanitizer.com/blog/zig-generics.html https://typesanitizer.com/blog/zig-generics.html

mjburgess 3y ago
I wasn't talking just about Zig's implementation of comptime, I was talking about collapsing the distinction between compiler and program -- if done well then the problems mentioned in that article need not occur. I think the key feature here is making compilation a program that you write wherein you can fully introspect your program. Worries here about IDE support fadeaway, as the full state of the compiler is available -- if some annotations on code are required to guide IDEs, so be it. I wasnt saying that type systems should be replaced with compile-time programs, only that they are compile time programs, and often really bad ones. (My claim: Programmers shouldnt only have the type system to program the compiler.) Better that the type system be extremely simple and let the programer transform their program as they wish.
- Ygg2 3y ago
  Sure. But I think my point still applies. Separation of compile vs runtime is as useful as separations between pure and non-pure functions. Pretending they are the same will lose you information. Sounds to me you just want Lisp or Smalltalk (self modifications galore, no difference between compile and runtime, small syntax).
  - manwe150 3y ago
    I think you are making more of a distinction between optimizations and runtime, not compilation. Doing compilation is about taking a lot of arbitrary user-input (in the form of urls and cmake files and code edits) and executing that. Then optimization occurs which is a lot of pure transforms. Then runtime occurs, which is when the program takes a lot of user-input again. But step one usually is in a different language than step three, which is extra effort to learn. And step 2 doesn’t really need to visible to users, since it is pure the user doesn’t really need to directly see the side effects from that.
    - Ygg2 3y ago
      That wasn't my intention if it came out that way. Compilation is something that happens (generally) once and returns an executable artifact. Runtime is execution of said artifact, and can happen many times, with many different environment setups. Compile time functions and types capture stuff that can be known during compilation.
      - mjburgess 3y ago
        runtime_result = f(static state, dynamic state)

pjc50 3y ago

Type systems aren't programs because sensible type systems are decidable. They're very specifically not a programming language but a constraint language. https://3fx.ch/typing-is-hard.html https://3fx.ch/typing-is-hard.html ; more people should understand Hindley-Milner. Now, there's also a case for compile-time programmability, but that is a feature that needs to be used carefully because it can make the question of what code is actually running very opaque, as well as arbitrarily increasing compile time. (C++, the bad bits) > making compilation a program that you write wherein you can fully introspect your program I'm experimenting with C# source generators at the moment, and wondering if there are any breakthrough features we could add this way by making it fully generalizable. (They have the limitation that they are additive-only, you can't mutate existing source)

SigmundA 3y ago
>They have the limitation that they are additive-only, you can't mutate existing source .Net has System.CodeDom which can load existing source code and mutate it.