5 ms·

Zig compile-time evaluation even allows to construct an arbitrary type like a struct with a platform-specific members all using the same core language. This be

Zig compile-time evaluation even allows to construct an arbitrary type like a struct with a platform-specific members all using the same core language. This beats even Rust macros and in retrospect I wonder why this not used in other languages? Even Lisp ended up with macros with own sub language instead of making the core language more useful for code generation. Or consider Go attempts at generics. Something like Zig approach will fit it more I think than various generics proposals.

haberman 6y ago
I agree it's an idea that seems so natural in retrospect that I'm also curious why it hasn't traditionally been more popular. One possible reason that comes to mind is that an imperatively-constructed type is harder to statically analyze. But on the other hand, even C++ templates are Turing complete and even C++ parse trees can depend on the output of a Turing-complete evaluation. So it is hard to see what benefit a C++-like template feature is buying in comparison.
- fpoling 6y ago
  Even Java and Rust ended up with Turing-complete generics.
  - bobthebuilders 6y ago
    Java turing complete generics are an edge case though, and never hit during normal compilation.
    - fpoling 6y ago
      Still the compiler has to deal with that. So one ends up with an interpreter in the compiler for very esoteric and hard to write/read language instead of just embedding an interpreter of a subset of Java. One alleged benefit of Java constrained genetics is that they allow in theory better error messages. Still in complex cases even after over 15 years with genetics in Java the error messages are not that great. Zig to some extend delegates to the compile-time library authors responsibility of producing good error messages. But then the error messages can be improved without waiting for the compiler to improve its reporting heuristics to cover all edge cases.
      - bobthebuilders 6y ago
        Incorrect Java does not have genetics. That is a C++ concept.
- jeltz 6y ago
  Tow issues I have noticed with Zig: not as good type signatures for functions and not as good error messages. I am not sure if those are fundamental issues with Zig's apporach or if they can be fixed with more work.
  - vram22 6y ago
    Can you elaborate a bit on the first issue?
    - jeltz 6y ago
      Take for example the definition of floor(), how would an IDE or documentation generator see which types are supported? This is just imperative code which is executed at compile time and at least to me it is not obvious how to interpret it when generating the documentation. On the other hand in Rust it is almost always easy to see which types can be used where. pub fn floor(x: anytype) @TypeOf(x) { const T = @TypeOf(x); return switch (T) { f16 => floor16(x), f32 => floor32(x), f64 => floor64(x), f128 => floor128(x), else => @compileError("floor not implemented for " ++ @typeName(T)), }; }
      - haberman 6y ago
        This seems reasonable to me. It seems comparable to the following in C++: template <class T> T floor(T x) { /* ... */ } I think a documentation generator could display your Zig signature verbatim and a user could make sense of that.
- pron 6y ago
  Not only was it not "more popular," I am not aware of even research languages taking the idea of general-purpose partial evaluation not through syntax macros to the same lengths as Zig (although that doesn't mean there weren't any). It's possible that no one had thought of that, perhaps because type systems, while known to be possibly considered as a form of partial evaluation, are generally treated and studied as a completely separate concept: partial evaluation is a compilation technology while types are part of the theory.
  - haberman 6y ago
    > I am not aware of even research languages taking the idea of general-purpose partial evaluation not through syntax macros to the same lengths as Zig One that comes to mind is Terra: http://terralang.org/ http://terralang.org/ In retrospect Terra seems a clear precursor to Zig (though I don't know if it was a direct influence).
    - pron 6y ago
      I've heard of Terra, but I didn't know it had predated Zig by a couple of years; thought they were introduced at about the same time. And yes, there are certainly similarities, although Terra thinks of itself as two interoperating languages, while in Zig there is just one (although there are things you can only do in comptime so perhaps it's more of a difference in perspective).
  - throwaway17_17 6y ago
    I’m not clear on what you define syntax macros, but the compile time evaluation that is present in Zig (based on the example and documentation I’ve seen) are an almost direct implementation of the multistage computation research done late 90s-mid 00s. The papers and such are typically ML based (with a caveat for the early Template Haskell paper in 02) and the underlying constructs of MetaML are very close to what Zig implements. All this work is retroactively semantically underpinned by Rowan Davies and Frank Pfenning work on Modal S4 logic. I don’t know if Andy based Zig on this research, but if he didn’t, the convergence toward the same theory is a great thing to see. My current work on a visual programming compatible language also uses this strategy for meta programming, so I’m very familiar with the academic literature. It certainly seems that Zig got this right and is doing well implementing it in a way that is usable and understandable.
afranchuk 6y ago
I would guess one reason it wasn't done in other languages was because people simply didn't have a good cross-architecture JIT compilation resource, and didn't want to write it themselves. LLVM makes this _really_ easy. I realize that Zig is transitioning to an LLVM-optional model now. But, for instance, I've been working on an s-expr language with f-expr application, and this combined with LLVM c bindings allows you to generate arbitrary compile time or runtime code. The JIT portion for compile time code was a single function call in LLVM! I started this a while before Zig came out, but alas I haven't devoted enough time to it over the years...
gmfawcett 6y ago
"Even Lisp ended up with macros with own sub language..." I assume you're talking about Scheme approaches like `syntax-case` and `syntax-rules`? In Common Lisp, macros are written in Lisp, the same way you would write runtime code. Unquoting and splicing are part of the core language (e.g. they can be applied to data as well as to code).
- fpoling 6y ago
  I was not aware that unquoting in Common Lisp can be applied to data. So it is not a specialized sub language just for macros, but a general purpose template system that could also generate code.
  - kazinator 6y ago
    That's right. There is no difference between: (defmacro my-quote (x) `(quote ,x)) and (defmacro my-quote (x) (list 'quote x)) That `(quote ,x) is just way of writing (list 'quote x), not a notation for writing macros. That's it.
  - ywei3410 6y ago
    It's really a shorthand for writing lists - sexp's are lists, so can use the same syntax. Macro's in Lisp are essentially "just" functions where the arguments are un-evaluated - you can use all the same functions etc... Writing a simple Lisp interpreter is really quite educational - when you add macros and quoting into the language, you suddenly have a realisation at how simple it is.
mhh__ 6y ago
We have a slightly vague proposal in D to basically construct programs as a library at compile time (i.e. we give you an AST, you play with it and give it back, if it's valid it.gets turned into a type). D has been making structs at compile time for years now, however.
JacobCarlborg 6y ago
This is very common in D and D doesn't not have any macro system. It uses regular syntax (more or less). D was doing this way before Zig existed and before C++ had constexpr. Simple example of platform specific members: struct Socket { version (Posix) int handle; else version (Windows) SOCKET handle; else static assert(false, "Unsupported platform"); } Another example is the checked numeric type [1] in the D standard library. It takes two types as parameters. The first being the underlying type and the second being a "hook" type. The hook type allows the user to decide what should happen in various error conditions, like overflow, divide by zero and so on. The implementation is written in a Design By Introspection style and inspects the hook type and adopts its implementation depending on what hooks it provides. If the hook type implements the hook for overflow, that hook will be executed, otherwise it falls back to some default behavior. [1] https://dlang.org/phobos/std_experimental_checkedint.html https://dlang.org/phobos/std_experimental_checkedint.html

haberman 6y ago

I agree it's an idea that seems so natural in retrospect that I'm also curious why it hasn't traditionally been more popular. One possible reason that comes to mind is that an imperatively-constructed type is harder to statically analyze. But on the other hand, even C++ templates are Turing complete and even C++ parse trees can depend on the output of a Turing-complete evaluation. So it is hard to see what benefit a C++-like template feature is buying in comparison.

fpoling 6y ago
Even Java and Rust ended up with Turing-complete generics.
- bobthebuilders 6y ago
  Java turing complete generics are an edge case though, and never hit during normal compilation.
  - fpoling 6y ago
    Still the compiler has to deal with that. So one ends up with an interpreter in the compiler for very esoteric and hard to write/read language instead of just embedding an interpreter of a subset of Java. One alleged benefit of Java constrained genetics is that they allow in theory better error messages. Still in complex cases even after over 15 years with genetics in Java the error messages are not that great. Zig to some extend delegates to the compile-time library authors responsibility of producing good error messages. But then the error messages can be improved without waiting for the compiler to improve its reporting heuristics to cover all edge cases.
    - bobthebuilders 6y ago
      Incorrect Java does not have genetics. That is a C++ concept.
jeltz 6y ago
Tow issues I have noticed with Zig: not as good type signatures for functions and not as good error messages. I am not sure if those are fundamental issues with Zig's apporach or if they can be fixed with more work.
- vram22 6y ago
  Can you elaborate a bit on the first issue?
  - jeltz 6y ago
    Take for example the definition of floor(), how would an IDE or documentation generator see which types are supported? This is just imperative code which is executed at compile time and at least to me it is not obvious how to interpret it when generating the documentation. On the other hand in Rust it is almost always easy to see which types can be used where. pub fn floor(x: anytype) @TypeOf(x) { const T = @TypeOf(x); return switch (T) { f16 => floor16(x), f32 => floor32(x), f64 => floor64(x), f128 => floor128(x), else => @compileError("floor not implemented for " ++ @typeName(T)), }; }
    - haberman 6y ago
      This seems reasonable to me. It seems comparable to the following in C++: template <class T> T floor(T x) { /* ... */ } I think a documentation generator could display your Zig signature verbatim and a user could make sense of that.
pron 6y ago
Not only was it not "more popular," I am not aware of even research languages taking the idea of general-purpose partial evaluation not through syntax macros to the same lengths as Zig (although that doesn't mean there weren't any). It's possible that no one had thought of that, perhaps because type systems, while known to be possibly considered as a form of partial evaluation, are generally treated and studied as a completely separate concept: partial evaluation is a compilation technology while types are part of the theory.
- haberman 6y ago
  > I am not aware of even research languages taking the idea of general-purpose partial evaluation not through syntax macros to the same lengths as Zig One that comes to mind is Terra: http://terralang.org/ http://terralang.org/ In retrospect Terra seems a clear precursor to Zig (though I don't know if it was a direct influence).
  - pron 6y ago
    I've heard of Terra, but I didn't know it had predated Zig by a couple of years; thought they were introduced at about the same time. And yes, there are certainly similarities, although Terra thinks of itself as two interoperating languages, while in Zig there is just one (although there are things you can only do in comptime so perhaps it's more of a difference in perspective).
- throwaway17_17 6y ago
  I’m not clear on what you define syntax macros, but the compile time evaluation that is present in Zig (based on the example and documentation I’ve seen) are an almost direct implementation of the multistage computation research done late 90s-mid 00s. The papers and such are typically ML based (with a caveat for the early Template Haskell paper in 02) and the underlying constructs of MetaML are very close to what Zig implements. All this work is retroactively semantically underpinned by Rowan Davies and Frank Pfenning work on Modal S4 logic. I don’t know if Andy based Zig on this research, but if he didn’t, the convergence toward the same theory is a great thing to see. My current work on a visual programming compatible language also uses this strategy for meta programming, so I’m very familiar with the academic literature. It certainly seems that Zig got this right and is doing well implementing it in a way that is usable and understandable.

afranchuk 6y ago

I would guess one reason it wasn't done in other languages was because people simply didn't have a good cross-architecture JIT compilation resource, and didn't want to write it themselves. LLVM makes this _really_ easy. I realize that Zig is transitioning to an LLVM-optional model now. But, for instance, I've been working on an s-expr language with f-expr application, and this combined with LLVM c bindings allows you to generate arbitrary compile time or runtime code. The JIT portion for compile time code was a single function call in LLVM! I started this a while before Zig came out, but alas I haven't devoted enough time to it over the years...

gmfawcett 6y ago

"Even Lisp ended up with macros with own sub language..." I assume you're talking about Scheme approaches like `syntax-case` and `syntax-rules`? In Common Lisp, macros are written in Lisp, the same way you would write runtime code. Unquoting and splicing are part of the core language (e.g. they can be applied to data as well as to code).

fpoling 6y ago
I was not aware that unquoting in Common Lisp can be applied to data. So it is not a specialized sub language just for macros, but a general purpose template system that could also generate code.
- kazinator 6y ago
  That's right. There is no difference between: (defmacro my-quote (x) `(quote ,x)) and (defmacro my-quote (x) (list 'quote x)) That `(quote ,x) is just way of writing (list 'quote x), not a notation for writing macros. That's it.
- ywei3410 6y ago
  It's really a shorthand for writing lists - sexp's are lists, so can use the same syntax. Macro's in Lisp are essentially "just" functions where the arguments are un-evaluated - you can use all the same functions etc... Writing a simple Lisp interpreter is really quite educational - when you add macros and quoting into the language, you suddenly have a realisation at how simple it is.

mhh__ 6y ago

We have a slightly vague proposal in D to basically construct programs as a library at compile time (i.e. we give you an AST, you play with it and give it back, if it's valid it.gets turned into a type). D has been making structs at compile time for years now, however.

JacobCarlborg 6y ago

This is very common in D and D doesn't not have any macro system. It uses regular syntax (more or less). D was doing this way before Zig existed and before C++ had constexpr. Simple example of platform specific members: struct Socket { version (Posix) int handle; else version (Windows) SOCKET handle; else static assert(false, "Unsupported platform"); } Another example is the checked numeric type [1] in the D standard library. It takes two types as parameters. The first being the underlying type and the second being a "hook" type. The hook type allows the user to decide what should happen in various error conditions, like overflow, divide by zero and so on. The implementation is written in a Design By Introspection style and inspects the hook type and adopts its implementation depending on what hooks it provides. If the hook type implements the hook for overflow, that hook will be executed, otherwise it falls back to some default behavior. [1] https://dlang.org/phobos/std_experimental_checkedint.html https://dlang.org/phobos/std_experimental_checkedint.html