6 ms·

> I don't understand the hype about Zig? As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating,

by pron 8d ago

> I don't understand the hype about Zig? As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating, and I also think it addresses a long-standing problem in low-level programming. I'll get to the problem later, but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. It's never been done before (other languages have partial evaluation mechanisms that are almost as general, but they're offered in addition to, not as a replacement of, other features). > rust mostly-solved the memory management problem at compile time and without a GC "Mostly" does a lot of work here because 1., if you look at the implementation of very efficient, possibly specialised data structures - the very thing you reach for a low-level language for - they typically require unsafe, and 2., it still suffers from the problem C++ has had for decades, which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower (huge runtimes like TCMalloc help, but not enough, because they can't move pointers). This problem, of programs that start out fast, but after five or ten years of evolution need to spend a lot of effort to remain fast, is one of the things moving collectors were designed to solve, but they require moving pointers, which doesn't work in low-level languages that are not meant to have an FFI layer between them and the hardware. To compete with the performance of moving GCs, which allocate through bumping a pointer, like on the stack, and free memory in bulk, low-level languages need to rely on arenas (which work based on a similar principle), and Zig is the first language that makes arenas almost user-friendly and hopefully sufficiently composable to withstand program evolution. Of course, time will tell how well this works in practice.

treyd 8d ago
> the very thing you reach for a low-level language for - they typically require unsafe There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks. This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way. There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need! > which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower What are you talking about? I've never encountered this and I've been using Rust for 10 years.
- pron 8d ago
  That's always been true in all the safe languages with unsafe escape hatches, except here these "primitives" are the main reason to reach for a low-level language in the first place - because they presumably require the control that low-level languages offer. Combining them in the same language might appeal to some and not to others who think that the high-level, safe parts are unnecessarily complicated because it needs to integrate with the low-level parts, and the low-level parts are unnecessarily complicated because they need to integrate with the safe parts. Anyway, some like this and some don't, but my point is that it's not "mostly solved". > What are you talking about? I've never encountered this and I've been using Rust for 10 years. Okay, but I've been doing low-level programming professionally for 25 years, and have encountered this over and over in large programs (over 500KLOC) as they evolve.
  - treyd 8d ago
    That's just not an accurate description of how you write Rust in practice. There's no separate "high level" and "low level" parts/forms of the language any more than the software development process already is all about building abstractions. You should be doing this in Zig, too. It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.
    - pron 8d ago
      They're not separate forms but they are separate modes, and it is precisely because the language tries to fit both these modes into the same language that both suffer. I fully understand the goal of trying to unify these modes into the same language (C++ does the same thing), but there have always been very experienced people who like this approach and those who dislike it, hence it's not "solved". Something is solved when there's a broad consensus it's solved, and there isn't one here. I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.
nialv7 7d ago
> partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). So sad dlang never gets the credit it deserves. None of these ideas in zig are novel.
- abainbridge 7d ago
  Yes. I don't really understand why Zig gets more attention than dlang. It seems that it is more to do with the personality of the authors, rather than the technicalities. I get the impression that Walter is more laissez-faire, while Andrew is more single-minded. I'm grateful to the contributions both have made.
  - laxd 7d ago
    Pardon any corruption in my memory. But I think D in it`s early days had non-optional garbage collection. Which seemed like a bad choice at a time when I think many of us wanted a better C, not a more inconvenient C#. Then they made garbage collection optional, which split the ecosystem into garbage collected and manual libraries. Not a great choice either. Then there was a drama and split because of two competing standard libs. And also the compiler was closed source which turned some people of in a time where open source compilers had become an expectation. So all in all, some wrong turns made it lose momentum and I don`t think it can recover in todays climate.
  - pron 7d ago
    For me it's because Zig presents a novel and even revolutionary new coherent design for low-level programming, whereas D is mostly a collection of many ideas, many of them are good, but they don't coalesce into a simple design philosophy. I think somebody once said that good (or coherent) design is what happens not when there's nothing left to add but when there's nothing left to remove. That product X has all the components of product Y and possibly more doesn't mean that it contains within it Y's design. The novelty of the iPhone wasn't that it had a touchscreen, but that it had little else. Such a coherent design grabs, and deserves, attention. For example, another language that immediately grabbed my attention (aspirationally; I haven't looked at it closely yet) is https://github.com/aardappel/goose/ https://github.com/aardappel/goose/. That's not because it has arenas, but because it doesn't have anything else.
vintermann 7d ago
> but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. That IS cool. I loved partial evaluation the first time I heard of it - Futamura projections, here we come! And if it can be used to obsolete language features I never was happy with (macros), so much better! But until now, I never heard of it as a selling point for Zig. I assumed Zig was just yet another "C replacement but we don't want to deal with Rust's borrow checker". I will certainly have to take a closer look on Zig now. But this should be up and front in their self-promotion!
- pron 7d ago
  > But this should be up and front in their self-promotion! It is! "A fresh approach to metaprogramming based on compile-time code execution and lazy evaluation" is the second selling point after simplicity: https://ziglang.org https://ziglang.org Obviously, they can't call it partial evaluation because not many people know what that is. Zig's approach was eye opening to me. I'm very familiar with how macros are used in Scheme, but comptime is intentionally weaker (unlike macros, it's referentially transparent, so strictly weaker) and I was surprised by just how far it can go. It's not everyday that you see a new kind of a partial evaluation construct, let alone a language that's almost entirely based on it (like Lisp only for comptime).
hn_submit 6d ago
As far as I understand it comptime is basically runtime compilation and integration into the running program (just-in-time compilation). That gives it great flexibility of course but it also requires each Zig application to carry a full compiler inside of it. I don't understand your C++ example about it getting slower and needing to "move pointers."
- pron 6d ago
  > As far as I understand it comptime is basically runtime compilation and integration into the running program (just-in-time compilation). That gives it great flexibility of course but it also requires each Zig application to carry a full compiler inside of it. Well, it isn't that and it doesn't require that. > I don't understand your C++ example about it getting slower and needing to "move pointers." Because low-level languages need to use machine pointers, their dynamic heap allocations have a high CPU overhead; it's that overhead that moving GCs are designed to reduce, but because they move pointers, using them requires an FFI between these pointers and machine pointers. This is why in low level languages we try to avoid dynamic allocations when we can, but that increases long-term maintenance costs.

by pron 8d ago

treyd 8d ago

> the very thing you reach for a low-level language for - they typically require unsafe There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks. This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way. There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need! > which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower What are you talking about? I've never encountered this and I've been using Rust for 10 years.

pron 8d ago
That's always been true in all the safe languages with unsafe escape hatches, except here these "primitives" are the main reason to reach for a low-level language in the first place - because they presumably require the control that low-level languages offer. Combining them in the same language might appeal to some and not to others who think that the high-level, safe parts are unnecessarily complicated because it needs to integrate with the low-level parts, and the low-level parts are unnecessarily complicated because they need to integrate with the safe parts. Anyway, some like this and some don't, but my point is that it's not "mostly solved". > What are you talking about? I've never encountered this and I've been using Rust for 10 years. Okay, but I've been doing low-level programming professionally for 25 years, and have encountered this over and over in large programs (over 500KLOC) as they evolve.
- treyd 8d ago
  That's just not an accurate description of how you write Rust in practice. There's no separate "high level" and "low level" parts/forms of the language any more than the software development process already is all about building abstractions. You should be doing this in Zig, too. It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.
  - pron 8d ago
    They're not separate forms but they are separate modes, and it is precisely because the language tries to fit both these modes into the same language that both suffer. I fully understand the goal of trying to unify these modes into the same language (C++ does the same thing), but there have always been very experienced people who like this approach and those who dislike it, hence it's not "solved". Something is solved when there's a broad consensus it's solved, and there isn't one here. I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.

nialv7 7d ago

> partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). So sad dlang never gets the credit it deserves. None of these ideas in zig are novel.

abainbridge 7d ago
Yes. I don't really understand why Zig gets more attention than dlang. It seems that it is more to do with the personality of the authors, rather than the technicalities. I get the impression that Walter is more laissez-faire, while Andrew is more single-minded. I'm grateful to the contributions both have made.
- laxd 7d ago
  Pardon any corruption in my memory. But I think D in it`s early days had non-optional garbage collection. Which seemed like a bad choice at a time when I think many of us wanted a better C, not a more inconvenient C#. Then they made garbage collection optional, which split the ecosystem into garbage collected and manual libraries. Not a great choice either. Then there was a drama and split because of two competing standard libs. And also the compiler was closed source which turned some people of in a time where open source compilers had become an expectation. So all in all, some wrong turns made it lose momentum and I don`t think it can recover in todays climate.
- pron 7d ago
  For me it's because Zig presents a novel and even revolutionary new coherent design for low-level programming, whereas D is mostly a collection of many ideas, many of them are good, but they don't coalesce into a simple design philosophy. I think somebody once said that good (or coherent) design is what happens not when there's nothing left to add but when there's nothing left to remove. That product X has all the components of product Y and possibly more doesn't mean that it contains within it Y's design. The novelty of the iPhone wasn't that it had a touchscreen, but that it had little else. Such a coherent design grabs, and deserves, attention. For example, another language that immediately grabbed my attention (aspirationally; I haven't looked at it closely yet) is https://github.com/aardappel/goose/ https://github.com/aardappel/goose/. That's not because it has arenas, but because it doesn't have anything else.

vintermann 7d ago

> but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. That IS cool. I loved partial evaluation the first time I heard of it - Futamura projections, here we come! And if it can be used to obsolete language features I never was happy with (macros), so much better! But until now, I never heard of it as a selling point for Zig. I assumed Zig was just yet another "C replacement but we don't want to deal with Rust's borrow checker". I will certainly have to take a closer look on Zig now. But this should be up and front in their self-promotion!

pron 7d ago
> But this should be up and front in their self-promotion! It is! "A fresh approach to metaprogramming based on compile-time code execution and lazy evaluation" is the second selling point after simplicity: https://ziglang.org https://ziglang.org Obviously, they can't call it partial evaluation because not many people know what that is. Zig's approach was eye opening to me. I'm very familiar with how macros are used in Scheme, but comptime is intentionally weaker (unlike macros, it's referentially transparent, so strictly weaker) and I was surprised by just how far it can go. It's not everyday that you see a new kind of a partial evaluation construct, let alone a language that's almost entirely based on it (like Lisp only for comptime).

hn_submit 6d ago

As far as I understand it comptime is basically runtime compilation and integration into the running program (just-in-time compilation). That gives it great flexibility of course but it also requires each Zig application to carry a full compiler inside of it. I don't understand your C++ example about it getting slower and needing to "move pointers."

pron 6d ago
> As far as I understand it comptime is basically runtime compilation and integration into the running program (just-in-time compilation). That gives it great flexibility of course but it also requires each Zig application to carry a full compiler inside of it. Well, it isn't that and it doesn't require that. > I don't understand your C++ example about it getting slower and needing to "move pointers." Because low-level languages need to use machine pointers, their dynamic heap allocations have a high CPU overhead; it's that overhead that moving GCs are designed to reduce, but because they move pointers, using them requires an FFI between these pointers and machine pointers. This is why in low level languages we try to avoid dynamic allocations when we can, but that increases long-term maintenance costs.