20 ms·
Zig self hosted compiler is now capable of building itself
- ledgerdev 4y agoI have been thinking about creating a little higher level language that targets server side web assembly. Zig looks very attractive, but concerned with how to handle things like strings and datetimes. How suitable is zig for such a task compared to say rust?
- pacaro 4y agoI may be misunderstanding you, but it feels like strings and datetimes are library related more than language related. I know that higher level languages will have primitives that aim to represent strings, but if you need to get into the weeds with Unicode then you'll be leaning on a library regardless
- ledgerdev 4y ago> I may be misunderstanding you, but it feels like strings and datetimes are library related more than language related. Yes would agree and see them as platform related. It's just too large a task to create from scratch. Like say on JVM you can compile to bytecode and have strings already built into platform, and java.time, and ability to access an ecosystem of libraries. With zig could one could use c or rust libraries?
- vonwoodson 4y agoFor great justice
- einpoklum 4y agoZig move zig...
- spicybright 4y agoA big achievement for any decently sized language. Nice work devs :)
- Shoop 4y agoFor reference: Tracking issue for overall progress on the self-hosted compiler: https://github.com/ziglang/zig/issues/89 https://github.com/ziglang/zig/issues/89 Zig's New Relationship with LLVM: https://kristoff.it/blog/zig-new-relationship-llvm/ https://kristoff.it/blog/zig-new-relationship-llvm/
- AndyKelley 4y agoHello HN! Here is some context to decorate this announcement: The Zig self-hosted compiler codebase consists of 197,549 lines of code. There are several different backends, each at varying levels of completion. Here is how many behavior tests are passing: LLVM: 1101/1138 (97%) WASM: 919/1138 (81%) C: 740/1138 (65%) x86_64: 725/1138 (64%) arm: 490/1138 (43%) aarch64: 411/1138 (36%) As you might guess, the one that this milestone is celebrating is the LLVM backend, which is now able to compile the compiler itself, despite 3% of the behavior tests not yet passing. The new compiler codebase, which is written in Zig instead of C++, uses significantly less memory, and represents a modest performance improvement. There are 5 more upcoming compiler milestones which will have a more significant impact on compilation speed. I talked about this in detail last weekend at the Zig meetup in Milan, Italy[1]. There are 3 main things needed before we can ship this new compiler to everyone. 1. bug fixes 2. improved compile errors 3. implement the remaining runtime safety checks If you're looking forward to giving it a spin, subscribe to [this issue](https://github.com/ziglang/zig/issues/89 https://github.com/ziglang/zig/issues/89) to be notified when it is landed in master branch. Edit: The talk recording about upcoming compiler milestones is uploaded now [1] [1]: https://www.youtube.com/watch?v=AqDdWEiSwMM https://www.youtube.com/watch?v=AqDdWEiSwMM
- icsa 4y ago> uses significantly less memory, and represents a modest performance improvement. The reduced memory has significant value. Being able to do the same build on less expensive hardware or do more with the same hardware is a significant financial performance improvement
- joshbaptiste 4y agoyup 18x memory reduction improvement 8.5GB -> 0.5GB according to the vid..
- erwincoumans 4y agoCongratulations with the milestone! Does using Zig over C++ lead to "less memory, and represents a modest performance"? Or was the C++ implementation a bit sloppy? (lacking data oriented design for instance) Also, what specifically are you most excited using Zig for?
- fouronnes3 4y agoAre there any languages out there that can only be compiled by a compiler written in their own language? Presumably because the original pre-dogfood compiler stopped being maintained years ago. So that if we somehow lost all binaries of the current compiler, that language would effectively be lost?
- dottedmag 4y agoC?
- viraptor 4y agoYou're commenting on an article about Zig which became self-hosted and can compile C. (There's also lots of other C compilers available)
- dundarious 4y agoZig compiles C/C++ by deferring the vast majority of the work to libclang, which is written in C++. Also note Zig is self-hosted when using the LLVM backend, which means deferring to C++ for much of the code generation. There is no "end-to-end Zig" self-hosted compiler yet, because the Zig native backends are not as near completion. See the creator's comment about the breakdown: https://news.ycombinator.com/item?id=31052234 https://news.ycombinator.com/item?id=31052234. (I'm excited about this progress, so this is not meant as any kind of knock on Zig, which I think is quite impressive) But you're right that C is not a good example.
- coder543 4y agoA lot of popular C compilers are actually written (at least partially) in C++ these days.
- dottedmag 4y agoIn this case, C++?
- jonpalmisc 4y agoI think Zig’s compatibility with C is such a valuable feature. I also wish we could rewrite everything in a modern language, but the reality is that we can’t and that if we could, it would take a LONG time. The ability to start new projects, or extend existing ones, with a modern and more ergonomic language—Zig—and be able to seamlessly work with C is incredible. I look forward to the self-hosted compiler being complete, and hopefully a package manager in the future. I’d really like to start using Zig for more projects.
- jitl 4y agoZig as the “Kotlin of C” makes it very appealing. Kotlin has seen fantastic adoption in JVM projects because you can convert files one at a time from .java to .kt, with only a modicum of one-time build system shenanigans up front. Then your team can gain experience gradually, fill in missing pieces like linters over time, all without redesigning your software. What zig offers is even better - because zig included a CC, you can actually reduce complexity with zig by getting a single compiler version for all platforms, rather than a fixed zig + each platform’s cc. And with it, trivially easy cross-platform builds - even with existing C code. That’s cool! Go has excellent cross-compilation, but Go with C, not so much. Rust is a powerful tool, but it’s a complex ecosystem unto itself - both conceptually and practically. It has great interoperability frameworks, but the whole set of systems comes at a substantial learning cost. Plus, porting an existing software design to Rust can be a challenge. It’s more like the “scala of C” if we’re trying to stretch the analogy past the breaking point.
- richardfey 4y agoThe "Kotlin of C", what a beautiful metaphor!
- pjmlp 4y agoKotlin has seen fantastic adoption in Android projects, because of the way Google pushes it, while stagnating Android Java on purpose. On the JVM world not really. https://www.infoq.com/news/2022/03/jrebel-report-2022/ https://www.infoq.com/news/2022/03/jrebel-report-2022/
- Shadonototra 4y agoBest alternative to C, built with solid foundation to ensure fast compile speed, fast iteration, and easy to maintain code
- mlindner 4y agoI don't understand why people want fast compile speeds so much. My day job is/was writing C on something that requires 30+ minutes for one compilation run. It feels like people want fast compilation speeds because they need to keep running their software for some reason. Fast compilation speeds are a nice to have but basically irrelevant for normal developing of software in my experience.
- hu3 4y agoSorry I may have missed the sarcasm. Your day job project takes half an hour to compile and you don't see the point in speeding that up?
- sanxiyn 4y agoGP did say it is a nice to have.
- hu3 4y agoBut also said that it's basically irrelevant. Hence my confusion.
- mlindner 4y agoIt wasn't sarcasm. My "irrelevant" point meant "it's irrelevant for a factor in picking a language", as in "it doesn't significantly contribute to whether a project is possible in the long term". If compile times are longer you just compile less often and write larger portions of code at a time and you generally keep writing code while it compiles.
- jackosdev 4y ago
- mhh__ 4y agoD has a C compiler built-in now, I wonder if we could bootstrap all the toolchain now.
- ByteJockey 4y agoDon't most toolchains have a rather large C++ component? Just parsing C++ is a rather large undertaking (my understanding is that it's syntax is turing complete).
- mhh__ 4y agoSo you start with a C++ compiler written in C. Hence bootstrap, rather than compile from source.
- ByteJockey 4y agoOh, gotcha. Sorry, brain wasn't parsing right before the coffee today.
- cercatrova 4y agoI was thinking of learning Rust but it seems a bit overkill due to manual memory management as compared to languages with similar speed like Nim, Zig, and Crystal. How would one compare these languages? Is it worth learning Rust or Zig and dealing with the borrow checker or manual memory management in general, or are GC languages like Nim or Crystal good enough? I'm not doing any embedded programming by the way, just interested in command line apps and their speed as compared to, say, TypeScript, which is what I usually write command line apps in.
- mlindner 4y agoMemory has to be managed by something. The more decisions that are made for you in how that happens the less flexibility there is for certain situations.
- cercatrova 4y agoSure but my use cases would be stuff I'd normally write in TypeScript or Python that already have garbage collection. Like I said I'm not doing embedded programming so I don't have too much of a need to manage memory. My question could be further constrained then to be, is learning Rust or Zig despite its manual memory management worth it for applications that are normally already garbage collected in their current implementations? Or are languages like Nim and Crystal enough? Does Rust and Zig have other benefits despite manual memory management?
- adgjlsfhk1 4y agoimo, most code can do just fine with GC. modern GCs can be relatively low overhead even with guaranteed small pauses (10ms). furthermore, most code that can't handle pauses can be written to not allocate any memory (so GC can be temporarily turned off). as such, the only two places where you need manual allocation are for OS development, and hard real time requirements.
- Comevius 4y ago
- deleted 4y ago[deleted]
- riffic 4y agoTake off every 'ZIG'!! edit: all your base are belong to us
- orblivion 4y agoProbably a stupid question, but is LLVM a sort of cheat for "self compiling"? Shouldn't you need to compile LLVM as well?
- okareaman 4y agoIt used to be the case that a compiler compiled itself to bare metal machine instructions, but now with so many cpu instruction set targets that's no longer the case. LLVM uses an intermediate language like an assemby language. Another way to look at it is the TypeScript compiler compiles itself which is written in TypeScript but the intermediate language is JavaScript. V8 handles the translation of JavaScript to machine code, similar to LLVM translating LLVM IR to machine code
- orblivion 4y agoOkay, so we're treating LLVM intermediate language or javascript as if "assembly" in the bare metal days. Makes sense.
- chrisseaton 4y ago> It used to be the case that a compiler compiled itself to bare metal machine instructions, but now with so many cpu instruction set targets that's no longer the case. Tons of compilers emit machine code.
- okareaman 4y agoSure, I didn't say they didn't. Are you refuting what I said and implying that a compiler is not self-hosted if it doesn't compile to machine code?
- chrisseaton 4y ago> I didn't say they didn't You said 'that's no longer the case'. Isn't that the same as saying they don't?
- diosito 4y ago
- diosito 4y ago
- randyrand 4y agoAndrew how will the self hosted compiler maintain a known trust back to assembly? i.e how can someone look at a self hosted zig compiler and build it themself from source, never needing to download blobs from the internet? Otherwise you lose the ability to trust anything you build.
- kristoff_it 4y agoCheckout the video Andrew linked in this thread. He talks about this in detail.
- amscanne 4y agoZig specifics aside… build it from source using what? Another compiler. Okay, so you can compile that one from source. But what does that? Another compiler. Unless you’re recording memory contents and executing instructions by hand, you’ve just discovered the Ken Thompson hack. At some point the pragmatic thing is to trust some bits from a trusted source (e.g. downloaded from an official repo w/ a known cert, etc.).
- randyrand 4y agoYou just need 1 compiler written in assembly, and work your way up from there. C17 -> assembly of your choice, written in assembly Zig -> C17 “transpiler”, written in C17
- gautamcgoel 4y agoQuick question, which maybe someone here can answer: I noticed that GitHub supports syntax highlighting in the Zig repo. How does that work for a new language such as Zig? Can you somehow upload a file which tells GitHub how you'd like programs written in your language to be displayed?
- aeldidi 4y agoI believe this is the syntax highlighter’s repo: https://github.com/github/linguist/blob/master/CONTRIBUTING.md#adding-a-language https://github.com/github/linguist/blob/master/CONTRIBUTING....
- gautamcgoel 4y agoThanks, this is what I was wondering! Seems a new language needs to have 200 repos which use it before GitHub will consider adding syntax highlighting for it..
- deleted 4y ago[deleted]
- mindwok 4y agoI’ve spent the last few weeks building an 8080 emulator in Zig to learn both emulator programming and the language. Gotta say, it’s been a pretty pleasant experience. My only issue was with dynamic dispatch, which lead me down quite a rabbit hole which I didn’t ever fully come out of. Seems that the current situation is build your own using compiler functions and pointer casts.
- endgame 4y agoIs there a plan to maintain a bootstrap path over the longer term?
- fithisux 4y agoHuge, congratulations.