12 ms·
Programming language experts told Andrew Kelley, the creator of the Zig programming language, that having code which could run at compile time was a rea
by belmarca 4y ago
Programming language experts told Andrew Kelley, the creator of the Zig
programming language, that having code which could run at compile time was a
really dumb idea. But he went ahead and implemented it anyway. Years later,
this has proven to be one of the killer features of Zig. In the Zig world, we
call it comptime, from the keyword used to mark code required to run at compile
time or variables to be known at compile time.
Which experts? The "comptime" is just macro-expansion from Scheme/Lisp which has been around for a long time. Aren't C++ templates also "code that runs at compile time"?
- wisnoskij 4y agoJai also leans very heavily into this I believe.
- schaefer 4y agoCan we get Jai compilers yet? Or is it still for in-house use only?
- eddsh 4y agoWhat's Jai?
- P5fRxh5kUvp2th 4y agoIt's a language written by Jonathan Blow, meant to replace C++ for game development. reportedly his game "The Witness" was written in it. He has a series of video's where he explores what the language will look like as he's developing it, and has claimed he'll open source it once he feels it's ready. He wants to maintain control until he feels it's no longer half-baked. IIRC, he demonstrates things like a music player running during compile time. It's a core tenet of his language.
- zem 4y agoplaying the jeopardy tune during long compilations would be awesomely hilarious (:
- memco 4y ago> He has a series of video's where he explores what the language will look like as he's developing it Probably the best place to see it in it's current form is his development streams on Twitch: https://www.twitch.tv/j_blow https://www.twitch.tv/j_blow. Right now the most active streamer besides Jon is Raphael, who does a lot of work on the Jai compiler: https://www.twitch.tv/raphael_luba https://www.twitch.tv/raphael_luba. Sadly, the other people I know who stream their work with it haven't been streaming much lately.
- panzi 4y ago> reportedly his game "The Witness" was written in it. No that was still written in C++. The new game that he's working on is written in Jai.
- P5fRxh5kUvp2th 4y agoI must have gotten my wires crossed.
- memco 4y agoIt is in private beta. You can send an email to request access. I'm not sure of the frequency or percentage of requests that are accepted, but I have seen some people get added over time.
- WalterBright 4y agoJai allows compile time execution of system calls. We (the D team) considered this, and rejected it because we didn't want users to be worried that there'd be some malicious source code that could delete their file system if they just ran it through the compiler.
- ghoward 4y agoNot GP. I agree with this, but there may still be times when it is needed. I'm working on a system where users can run a build in an interpreter that kills the build if it tries to do anything they don't allow. An important and trusted project that needs a special permission will probably be allowed to do that, but random packages? Not at all.
- WalterBright 4y agoThe trouble is, malicious people can be very clever at evading such protections. We didn't want to get into an arms race with them. I'm old enough to remember ActiveX.
- mhh__ 4y agoThey can, but this is a trillion dollar problem (cloud) now so it's harder than you might imagine
- ghoward 4y agoI think you meant to reply to me, not Walter Bright. And yes, this is a trillion dollar problem. I may be just some bloke with an idea, trying to make it happen. However, I have the free time to try, so why not?
- mhh__ 4y agoI meant to reply to Walter, he isn't a server guy so be probably isn't familiar with how modern systems defend against this problem.
- arunc 4y agoIn C++ template code is "generated" at compile time. `constexpr` code is ran at compile time.
- Ferret7446 4y agoWhat is generating code at compile time, if not code running at compile time?
- anonymoushn 4y agoIf one wishes to be pedantic, constexpr is a C++-like language for generating C++ values at compile time and templates are a quite different language for doing the same thing (and some other things). The pitch of the comptime feature is that you can do all of the things you would do with templates and constexpr (and more things due to some limitations being relaxed and type introspection existing) in the language that is similar to the language you use for runtime code. You don't have to do anything in a weird metalanguage.
- gpderetta 4y agoC++ has been trying to get compile time introspection and synthesis into the language for a while. There is a proposal going slowly through standardization, but it is not yet know when it will be ready (I believe it missed 2 or more release trains already). Circle, mentioned elsewhere, is a C++ dialect with fully working compile time metaprogramming.
- rurban 4y agoC++ has only half of a fully working compile time metaprogramming. constexpr are limited, consteval, would be the full part, but is usually an error. And the error handling is even only 10% working. clang is fine, gcc is broken for years.
- duped 4y agoThe big thing is the macro language is restricted to generating additional code when it's evaluated whereas compile time expressions are more about reducing expressions into literals.
- klyrs 4y agoYeah, that's a bit of a turnoff. The rest of the article (until the paywall) was alright. Zig's use of comptime is fairly revolutionary in a c-like language. Compared to the shenanigans required by c++, the entire language* is (should be) available at comptime. Compared to the backtick/quote/macro magic available in lisp... the awesomeness isn't that it's comptime, the awesomeness is that it's type safe. But, I'd say it isn't even the use of comptime that's particularly noteworthy, it's the lack of pretty much every feature that isn't comptime. This wouldn't be a good idea without comptime being as robust and featureful as it is. * nope, see puffoflogic's reply
- bogwog 4y agoI think Haxe has pretty much the same feature, which it just calls "macros" (https://haxe.org/manual/macro.html https://haxe.org/manual/macro.html). I haven't used Zig (yet, maybe one day), but does anyone know if there's a difference between Haxe macros and Zig comptime? AFAIK Circle also offers something similar for C++ (https://www.circle-lang.org/ https://www.circle-lang.org/)
- curtisf 4y agoZig's comptime isn't like "macros" in Haxe or Rust or Scala or Lisp which transform one AST into another AST. It's just code that runs when the code is compiled, rather than when the resulting binary is run. This makes it less expressive, but it also means it is easier to learn and use. (It's a lot like C++'s constexpr) The interesting part comes from recognizing that by making constructs like structs first-class-values in comptime, you can achieve most of the common things that templates, macros, conditional compilation, etc give you, but with a much simpler feature (which is just running code at compile time)
- dleslie 4y agoLisp and Scheme macros are just code that happens to execute at runtime, too. They don't necessarily need to transform the AST, though that's what they're often used for. Like with Zig, you are limited mostly by what is available in the environment at macro expansion time; unlike Zig, you can do things that generate allocations.
- chrisseaton 4y agoJava's SubstrateVM was doing arbitrary execution of code at compile-time, including allocating, and preserving the objects to runtime in 2013 or earlier. Zig only came out in 2016.
- openfuture 4y agoMcCarthy made lisp in 1960.
- dleslie 4y agoMacros didn't arrive in lisp until 1963, via a memo from Timothy P. Hart. See AIM-57: http://publications.csail.mit.edu/ai/browse/0000browse.shtml http://publications.csail.mit.edu/ai/browse/0000browse.shtml
- P5fRxh5kUvp2th 4y agoI stopped reading after the first sentence, it's basically an outright lie. And for what, exactly? Your audience is the same audience you're claiming told Andrew it was a dumb idea, you're basically calling your audience dumb.
- socialdemocrat 4y agoJust commenting here as the author. Don't call people liars. Being wrong about something and lying are two very different things. I was writing that intro based on recalling Andrew Kelley stating in a video that he had been advised against running code at compile time by several people. I could not recall if he ever said explicitly who that was. It was an off-hand comment. I choose to par-phrase him, and had no idea that this would get anyone this upset and lead to me being accused of being a liar. How much of experts these people where I cannot vouch for. Maybe they were second rate language designers or maybe Kelley referred to older papers advising against it. Either way I cannot find the exact video anywhere where he made this remark despite looking through several. Does it really matter? Can you say for a fact that no language designer expert ever told Andrew Kelley this? I am not claim this is the general opinion among designers, only that this is what Andrew got told. If anyone has the correct quote, I'll be happy to update the article.
- P5fRxh5kUvp2th 4y ago
- xupybd 4y agoWow, stop the personal attacks. Remember you're to talking to a person with feelings that put in the work to write an interesting blog. This behavior is nothing short of antisocial. You're not attacking ideas here you're attacking a person. Having a little grace for people goes a long way.
- bowsamic 4y ago
- socialdemocrat 4y agoAuthor here, I cannot vouch for exactly the competency level here, but I was paraphrasing a remark Andrew Kelley made in a video discussing Zig. From what I remember he said that many told him that running code at compilation time wasn't a good idea and you shouldn't do it. My understanding was that these must have been people with some experience with programming language design. Why would Andrew Kelley listen to random opinionated guys on the internet? I will try to lookup this video when I have time. I was a bit surprised that this remark was a such a turn-off. Had I known, I should have looked up the video to get the exact quote. It was just meant as a bit of a funny dramatic entry to discussing an unusual language design choice for a statically type language. Sure we are seeing more things like const expressions in C++ today, but at the time this wasn't very common.
- belmarca 4y agoI don't think it's such a big issue but it is quite surprising to read, since these concepts have been known and used for many decades :)
- coldtea 4y agoI don't think that the precedent matters that much in this situation, as to whether people would dismiss such an idea or not. People can dismiss language ideas and concepts that "have been known and used for many decades" just the same. For a trivial example: significant whitespace. So, e.g. "Lisp has had that since forever" is not a guarantee people would be OK with it, unless they also like Lisp - not to mention they might not even know about it in Lisp. As for C++, many people consider templates - at least as implemented - a bad idea. So some people knowing that it existed in C++ already, and considering it a bad idea to have in Zig is not contradictory.
- FeepingCreature 4y agoSure but then it's like... for example, if you land rockets and your advertisement says "People told me it couldn't be done!", then fair enough. But if you, say, build an electric car and your advertisement says "People told me it couldn't be done!", then ... who exactly have you been talking to? The phrase "They said it couldn't be done!" is an evocative proxy for technological novelty. If the experts are not representative, the claim of novelty will be misleading.
- SighMagi 4y agoLisp user here, I couldn’t quiet tell from all the Zig syntax, but is “comptime” here like having the whole language available to you to do whatever you like at at compile time like in Common Lisp?
- TUSF 4y agoI've never used lisp, but I'm pretty sure you have the right idea. The only limitation is that it can only act on data that is compile-time known, which is a defined concept in Zig. At the moment you can't do comptime allocations, so everything has to exist on the stack, but you can, for example, embed a file like a config or dataset, and run it thru a parser to turn it into some data structure, which is something you might otherwise work into the build system, running python/perl scripts to generate temporary code files, but now you can cut out the extra dependency by having that code generation in the same language.
- WalterBright 4y agoD has has compile time function evaluation for 15 or 16 years now. It doesn't have a special keyword, either. It's simple - everywhere the grammar has a const-expression, then compile time function evaluation is in play.
- nurettin 4y agoI never liked the original D in the early 2000s, especially the anti-C++ marketing rhetorics. But the post-alexandrescu D is amazing. Just last week I placed a delegate bound to a closure into an immutable struct within a child thread's run function and passed it into the main thread via send to be called and it actually worked! What an amazing language. There aren't many 3rd party libraries, though. And whatever exists is highly questionable.
- kazinator 4y ago> And whatever exists is highly questionable. Could it be because some of it, say, places a delegate, bound to a closure, into an immutable struct within a child thread's run function, and passes that to another thread via a send to be called?
- nurettin 4y agoI think so, I mean, if it is a one-off POC, nobody is going to build their uber-for-can-opener startup on top of it. But if billions of closures are actively passed around every day by thousands of projects, and lots of bug fixes and features are deployed continuously, why not build your airbnb-for-doghouses empire using that library today?
- WalterBright 4y agoFortunately, D now has a C compiler embedded into it (ImportC). ImportC will compile C11 Standard code, and automatically make the interface to that C code available to D. This means, if a C library is available, D can use it. (There are some exceptions. While ImportC will handle C preprocessor metaprogramming just fine, those metaprogramming macros won't be available to the D code, other than simple #define manifest constant declarations.)
- shultays 4y agoI also remember Zig inventing bitfields in another blog post, it is a revolutionary language
- chx 4y agoElixir too. https://medium.com/@fxn/how-does-elixir-compile-execute-code-c1b36c9ec8cf https://medium.com/@fxn/how-does-elixir-compile-execute-code...
- drewhk 4y agoAnd don't forget multi-stage programming which generalizes the concept and allows for partial evaluation at compile time and runtime as well.
- jcelerier 4y agoZig goes a fair bit further than c++ afaik. The ideal would be to be able to call run-time functions indiscriminately at compile-time (e.g. fopen) instead of having to build separate other tools that will generate source code...
- rurban 4y agoThough here be dragons: All side-effects evaluated at compile-time, are not run at compile-time! So fopen, print and all IO or OS functions should be avoided, or even warned. It might be useful in certain edge-cases, but in my 20 years of compile-time programming, it's almost ever a bug, not a feature.
- dolmen 4y agoPerl 5 had compile-time execution forever. And this feature is used in everyday's code as that feature is at the core of the implementation of exporting symbols from a module into its importer (See module Exporter). Example: the BEGIN block is executed during the compile phase as the parser encounters it. The code injected in BEGIN blocks can affect how the rest of the file is parsed. $ perl -E 'exit; BEGIN { say "Hello" }' Hello "perl -c" stops the program after compile, but before execution. In the following code the BEGIN block runs, but not "exit 1". $ perl -c -E 'exit 1; BEGIN { say "Hello" }'; echo $? Hello -e syntax OK 0 More info: * https://perldoc.perl.org/perlmod#BEGIN,-UNITCHECK,-CHECK,-INIT-and-END https://perldoc.perl.org/perlmod#BEGIN,-UNITCHECK,-CHECK,-IN... * https://perldoc.perl.org/perlfunc#use-Module-VERSION-LIST https://perldoc.perl.org/perlfunc#use-Module-VERSION-LIST
- rowanG077 4y agoThis is probably a complete fabrication. Running code a compile time, even automatically has been extensively studied in CS. Just search for partial evaluation or supercompilation.