11 ms·

I Believe Zig Has Function Colors

142 points by gavinhoward 5y ago

deleted 5y ago
[deleted]
staticassertion 5y ago
I wish we had more colors for functions. I don't like the whole "colorless functions" idea. I want to, as a function, be able to constrain my caller. "You can only call me in an async function" is a cool thing to communicate. "You can only call me if you can handle errors", "you can only call me if you have read/write access to the file system", etc - I want arbitrary colors for my functions, not colorless functions. Not only does this mean that I can express arbitrary constraints on callers, it also means that all constraints and stateful expectations are explicit, verifiable, and clear to the reader. The post is interesting because it highlights that, colorless or not, functions have constraints and you can try to hide them and handle things automatically, via inference as is demonstrated here, but they're always there. Async is probably one of the less interesting constraints, but I find it so much more confusing when languages have colorless functions... I can't see suspension points, I can't tell wtf is going on. It's extremely frustrating. > However, before I continue, let me say a little bit of praise for Zig: this is innovation. This is a piece of good work that, most of the time, async functions can be called the same as normal functions. It's definitely cool and I like the definition of async they have. I think suspension is probably one of the less interesting things that a function can do, vs, say, IO, which is similarly "colorless" in languages. I just wish there were colors for all of these things.
- captainmuon 5y ago
  Nim toyed with this at some point as an "effect system". I wrote some pragmas (decorators) that ensured a certain function was only run on the network or GUI thread. In Python, I had the same, but as runtime assertions. Another thing that you could do is in the command pattern for undo/redo, to make sure changes to your data model only happen from command objects. In Python, I had a decorator that said "when this property is set, assert that Action.do is on the call stack". It would be really powerful to be able to do that at compile time, too.
  - elcritch 5y ago
    The effect system is still in Nim and supported. Though I've never used it, except to ensure top level procedures catch all exceptions properly.
    - xigoi 5y ago
      IIRC, Araq mentioned that he doesn't like the current effect system and might deprecate it.
- cprecioso 5y ago
  I enjoy Swift’s version of this, where you mark if a function can throw and you need to mark the callers up the chain until you handle those errors. There’s also Haskell’s system, which IIRC uses an IO monad.
  - inglor 5y ago
    Haskell's system (with the io monad + do not all monads or in general) is extremely similar to async/await and most languages with co-routines can simulate do notation in Haskell pretty well syntactically. The advantage of something like JavaScript's async/await over Haskell's "do" is the fact because async/await is more limited than `do` (or yield and coroutines in JS) it's a lot easier to create tooling for.
    - kaba0 5y ago
      I would say it is a huge disadvantage -- do is just syntactic sugar for a monad expression of binds, which is a native part of the language, you can write a simple function to manipulate it however you like. While you would need to effectively parse AST in case of JS.
      - inglor 5y ago
        `yield` is just syntactic sugar for the two-way generator communication protocol where the consumer calls `.next(value)` which returns a value in turn. This is as expressive and you can build `do` semantics on top of stuff like lists/either/state/io on top of it. No AST parsing needed.
        kaba0 5y ago
        And do syntax sugar is just nested calls of `bind` which is a function of the Monad trait. Lists are also a Monad, so do notation just works right now with lists, either, state everything in Haskell. I really don’t get how is Haskell “behind”.
    - ImprobableTruth 5y ago
- marcus_cemes 5y ago
  This is my sentiment as well, I understand some people's reservations and the fact that "async pollutes every calling function, as now everything must async as well", but to me this is correct behaviour and beneficial for the programmer. To me, this is akin to the pure/unpure function classification in FP, it makes sense that calling an unpure function would also "taint" the caller. Why should "fetch_user_db_which_may_be_on_the_other_side_of_the_world()" look the same as "Math.abs()", when their behaviour for the caller is so different? I recently tried Go to speed up some Cloud Functions that use Firestore, it's very difficult to know which of the chained method calls of the Google-provided SDK is the one to actually "blocking" the green thread, perhaps they are all are unnecessarily? Without studying the source code, it's hard to know. I prefer explicitness over absolute simplicity. It also makes it easier to optimise code in my opinion as it's easier to identify sequential async calls that could actually happen concurrently, either by merging SQL queries or by using Promise.all() or the equivilent. JavaScript does a good job with the await keyword in my opinion, you know exactly when something could take a significant delay, when it's doing something more behind the scenes. Rust does one better by making ".await" a suffix, allowing for nifty call chains. It communicates that this is an explicit yield point and could take a while.
  - kaba0 5y ago
    But with go's green threads, does it really matter if something blocks or not? Given, I'm not too familiar with Go, but in the case of Java's upcoming Loom which will do something similar, you would just create a virtual thread and if you have to do something with the output of a previous call, you just write plain easy "blocking" code. The JVM will switch to another virtual thread during the "blocking" part automagically and you are done. Whether something is async or not doesn't add anything to the idea you want to implement (in managed languages that is -- rust do need the async keyword because it is a low-level language).
    - egeozcan 5y ago
      > But with go's green threads, does it really matter if something blocks or not? Then you keep pushing stuff to other threads "just to be sure", which may hurt performance but the worst part becomes reading that code.
- anonymoushn 5y ago
  Aside from the demand for explicitness, we may not have to choose one or the other. The promise of Zig's semantics w.r.t. async is that you can write a library that accepts allocators, writers, and readers from the end user, and your library's functions will be specialized at compile time to be async or not-async based on what kind of writers and readers were passed in. If we had a broader effects/capabilities/whatever system, it would still be nice if library authors could address the needs of many different users without caring whether those users want to do IO. If we really need to be explicit, we begin to need different libraries for each combination of effects/capabilities/whatever, sort of like how Rust users need at least 4 separate sync/async/nostd-sync/nostd-async libraries just to speak websockets. This seems like a waste to me. Zig's error system in which errors must be handled and functions can opt to have their error sets inferred by the compiler (again, this includes specialization to different error sets for instances of the function that are parameterized by functions with different error sets) sort of gives us the same thing for errors that we have for async-ness, but I can sympathize with a perspective that feels that special casing these two things and not allowing any others is inelegant or whatever. > Async is probably one of the less interesting constraints, but I find it so much more confusing when languages have colorless functions... I can't see suspension points, I can't tell wtf is going on. It's extremely frustrating. Zig's view on this, which I endorse, is that suspending to your event loop is not substantially different from suspending to the kernel. Zig sort of assumes that we don't want to manually mark each function with which syscalls it and its callees are allowed to make, but maybe it's worthwhile to make a language that does do that.
  - throwaway82652 5y ago
    >and your library's functions will be specialized at compile time to be async or not-async based on what kind of writers and readers were passed in Based on what I've seen in Rust's async, you don't actually want this in a low level language. Those calling scenarios are fundamentally different. If you magically make something async, other code can now suddenly mutate state that your function is referencing on its own stack and really mess things up. In Rust this isn't an issue because you're not allowed to pass references into an async function that outlive the future. If you take a synchronous function that holds references and add the async keyword, you probably will get compile errors about this. I don't think Zig has anything like that. >Zig's view on this, which I endorse, is that suspending to your event loop is not substantially different from suspending to the kernel. No, that doesn't make sense, not even in the context of Zig. Suspending to the event loop allows other coroutines to run. Blocking in a syscall stops the whole program and all the coroutines.
- kaba0 5y ago
  I'm not necessarily disagreeing with you, but is there an example for a function that doesn't make sense to call from both an async and non-async function, other than an implementation detail of the programming language? But continuing on your idea, one colouring I would really like to see everywhere is pureness. C++'s `const` comes close, and perhaps Haskell does it right by having everything be (mostly) pure by default and using the type system to encode other properties. And yeah - different monads (like IO) are likely a good way to handle this whole topic in a user-definable way, but not many people like them (at least when they know they are using them, because unknowingly many programmer do use it, eg. most async syntactic sugar is basically a do notation for a single hard-coded Monad)
  - kubb 5y ago
    in javascript's programming model, literally everything that's running in response to user action (clicks, etc.) absolutely has to complete immediately (from the browser user's point of view), or it will hang the UI. async event handlers really help with that. a non-async handler blocking until a long standing operation is complete (i.e awaiting an async function) is unacceptable the async/non async functions in javascript are good and well designed. the whole colored functions thing is nonsense and it's weird that it gained so much traction on here
    - kaba0 5y ago
      Starting up a new virtual thread and writing “blocking” code in it, while the VM schedules it on top of a single thread is entirely possible and will make everything happen quasi-instantly, the same way as async is.
  - feanaro 5y ago
    I think that people that dislike monads mostly fall in one of two categories: those that know about a fancier alternative such as algebraic effects and those that are scared off by an alien-looking unfamiliar concept. Those two groups are minorities in the set of all people that are even aware of monads.
- kjeetgill 5y ago
  So in someways, the best generalization like this I already commonly see is a capabilities system, which can be done essentially via function parameters. I've used this as an OOP pattern once or twice and I think it's the easiest way to explain it: Imagine you have 2 singleton objects in your 'void main()', redkey and bluekey of types Redkey and Bluekey. Somewhere else, you declare your function: 'int foo(Redkey redkey, int x, int y)' that needs a Redkey object of which only one exists: in your main. This by itself forces every call on the path between 'main()' and a call to 'foo()' to also include a parameter Redkey. In the extreme cases where a pattern like this is useful (tracing IO calls, you named it) it can really help cut through a codebase after an hour of refactoring. But it can be limiting. Async and Checked Exceptions are probably the most colored functions and they both need escape hatches because of that.
- pooya72 5y ago
  Wouldn't this be something similiar to Monads? I remember someone gave a talk and described Monads as "worlds". (I forgot which video it was). So a function goes into a "world" does all it's stuff, then it exits and comes down.
  - tsimionescu 5y ago
    The "Monads=worlds" comparison sounds appealing for some monads (IO, ST), but is extremely unintuitive for others (List).
- the_duke 5y ago
  What you want is an effect system. Check out Koka lang for a very interesting implementation: https://koka-lang.github.io/koka/doc/book.html#why-effects https://koka-lang.github.io/koka/doc/book.html#why-effects
  - artemonster 5y ago
    I wonder how many times people will have to reinvent algebraic effects until it becomes mainstream :)
    - the_duke 5y ago
      Until a large company either creates and heavily invests in a new language, or integrates an effect system into a very popular language like C# or Java. Sadly I don't see that happening anytime soon.
    - mananaysiempre 5y ago
      I know this is a very basic question, but what is it about algebraic effects that makes them compose when monad transformers don’t? I’ve read some of Bauer’s early Eff work, which IIUC was explicitly motivated by this, but for all the neat stuff about resumable exceptions and delimited control and whatnot I don’t feel I ever saw (or recognized) the definitive answer to this.
  - staticassertion 5y ago
    Oh yeah, I know.
- delusional 5y ago
  The problem with colors is, as most things, bad programmers. The seminal example for me is Java Exceptions. Java tried to color functions based on which errors you had to handle in order to call that function. Well, most beginners didn't understand what an "exception" was, so in order to learn they were told to just ignore if for now. A Decade of bad advice like "just extend RuntimeException" and most of the production code at my job now is littered with catch(Exception e) because colored function were too hard for people. I still get confused looks when I say "Well if I change the types of errors this function can produce, i want the compiler to stop me until I've actually handled them". You see the same thing happen in async circles. All of a sudden every single function has to be labeled async, whether it needs to or not.
  - CornCobs 5y ago
    I don't think it's fair to label the failure of checked exceptions as "beginners didn't understand". It's the fact that returning null was deemed an acceptable return value on some types of failure throughout the standard library, rendering the whole idea basically moot since it didn't actually give you much guarantees. And later on, checked exceptions were incompatible with new stuff like lambdas and streams
  - anonymoushn 5y ago
    One reason checked exceptions are a not great is that library authors would like you to implement their interfaces or extend their classes, but they've decided that you can't throw anything :) Zig actually does pretty well on this front. you can write a library that accepts functions that return errors, and your functions will be specialized to return those errors in addition to their own if you don't handle them. Callers who want to handle errors exhaustively with a switch will get compile errors if the error set changes and they fail to handle new errors.
  - mananaysiempre 5y ago
    I’m not convinced programmers are the problem. Haskell’s use of monads is perhaps the ultimate in colouring (for values not functions, but that doesn’t make that much of a difference); you routinely colour everything with which pieces of state it needs to read and/or write to do its thing. The result is you either have everything painted in “global state”, which just feels like unnecessary boilerplate, or you spend most of your time defining and converting between elaborately carved pieces of said state. People could and did invent various intricate constructions which would do the conversions and possibly even the carving for you, but overall I think most agreed it was a gigantic pain. It’s possible the situation is fixable with a correct combination of features (records, polymorphic variants, implicit parameters, possibly even algebraic effects), as a good portion of the pain comes from not having a standard solution every library would use (they all have drawbacks). But then the checked exceptions story in Java, IIRC, is from pre-Java 5 times, and not being able to be polymorphic in the exception signature of a callback you take is really stupid however you look at it, as well as also a case of inadequate language features.
- syntheweave 5y ago
  When I've encountered colored functions(and here I'm specifically thinking of D's pure annotation) I have noticed that the architecture tends to collapse to the most permissive forms very quickly, because you will discover layers that were meant to encapsulate the coloration, and did "quack like it", but in a strict sense do not. For example, one's expectations of a pure function would mean that you could do some arbitrary numeric computation on inputs and return a value. Unfortunately, the compiler is not so naive. If you should do some transcendental math and call a standard library function to do so, you have manipulated hardware state. You are now impure, and so are all your callees. And so you have to decide at that point: is the code you are writing wrong? Or is it just using too coarse an assumption for its invariants? A huge amount of what determines our code architectures are the way in which we've organized our own thoughts. In this respect we always go from looser to tighter specification over time as the code matures, until tightness has produced brittleness and cracks. I believe coloration forces some tightening decisions to be scheduled earlier. In that respect it should be treated with some suspicion, as it may produce surprising couplings.
  - iamwil 5y ago
    How come using transcendental math changes hardware state?
    - adgjlsfhk1 5y ago
      they probably either set rounding modes or use x87 registers
  - simulate-me 5y ago
    Adding colors without a concrete reason to do so will always be a waste of time. Take type checking as an example. If you use type A as a parameter that requires type B, your program will not work. Type checking provides a lot of value. Or in the GP’s example, forcing functions into async contexts ensures that, for example, the main UI thread doesn’t get blocked. The program will appear broken to the user if this rule is broken. Now compare this to pure functions. What happens if you break the pure function contract? Probably nothing. It’s still totally possible for the program to work. So you end up doing a lot of work that ultimately doesn’t change anything, which is what you’re observing with the proliferation of impure code.
- feanaro 5y ago
  I think you're missing the point. The problem of colour is not a problem of limitations. It is indeed a good idea to limit your functions to the least required amount of power via the type system. One great way of achieving this is using an algebraic effects system which someone already linked to elsewhere in this thread. The problem of colour is rather that even though two functions should have the same structure, regardless of their effects, the language still forces you to write two functions each with its own syntactical structure. In type theory terms, this is a problem stemming from a lack of effect polymorphism. Ideally you'd write one function, polymorphic in its effects, and then just be able to use it in both sync and async contexts. In the sync context, the function's type would specialize in a way that it lacks the `Async` effect, so it wouldn't be able to do async stuff there. In the async context it would specialize in a way that it does have the `Async` effect. Therein you would be unable to apply it in contexts requiring the lack of an `Async` effect.
  - mjburgess 5y ago
    ie., we need a color polymoprhism
  - staticassertion 5y ago
    Yes, I probably should have expanded my post to say that an effects system is how this should all be handled.
- flohofwoe 5y ago
  I think the main problem here is that you would end up with multiple copies of the same API, e.g. node.js has three "stamped out" versions of the filesystem API: (1) synchronous, (2) asynchronous with callbacks, and (3) asynchronous with promises. This duplication sucks both for the library maintainer as well as the user (IMHO).
  - francasso 5y ago
    If js had async/await from the beginning there would have been no need for the callback version. Also the sync apis would exist in any case because of the event based architecture of node and the fact that at times you don't want to release control back to the event loop.
  - chrisshroba 5y ago
    This seems like more of an issue with porting an existing language than creating a new one. With the “can write to the file system” example, you would only need one version of a “writeJSON” function, for example, because writing without write access is nonsensical. Then you’d specify exactly what you need through these constraints, and if they weren’t met, you couldn’t call the function. This already exists and works well when it comes to const-correctness in a language like C++. You can’t call a non-const method from a const one, for example, and this doesn’t lead to multiple versions of each function, because if a function mutates state, you know that it will never be able to be called from a const method.
    - anonymoushn 5y ago
      You probably need at least one WriteJSON that's synchronous and one that's asynchronous, with different colors. If you want a version that uses memoization internally, it needs the "can allocate memory" color, which is independent of the async/sync color.
      - virchau13 5y ago
        Could it be possible to infer optional colors from the environment? You denote that the color is optional in the function's type signature, and inside the body of the function, you have a construct that lets you execute different code depending on the state of the function's colors. The call site could either automatically infer the colors used or the colors used could be manually specified. Basically a type parameter with compile-time if statements.
- edem 5y ago
  Be the change you want to see...or just use a strictly typed FP language like Haskell and you'll get all your colors.
- mikepurvis 5y ago
  Traits is imo kind of this, in rust and to a lesser extent c++. At the very least, a templated function can insist on being injected with what it needs to engage with the outside world as far as an allocator, IO, error handling, and so on.
- zarzavat 5y ago
  People mean one of two different things when they talk about “function colors”. Let’s take the example of JS. Does JS have function colors? In one sense, yes, there are async functions and sync functions and you can only call an async function from an async function. So under a certain definition, JS functions are colorful. But wait... a JS async function is just a normal function that returns a Promise object. So under another definition, JS functions are colorless. Perhaps someone might not like how Promises work in JS, but this has nothing to do with what color the functions are. There are some languages that do break this and implement async functions as a completely different entity to normal functions. I believe this kind of function colors is lazy and bad language design and these async implementations tend to feel like they don’t belong with the rest of the language and were duct taped on.
  - Dylan16807 5y ago
    The async keyword strongly changes the internals of a function to let it use "await". Even if you do it manually, it's a very particular and restricted style that infects your caller. That's a color.
- gpderetta 5y ago
  Let's say that you have a pure function f that does a bunch if calculations and return the result. The caller then prints it. Now consider a the same function f' but instead of returning the result does a tail call to a function r, passed as an additional parameter. The two scenarios are the same, yet in a colored functions language f' will need to be annotated if r does, for example io. The way I see it, a language should either allow 'closing' over the color (or effect or capability, or whatever) hiding it, so that f' doesn't beed to care about the details of r; or it should provide enough of polymorphic behaviour so that f' the signature of f' can be inferred from that of r. Most languages with async do not provide this capability.
  - dragonwriter 5y ago
    > The way I see it, a language should either allow 'closing' over the color (or effect or capability, or whatever) hiding it, so that f' doesn't beed to care about the details of r; or it should provide enough of polymorphic behaviour so that f' the signature of f' can be inferred from that of r. > Most languages with async do not provide this capability. Those where async is syntax sugar for something else (e.g., JavaScript, Python, I forget if C# qualifies this way) allow hiding; you don't need annotation to call an async function, you only need it if you choose to use async syntax in the calling function.
    - gpderetta 5y ago
      As far as I know, JS does not allow hiding. Python kind of does, but it is very leaky as the event loop is not reentrant.
    - anonymoushn 5y ago
      The proposed hiding is basically making the caller generic over (and inherit) the effectfulness of the callee. There are probably some FP languages that do this in the general case. Zig can do this for async and for error sets, but you can explicitly opt out of both kinds of hiding if you want to by writing an event loop that uses `resume` but does not use `await` (or uses `nosuspend` when it uses `await`) or specifying your exact error set + handling errors. Lua does this for suspending (and you can opt out with e.g. coroutine.wrap) and errors (and you can opt out using pcall). JavaScript does this for exceptions (and you can opt out by catching them) but not for async. > you don't need annotation to call an async function, you only need it if you choose to use async syntax in the calling function. The proposed thing is "I want to get a function, and call it, and use the result, and if it is effectful in a way then I am effectful in that way, and if it is not then I am not" where "the result" is defined (among other things) as the value of the expression that appears in a return statement in the body of the function. Redefining "the result" to be some object representing a continuation or whatever is missing the point. The lack of async-hiding in JavaScript is why the original "What Color is Your Function?" blog post exists.
- ghoward 5y ago
  Author here. Despite not liking the idea of function colors, I do think I like your idea of having functions express what capabilities you have to have permission for to call them. A long time ago, I saw someone talk about fine-grained permissions within a programming language, and I think they are the same idea. I would love if that could be done without function colors, though.
  - staticassertion 5y ago
    Yeah, I should have elaborated. Like, fundamentally you want to express "caller has capability x", and coloring does that. But coloring isn't ergonomic, so some languages try to infer their way out of it or hide it. But I'd rather they instead just make these things more explicit and lean into it a bit more, like in a capabilities system.
    - ghoward 5y ago
      I actually thought of another way to do it. In the "real code" I linked to in the post, I have a concept I call "context stacks". The idea comes from Jai, and I use to do implement things such as error handling and allocators in such a way that functions don't need to take an allocator argument. To make callees use your allocator, you push your allocator onto the allocator context stack and then call the functions you want to use that allocator. I made it useful for many more things, and users could add their own. In particular, I realized that it could be extended to where functions could get permissions (such as popping up an Admin dialog box on Windows or asking for sudo on Linux), push a capability [1] onto the capability context stack, and then call the functions that actually do the work. These functions could then grab that capability and do what they need to do. One good thing about this is that permissions don't leak up to callers that should not have them. In other words, this system keeps permissions from infecting code that really shouldn't have them, but might need them because they might call functions that do. I don't know if all of this made sense, sorry if not. I wrote it really fast. [1]: https://en.wikipedia.org/wiki/Capability-based_security https://en.wikipedia.org/wiki/Capability-based_security
- deleted 5y ago
  [deleted]
  - deleted 5y ago
    [deleted]
- kwiooim 5y ago
  Isn't this hard to make ergonomic in languages without higher-kinded types, because you need to specialize implementations of functions that otherwise could be generic over colors? like the filter() example in the original What Color is your Function post.
hsn915 5y ago
You omit the most important point about the definition of function colors. Point 3. You can only call red functions from other red functions. That's the only problem with `async` functions. Now, I don't know much about Zig's async story to comment on this post, but this is exactly my problem with Zig's async. Zig's top self-promoted feature is language simplicity. "You don't have to debug your knowledge about the language". But with async, that's exactly what I would have to do. I tried to read the articles about Zig's async several times but the idea never clicked in my head. Maybe I'm just dumb.
- staticassertion 5y ago
  > My definition, however, does not fit point 3. This is on purpose; from the first time that I read Bob Nystrom’s post, I never believed that point 3 was a prerequisite for function colors. > This difference of opinion is probably what caused most of the conflict I alluded to earlier, so I thought that I would lay it out, then make my case for it, along with the case that Zig has function colors according to my definition. You're right, but it's not like they don't address that.
- ghoward 5y ago
  Author here. Besides where a sibling comment says I mentioned it, I also mentioned it near the end: > Third, async is, in fact, viral. The language reference says, >> Zig infers that a function is async when it observes that the function contains a suspension point. Async functions can be called the same as normal functions. A function call of an async function is a suspend point. > (Emphasis added.) > In other words, if you call an async function, your function gets a suspend point, which means that your function becomes async. > In fact, if you look at Bob Nystrom’s point 3, “You can only call a red function from within another red function,” this means that Zig fits point 3! > What Zig does differently than Bob Nystrom’s “hypothetical” language is that if it sees you call a red function from a blue function, it shrugs its shoulders, makes the blue function red, and does not tell you. This gives the appearance of being able to call red functions from blue functions while still only allowing red functions to call red functions. So I argue that Zig actually does fit point 3.
  - 5y ago
richardfey 5y ago
> This post is not here to ascribe any intention to Zig proponents, especially the leadership and employees of the Zig Software Foundation. I did that once, and I regret it. Where is such blog post?
- vanderZwan 5y ago
  Maybe we should respect his implied wishes to not go there again, since he's clearly trying his best for this blog post to be useful constructive critique?
  - richardfey 5y ago
    I can appreciate two posts written in two different ways by the same person and be longing to see the difference.
    - vanderZwan 5y ago
      That's not the point.
      - richardfey 5y ago
        What point? It's the reason why I asked. You were implying I want to read the other post to judge the author, I am explaining that it didn't even cross my mind and I am miles away from that intention.
        vanderZwan 5y ago
        I didn't imply anything about you, I just reminded you that it is the author's wish that we don't dig that up. I see no reason to not respect that request
- tingletech 5y ago
  maybe this one https://gavinhoward.com/2021/02/rust-zig-and-the-futility-of-replacing-c/ https://gavinhoward.com/2021/02/rust-zig-and-the-futility-of...
anonymoushn 5y ago
This is a worthwhile post, and I hope to have more to say about it soon. For now: It seems like we need some other terminology to refer to this group of issues, not the terminology that we use to refer to another much more annoying group of issues. This group of issues is similar to the group of issues you would face if you wanted to dynamically decide to call one of two functions with different signatures, and we don't generally say that language X has function colors because functions with different signatures must be called differently while language Y doesn't have function colors because it just fills in all the omitted values with `undefined` and drops extra arguments on the floor. In fact, we might criticize language Y for doing that, and here we could reasonably criticize Zig for even letting us store pointers to async and non-async functions in the same array of function pointers, given that they must be called differently. As an aside, while Zig really does end up with some detectable differences between functions that are async and functions that are not, various other implementations of similar ideas like greenlet or Lua coroutines do not have these issues. The author maybe did not look into those implementations because they don't call themselves `async`, but this is a superficial detail. In Lua for example, `suspend` is written `coroutine.yield`, `resume` is written `coroutine.resume`, and `await` is written by calling `coroutine.resume` in a loop until the coroutine has errored or returned. The author could conduct a similar investigation of Lua coroutines and not run into any of the same issues because there really is only one calling convention in Lua (as far as call sites written in Lua are concerned, anyway). C primitives like ucontext_t or those exposed by libco also expose the same operations without introducing additional calling conventions. The current system of calling conventions in Zig is imo somewhat troublesome because "async/not async" and "inlined/not inlined" should be independent concerns. This has caused some issues in my experience where using `suspend` to jump out of a search when a solution is found causes much worse codegen than the alternatives because the compiler ends up emitting many more distinct functions.
- gpderetta 5y ago
  I think calling lua style coroutines async is muddying the waters. Then even pthread could be called async. I think async should be reserved specifically for stackless coroutines runtimes.
lpedrosa 5y ago
I don't get the author's conclusion section, especially "structured concurrency does not suffer the colored function problem". Structured concurrency has nothing to do with function colouring. It is how you organise and spawn tasks, so that you can have certain guarantees e.g. the group responds to cancellation as you'd expect, etc. I don't see how async/await plays a part in this. You could have structured concurrency with a os threading model, as opposed to a userland threading model. You can argue that the ergonomics of Zig's async story needs better documentation, especially if you're playing with function pointers. But please don't mislead readers that structured concurrency solves this.
- mjburgess 5y ago
  Isn't structured conc. about using scopes (, syntax) to provide concurrency "control flow"? If so, then it does solve the color problem. The alternative to structural (ie., scopy-syntaxy) flow primatives, is to reuse existing control flow primtives (ie., function calls) and just layer on coloring. By declining to reuse functions this way, you are dropping the need for color.
- gavinhoward 5y ago
  Author here. > But please don't mislead readers that structured concurrency solves this. It absolutely does. The reason is that, in structured concurrency, when a function spawns a thread, you can expect that that thread is joined before the function returns. For callers, this means that they don't have to worry about whether the function they are calling spawns threads or not; they just call the function, and the function does its thing. It is the same principle as standard control flow constructs: they use `goto` underneath, but you don't need to worry about the use of `goto` because they guarantee that execution will always end up at the same place afterward (barring early returns and things like that). So when I say that structured concurrency does not suffer from the colored function problem, what I am saying is that callers, and programmers, do not need to care what functions do underneath when they are called. Also, structured concurrency solves the function coloring problem because you can use any function, whether it spawns threads or not, as first-class functions, or function pointers, and it all works without needing any special code, or work, or compiler trickery.
DixieDev 5y ago
This article was my first exposure to async in Zig, and I find it rather at odds with the "No hidden control flow" selling point of the language. fn foo() void { bar(); std.log.info("bar is finished"); baz(); std.log.info("baz is finished"); } I cannot just look at this function and know what it's doing and how to use it. If bar()'s implementation contains a suspend or await keyword then this won't compile when called from main() unless its call is prefixed with async, and even if I do so then I also have to make sure I resume foo enough times for bar (or baz) to finish too. Am I missing some key detail here? It seems out of place compared to many other language features.
- DixieDev 5y ago
  My original reply is kind of sloppy, here's a snippet to better illustrate my confusion: https://gitlab.com/-/snippets/2290425 https://gitlab.com/-/snippets/2290425 Specifically, examine this: fn foo() void { bar(); std.log.info("foo() is finished", .{}); } It looks dead simple, but it's a pain in the butt to use because it's actually async, and you can know neither that nor how to get this function to actually finish without looking inside of bar().
  - samatman 5y ago
    You can't know this function will actually finish without looking inside of bar() in any Turing Complete language, because this is the Halting Problem. Not in a trivial way either. A good async language will never let std.log.info print until/unless bar resumes. The difference between a "blue foo" and a "red foo" is also subtler than we pretend it is. Both will finish eventually, unless they don't, and both may be paused for as long as the OS wants in any preëmptive operating system.
    - lostmsu 5y ago
      Can I call foo from main and expect, that by the time program completes, info is logged? If not, I agree with GP - that is a very bad design.
egeozcan 5y ago
I'm trying to grasp the whole fuss about the colors of the functions still, and yes, I did read all the related posts back then, and skimmed them today as a refresher. You can call an async function in a sync context. You just can't unwrap the value from the result container, as it's not available. But you do get a result, which is Promise<T>. Any language that has concurrency, pointers or any mechanism that allows the value to change from under your feet has colored functions, doesn't it? Also IMHO: Not being able to use "await" or not depending on the context, and wrong usage being a parse error, is much better than black magic happening at the compilation.
daurnimator 5y ago
> But I have not needed to prove it because (as far as I know) only one programming language that uses async as its concurrency model claims to not have function colors. The zig "colourless" innovation was informed by my work with cqueues in Lua[1]; which I gave as an example of good colourless-ness. [1] http://25thandclement.com/~william/projects/cqueues.html
flohofwoe 5y ago
I may be totally wrong with this assumption, but the way I understoog Zig's color-less async support is that the compiler either creates a "red" or "blue" function body from the same source code based on how the function is called (so on the language level, function coloring doesn't matter, but it does in compiler output). The compiler still needs to stamp out colored function bodies because the generated code for a function with async support needs to look different - the compiler needs to turn the code into a state machine instead of a simple sequence). It's a bit unfortunate that red and blue functions appear to have a different "ABI signature", but I guess that's needed to pass an additional context pointer into a function with async support (which would otherwise be the implicit stack pointer).
- anonymoushn 5y ago
  This is all correct. I don't think it needs to be this way though. It could plausibly work in a way that doesn't make async functions special other than that the compiler must calculate the size of the deepest stack they can use, and then TFA's code would work rather than not working. (Well, other than the thing where the author wanted to make a function that suspends only once, then call it, then resume it, the await it, which will never work by design)
  - dundarious 5y ago
    Pretty sure stack growth knowledge like this is on the Zig roadmap. Stack memory is a free-for-all in most languages, but in many domains, it's as crucial to manage as the heap.
    - anonymoushn 5y ago
      A similar type of information is already recorded by the compiler for async functions to figure out how large their frames must be. So the proposed change would change the size of some frames but otherwise do nothing to most code.
      - dundarious 5y ago
jmull 5y ago
This appears to redefine the concept of function colors to something different and not particularly problematic. The issue that the function color concept illustrates is how subtle function incomposibility can seem perfectly workable for a while, but sometimes leads you to a deep dead end that takes a lot of reworking of otherwise good code to resolve. The key limitation is that “red” functions can only be called from other “red” functions. So that if you’ve developed a bunch of code composed of “blue” functions, possibly with deep, complex call chains, and then find you need to call a “red” function at a low level, you suddenly find yourself in a dead end with no good way forward. This article seems to drop that key point from function colors. (I say seems because it only speaks of its own definition indirectly. Maybe it’s in the deleted post mentioned at the start?) I think that makes this whole article pretty much pointless.
- ghoward 5y ago
  Author here. The definition that I use is the bolded one near the beginning: > some functions cannot be used without special work.
  - jmull 5y ago
    The article would be much better if you dropped mentioning function colors at all then, since your definition really has little to do with the original concept. As-is, bringing it up just muddies your argument. Critically, function colors are an issue of the language design and here you’re finding limitations in the language implementation.
    - ghoward 5y ago
      > The article would be much better if you dropped mentioning function colors at all then, since your definition really has little to do with the original concept. I heavily disagree, but near the end of the post, I also show that Zig does fit the original definition: > Third, async is, in fact, viral. The language reference says, >> Zig infers that a function is async when it observes that the function contains a suspension point. Async functions can be called the same as normal functions. A function call of an async function is a suspend point. > (Emphasis added.) > In other words, if you call an async function, your function gets a suspend point, which means that your function becomes async. > In fact, if you look at Bob Nystrom’s point 3, “You can only call a red function from within another red function,” this means that Zig fits point 3! > What Zig does differently than Bob Nystrom’s “hypothetical” language is that if it sees you call a red function from a blue function, it shrugs its shoulders, makes the blue function red, and does not tell you. This gives the appearance of being able to call red functions from blue functions while still only allowing red functions to call red functions.
shadycuz 5y ago
Didn't really get much out of the blog post. Seems obvious to me that written zig is colorless but compiled zig is not. Except for a few use cases that the zig authors pointed out. Is that really much different than how python abstracts away types?
adamgordonbell 5y ago
Colors are good. The most advanced color implementation if seen is Pony's capabilities system. https://blog.beardhatcode.be/2018/10/pony-capabilities.html https://blog.beardhatcode.be/2018/10/pony-capabilities.html
bullen 5y ago
"Wanna know one that doesn’t? Java. I know right? How often do you get to say, “Yeah, Java is the one that really does this right.”? But there you go."
amluto 5y ago
On a cursory look at the problematic examples, this seems to be an issue with Zig’s type system. There are async functions and non-async functions, and they have the same syntax, but they have what should be different inferred types. But Zig allows pointers to both to be stored in the same array without an obviously unsafe cast, and the result doesn’t work. ISTM this could be fixed in the type system, possibly at the cost of breaking some existing working code.
- MrBuddyCasino 5y ago
  Sounds about right. Zig couldn’t claim to be colorless anymore, but this seems like an unsound type system to me. Still think Zig‘s approach is a step into the right direction. Is the JVM the only remaining runtime that properly fixes the issue?
- anonymoushn 5y ago
  There seems to be a defect in the type inference. In TFA's programs, using (but not calling) red gets you a value of the wrong type. Adding an explicit callconv(.Async) to red causes you to get this error: ./src/main.zig:42:32: error: expected type 'fn(*u64) void', found 'fn(*u64) callconv(.Async) void' const fns = [2]@TypeOf(blue) { red, blue }; After that, changing the array to be an array of @TypeOf(red) gets you a working program. That is, it's apparently fine to store blue in a value of red's type and use blue with @asyncCall as if it was red's type at runtime, but trying to store red in a variable of blue's type is a compile error. here's a gist demonstrating this stuff that a user posted in the Zig Discord: https://gist.github.com/sharpobject/49bd88416e68606d7812d609831cc8ae https://gist.github.com/sharpobject/49bd88416e68606d7812d609...
  - amluto 5y ago
    Hmm, seems like it’s maybe not in type inference per se. It seems more like the compiler actually believes that red (used as a function pointer) is not async! So it has the wrong type in @TypeOf and it also coerces itself with no warning to that wrong type. Ouch.
    - 5y ago
pverghese 5y ago
I think you should state in the title that this is based on your personal definition of function coloring.
- ghoward 5y ago
  Author here. Maybe, but later in the post, I show that Zig does, in fact, fit Bob Nystrom's definition: > Third, async is, in fact, viral. The language reference says, >> Zig infers that a function is async when it observes that the function contains a suspension point. Async functions can be called the same as normal functions. A function call of an async function is a suspend point. > (Emphasis added.) > In other words, if you call an async function, your function gets a suspend point, which means that your function becomes async. > In fact, if you look at Bob Nystrom’s point 3, “You can only call a red function from within another red function,” this means that Zig fits point 3! > What Zig does differently than Bob Nystrom’s “hypothetical” language is that if it sees you call a red function from a blue function, it shrugs its shoulders, makes the blue function red, and does not tell you. This gives the appearance of being able to call red functions from blue functions while still only allowing red functions to call red functions.
riffic 5y ago
move Zig for great justice
chubot 5y ago
Putting aside details, I would like to read something about Zig that addresses both the "no hidden control flow" claim and the "colorblind async/await" claim. On the face of it those 2 things seem incompatible. Why are they compatible? If Zig followed the C philosophy, then your async code would basically look like nginx, i.e. state machines and callbacks. Not saying this is desirable, but it does work (for some definition of "work").
xmorse 5y ago
It looks to me that the main problem of zig's async implementation is that it cannot understand if a runtime function pointer is async or not Maybe Zig could add something to these pointer to mark them async?
- anonymoushn 5y ago
  This is directionally true, see here: https://news.ycombinator.com/item?id=30984356 https://news.ycombinator.com/item?id=30984356
pornel 5y ago
The original article that introduced the term unfortunately doesn't contain a clear definition of what "colored" functions are, so people endlessly argue about their particular interpretation of the article. The original article contains an anecdote that describes a limitation specific to JS, and a couple of opinions that aren't universally applicable. Sometimes it's nice to have magic that sweeps away implementation details, sometimes it's good to have full explicit control.
henrydark 5y ago
A major gap in my understanding is "what class of programs does structured concurrency solve?" Put in other words, what is the Structured Programming Theorem equivalent for structured concurrency?
- ghoward 5y ago
  Author here. I have no proof of it yet (I want to create the proof), but I believe the structured concurrency equivalent of the Structured Programming Theorem could be something like this: > For all possible programs in CSP style or in async style, there exists a program that uses structured concurrency that will accomplish the same result.
randyrand 5y ago
The definition the author uses for colored function is too broad imo. void green(int p) void yellow(char k[10) Under his definition, these are colored functions. The compiler will create colors for them, and you can only pass them to places that expect these colors. I don’t think it’s a useful definition of color. color != function signature.
- anecd0te 5y ago
  The author is directly quoting the original description, from "What color is your function?" They even link it, here it is again : https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
  - randyrand 5y ago
    Right. But they also ignore most of the points and redefine it. It’s the beginning of the article.
anonymoushn 5y ago
A user in the Zig Discord posted this gist which apparently implements the program TFA was attempting to implement: https://gist.github.com/sharpobject/49bd88416e68606d7812d609831cc8ae https://gist.github.com/sharpobject/49bd88416e68606d7812d609...

142 points by gavinhoward 5y ago

deleted 5y ago

[deleted]

I wish we had more colors for functions. I don't like the whole "colorless functions" idea. I want to, as a function, be able to constrain my caller. "You can only call me in an async function" is a cool thing to communicate. "You can only call me if you can handle errors", "you can only call me if you have read/write access to the file system", etc - I want arbitrary colors for my functions, not colorless functions. Not only does this mean that I can express arbitrary constraints on callers, it also means that all constraints and stateful expectations are explicit, verifiable, and clear to the reader. The post is interesting because it highlights that, colorless or not, functions have constraints and you can try to hide them and handle things automatically, via inference as is demonstrated here, but they're always there. Async is probably one of the less interesting constraints, but I find it so much more confusing when languages have colorless functions... I can't see suspension points, I can't tell wtf is going on. It's extremely frustrating. > However, before I continue, let me say a little bit of praise for Zig: this is innovation. This is a piece of good work that, most of the time, async functions can be called the same as normal functions. It's definitely cool and I like the definition of async they have. I think suspension is probably one of the less interesting things that a function can do, vs, say, IO, which is similarly "colorless" in languages. I just wish there were colors for all of these things.

captainmuon 5y ago
Nim toyed with this at some point as an "effect system". I wrote some pragmas (decorators) that ensured a certain function was only run on the network or GUI thread. In Python, I had the same, but as runtime assertions. Another thing that you could do is in the command pattern for undo/redo, to make sure changes to your data model only happen from command objects. In Python, I had a decorator that said "when this property is set, assert that Action.do is on the call stack". It would be really powerful to be able to do that at compile time, too.
- elcritch 5y ago
  The effect system is still in Nim and supported. Though I've never used it, except to ensure top level procedures catch all exceptions properly.
  - xigoi 5y ago
    IIRC, Araq mentioned that he doesn't like the current effect system and might deprecate it.
cprecioso 5y ago
I enjoy Swift’s version of this, where you mark if a function can throw and you need to mark the callers up the chain until you handle those errors. There’s also Haskell’s system, which IIRC uses an IO monad.
- inglor 5y ago
  Haskell's system (with the io monad + do not all monads or in general) is extremely similar to async/await and most languages with co-routines can simulate do notation in Haskell pretty well syntactically. The advantage of something like JavaScript's async/await over Haskell's "do" is the fact because async/await is more limited than `do` (or yield and coroutines in JS) it's a lot easier to create tooling for.
  - kaba0 5y ago
    I would say it is a huge disadvantage -- do is just syntactic sugar for a monad expression of binds, which is a native part of the language, you can write a simple function to manipulate it however you like. While you would need to effectively parse AST in case of JS.
    - inglor 5y ago
      `yield` is just syntactic sugar for the two-way generator communication protocol where the consumer calls `.next(value)` which returns a value in turn. This is as expressive and you can build `do` semantics on top of stuff like lists/either/state/io on top of it. No AST parsing needed.
      - kaba0 5y ago
        And do syntax sugar is just nested calls of `bind` which is a function of the Monad trait. Lists are also a Monad, so do notation just works right now with lists, either, state everything in Haskell. I really don’t get how is Haskell “behind”.
  - ImprobableTruth 5y ago
marcus_cemes 5y ago
This is my sentiment as well, I understand some people's reservations and the fact that "async pollutes every calling function, as now everything must async as well", but to me this is correct behaviour and beneficial for the programmer. To me, this is akin to the pure/unpure function classification in FP, it makes sense that calling an unpure function would also "taint" the caller. Why should "fetch_user_db_which_may_be_on_the_other_side_of_the_world()" look the same as "Math.abs()", when their behaviour for the caller is so different? I recently tried Go to speed up some Cloud Functions that use Firestore, it's very difficult to know which of the chained method calls of the Google-provided SDK is the one to actually "blocking" the green thread, perhaps they are all are unnecessarily? Without studying the source code, it's hard to know. I prefer explicitness over absolute simplicity. It also makes it easier to optimise code in my opinion as it's easier to identify sequential async calls that could actually happen concurrently, either by merging SQL queries or by using Promise.all() or the equivilent. JavaScript does a good job with the await keyword in my opinion, you know exactly when something could take a significant delay, when it's doing something more behind the scenes. Rust does one better by making ".await" a suffix, allowing for nifty call chains. It communicates that this is an explicit yield point and could take a while.
- kaba0 5y ago
  But with go's green threads, does it really matter if something blocks or not? Given, I'm not too familiar with Go, but in the case of Java's upcoming Loom which will do something similar, you would just create a virtual thread and if you have to do something with the output of a previous call, you just write plain easy "blocking" code. The JVM will switch to another virtual thread during the "blocking" part automagically and you are done. Whether something is async or not doesn't add anything to the idea you want to implement (in managed languages that is -- rust do need the async keyword because it is a low-level language).
  - egeozcan 5y ago
    > But with go's green threads, does it really matter if something blocks or not? Then you keep pushing stuff to other threads "just to be sure", which may hurt performance but the worst part becomes reading that code.
anonymoushn 5y ago
Aside from the demand for explicitness, we may not have to choose one or the other. The promise of Zig's semantics w.r.t. async is that you can write a library that accepts allocators, writers, and readers from the end user, and your library's functions will be specialized at compile time to be async or not-async based on what kind of writers and readers were passed in. If we had a broader effects/capabilities/whatever system, it would still be nice if library authors could address the needs of many different users without caring whether those users want to do IO. If we really need to be explicit, we begin to need different libraries for each combination of effects/capabilities/whatever, sort of like how Rust users need at least 4 separate sync/async/nostd-sync/nostd-async libraries just to speak websockets. This seems like a waste to me. Zig's error system in which errors must be handled and functions can opt to have their error sets inferred by the compiler (again, this includes specialization to different error sets for instances of the function that are parameterized by functions with different error sets) sort of gives us the same thing for errors that we have for async-ness, but I can sympathize with a perspective that feels that special casing these two things and not allowing any others is inelegant or whatever. > Async is probably one of the less interesting constraints, but I find it so much more confusing when languages have colorless functions... I can't see suspension points, I can't tell wtf is going on. It's extremely frustrating. Zig's view on this, which I endorse, is that suspending to your event loop is not substantially different from suspending to the kernel. Zig sort of assumes that we don't want to manually mark each function with which syscalls it and its callees are allowed to make, but maybe it's worthwhile to make a language that does do that.
- throwaway82652 5y ago
  >and your library's functions will be specialized at compile time to be async or not-async based on what kind of writers and readers were passed in Based on what I've seen in Rust's async, you don't actually want this in a low level language. Those calling scenarios are fundamentally different. If you magically make something async, other code can now suddenly mutate state that your function is referencing on its own stack and really mess things up. In Rust this isn't an issue because you're not allowed to pass references into an async function that outlive the future. If you take a synchronous function that holds references and add the async keyword, you probably will get compile errors about this. I don't think Zig has anything like that. >Zig's view on this, which I endorse, is that suspending to your event loop is not substantially different from suspending to the kernel. No, that doesn't make sense, not even in the context of Zig. Suspending to the event loop allows other coroutines to run. Blocking in a syscall stops the whole program and all the coroutines.
kaba0 5y ago
I'm not necessarily disagreeing with you, but is there an example for a function that doesn't make sense to call from both an async and non-async function, other than an implementation detail of the programming language? But continuing on your idea, one colouring I would really like to see everywhere is pureness. C++'s `const` comes close, and perhaps Haskell does it right by having everything be (mostly) pure by default and using the type system to encode other properties. And yeah - different monads (like IO) are likely a good way to handle this whole topic in a user-definable way, but not many people like them (at least when they know they are using them, because unknowingly many programmer do use it, eg. most async syntactic sugar is basically a do notation for a single hard-coded Monad)
- kubb 5y ago
  in javascript's programming model, literally everything that's running in response to user action (clicks, etc.) absolutely has to complete immediately (from the browser user's point of view), or it will hang the UI. async event handlers really help with that. a non-async handler blocking until a long standing operation is complete (i.e awaiting an async function) is unacceptable the async/non async functions in javascript are good and well designed. the whole colored functions thing is nonsense and it's weird that it gained so much traction on here
  - kaba0 5y ago
    Starting up a new virtual thread and writing “blocking” code in it, while the VM schedules it on top of a single thread is entirely possible and will make everything happen quasi-instantly, the same way as async is.
- feanaro 5y ago
  I think that people that dislike monads mostly fall in one of two categories: those that know about a fancier alternative such as algebraic effects and those that are scared off by an alien-looking unfamiliar concept. Those two groups are minorities in the set of all people that are even aware of monads.
kjeetgill 5y ago
So in someways, the best generalization like this I already commonly see is a capabilities system, which can be done essentially via function parameters. I've used this as an OOP pattern once or twice and I think it's the easiest way to explain it: Imagine you have 2 singleton objects in your 'void main()', redkey and bluekey of types Redkey and Bluekey. Somewhere else, you declare your function: 'int foo(Redkey redkey, int x, int y)' that needs a Redkey object of which only one exists: in your main. This by itself forces every call on the path between 'main()' and a call to 'foo()' to also include a parameter Redkey. In the extreme cases where a pattern like this is useful (tracing IO calls, you named it) it can really help cut through a codebase after an hour of refactoring. But it can be limiting. Async and Checked Exceptions are probably the most colored functions and they both need escape hatches because of that.
pooya72 5y ago
Wouldn't this be something similiar to Monads? I remember someone gave a talk and described Monads as "worlds". (I forgot which video it was). So a function goes into a "world" does all it's stuff, then it exits and comes down.
- tsimionescu 5y ago
  The "Monads=worlds" comparison sounds appealing for some monads (IO, ST), but is extremely unintuitive for others (List).
the_duke 5y ago
What you want is an effect system. Check out Koka lang for a very interesting implementation: https://koka-lang.github.io/koka/doc/book.html#why-effects https://koka-lang.github.io/koka/doc/book.html#why-effects
- artemonster 5y ago
  I wonder how many times people will have to reinvent algebraic effects until it becomes mainstream :)
  - the_duke 5y ago
    Until a large company either creates and heavily invests in a new language, or integrates an effect system into a very popular language like C# or Java. Sadly I don't see that happening anytime soon.
  - mananaysiempre 5y ago
    I know this is a very basic question, but what is it about algebraic effects that makes them compose when monad transformers don’t? I’ve read some of Bauer’s early Eff work, which IIUC was explicitly motivated by this, but for all the neat stuff about resumable exceptions and delimited control and whatnot I don’t feel I ever saw (or recognized) the definitive answer to this.
- staticassertion 5y ago
  Oh yeah, I know.
delusional 5y ago
The problem with colors is, as most things, bad programmers. The seminal example for me is Java Exceptions. Java tried to color functions based on which errors you had to handle in order to call that function. Well, most beginners didn't understand what an "exception" was, so in order to learn they were told to just ignore if for now. A Decade of bad advice like "just extend RuntimeException" and most of the production code at my job now is littered with catch(Exception e) because colored function were too hard for people. I still get confused looks when I say "Well if I change the types of errors this function can produce, i want the compiler to stop me until I've actually handled them". You see the same thing happen in async circles. All of a sudden every single function has to be labeled async, whether it needs to or not.
- CornCobs 5y ago
  I don't think it's fair to label the failure of checked exceptions as "beginners didn't understand". It's the fact that returning null was deemed an acceptable return value on some types of failure throughout the standard library, rendering the whole idea basically moot since it didn't actually give you much guarantees. And later on, checked exceptions were incompatible with new stuff like lambdas and streams
- anonymoushn 5y ago
  One reason checked exceptions are a not great is that library authors would like you to implement their interfaces or extend their classes, but they've decided that you can't throw anything :) Zig actually does pretty well on this front. you can write a library that accepts functions that return errors, and your functions will be specialized to return those errors in addition to their own if you don't handle them. Callers who want to handle errors exhaustively with a switch will get compile errors if the error set changes and they fail to handle new errors.
- mananaysiempre 5y ago
  I’m not convinced programmers are the problem. Haskell’s use of monads is perhaps the ultimate in colouring (for values not functions, but that doesn’t make that much of a difference); you routinely colour everything with which pieces of state it needs to read and/or write to do its thing. The result is you either have everything painted in “global state”, which just feels like unnecessary boilerplate, or you spend most of your time defining and converting between elaborately carved pieces of said state. People could and did invent various intricate constructions which would do the conversions and possibly even the carving for you, but overall I think most agreed it was a gigantic pain. It’s possible the situation is fixable with a correct combination of features (records, polymorphic variants, implicit parameters, possibly even algebraic effects), as a good portion of the pain comes from not having a standard solution every library would use (they all have drawbacks). But then the checked exceptions story in Java, IIRC, is from pre-Java 5 times, and not being able to be polymorphic in the exception signature of a callback you take is really stupid however you look at it, as well as also a case of inadequate language features.
syntheweave 5y ago
When I've encountered colored functions(and here I'm specifically thinking of D's pure annotation) I have noticed that the architecture tends to collapse to the most permissive forms very quickly, because you will discover layers that were meant to encapsulate the coloration, and did "quack like it", but in a strict sense do not. For example, one's expectations of a pure function would mean that you could do some arbitrary numeric computation on inputs and return a value. Unfortunately, the compiler is not so naive. If you should do some transcendental math and call a standard library function to do so, you have manipulated hardware state. You are now impure, and so are all your callees. And so you have to decide at that point: is the code you are writing wrong? Or is it just using too coarse an assumption for its invariants? A huge amount of what determines our code architectures are the way in which we've organized our own thoughts. In this respect we always go from looser to tighter specification over time as the code matures, until tightness has produced brittleness and cracks. I believe coloration forces some tightening decisions to be scheduled earlier. In that respect it should be treated with some suspicion, as it may produce surprising couplings.
- iamwil 5y ago
  How come using transcendental math changes hardware state?
  - adgjlsfhk1 5y ago
    they probably either set rounding modes or use x87 registers
- simulate-me 5y ago
  Adding colors without a concrete reason to do so will always be a waste of time. Take type checking as an example. If you use type A as a parameter that requires type B, your program will not work. Type checking provides a lot of value. Or in the GP’s example, forcing functions into async contexts ensures that, for example, the main UI thread doesn’t get blocked. The program will appear broken to the user if this rule is broken. Now compare this to pure functions. What happens if you break the pure function contract? Probably nothing. It’s still totally possible for the program to work. So you end up doing a lot of work that ultimately doesn’t change anything, which is what you’re observing with the proliferation of impure code.
feanaro 5y ago
I think you're missing the point. The problem of colour is not a problem of limitations. It is indeed a good idea to limit your functions to the least required amount of power via the type system. One great way of achieving this is using an algebraic effects system which someone already linked to elsewhere in this thread. The problem of colour is rather that even though two functions should have the same structure, regardless of their effects, the language still forces you to write two functions each with its own syntactical structure. In type theory terms, this is a problem stemming from a lack of effect polymorphism. Ideally you'd write one function, polymorphic in its effects, and then just be able to use it in both sync and async contexts. In the sync context, the function's type would specialize in a way that it lacks the `Async` effect, so it wouldn't be able to do async stuff there. In the async context it would specialize in a way that it does have the `Async` effect. Therein you would be unable to apply it in contexts requiring the lack of an `Async` effect.
- mjburgess 5y ago
  ie., we need a color polymoprhism
- staticassertion 5y ago
  Yes, I probably should have expanded my post to say that an effects system is how this should all be handled.
flohofwoe 5y ago
I think the main problem here is that you would end up with multiple copies of the same API, e.g. node.js has three "stamped out" versions of the filesystem API: (1) synchronous, (2) asynchronous with callbacks, and (3) asynchronous with promises. This duplication sucks both for the library maintainer as well as the user (IMHO).
- francasso 5y ago
  If js had async/await from the beginning there would have been no need for the callback version. Also the sync apis would exist in any case because of the event based architecture of node and the fact that at times you don't want to release control back to the event loop.
- chrisshroba 5y ago
  This seems like more of an issue with porting an existing language than creating a new one. With the “can write to the file system” example, you would only need one version of a “writeJSON” function, for example, because writing without write access is nonsensical. Then you’d specify exactly what you need through these constraints, and if they weren’t met, you couldn’t call the function. This already exists and works well when it comes to const-correctness in a language like C++. You can’t call a non-const method from a const one, for example, and this doesn’t lead to multiple versions of each function, because if a function mutates state, you know that it will never be able to be called from a const method.
  - anonymoushn 5y ago
    You probably need at least one WriteJSON that's synchronous and one that's asynchronous, with different colors. If you want a version that uses memoization internally, it needs the "can allocate memory" color, which is independent of the async/sync color.
    - virchau13 5y ago
      Could it be possible to infer optional colors from the environment? You denote that the color is optional in the function's type signature, and inside the body of the function, you have a construct that lets you execute different code depending on the state of the function's colors. The call site could either automatically infer the colors used or the colors used could be manually specified. Basically a type parameter with compile-time if statements.
edem 5y ago
Be the change you want to see...or just use a strictly typed FP language like Haskell and you'll get all your colors.
mikepurvis 5y ago
Traits is imo kind of this, in rust and to a lesser extent c++. At the very least, a templated function can insist on being injected with what it needs to engage with the outside world as far as an allocator, IO, error handling, and so on.
zarzavat 5y ago
People mean one of two different things when they talk about “function colors”. Let’s take the example of JS. Does JS have function colors? In one sense, yes, there are async functions and sync functions and you can only call an async function from an async function. So under a certain definition, JS functions are colorful. But wait... a JS async function is just a normal function that returns a Promise object. So under another definition, JS functions are colorless. Perhaps someone might not like how Promises work in JS, but this has nothing to do with what color the functions are. There are some languages that do break this and implement async functions as a completely different entity to normal functions. I believe this kind of function colors is lazy and bad language design and these async implementations tend to feel like they don’t belong with the rest of the language and were duct taped on.
- Dylan16807 5y ago
  The async keyword strongly changes the internals of a function to let it use "await". Even if you do it manually, it's a very particular and restricted style that infects your caller. That's a color.
gpderetta 5y ago
Let's say that you have a pure function f that does a bunch if calculations and return the result. The caller then prints it. Now consider a the same function f' but instead of returning the result does a tail call to a function r, passed as an additional parameter. The two scenarios are the same, yet in a colored functions language f' will need to be annotated if r does, for example io. The way I see it, a language should either allow 'closing' over the color (or effect or capability, or whatever) hiding it, so that f' doesn't beed to care about the details of r; or it should provide enough of polymorphic behaviour so that f' the signature of f' can be inferred from that of r. Most languages with async do not provide this capability.
- dragonwriter 5y ago
  > The way I see it, a language should either allow 'closing' over the color (or effect or capability, or whatever) hiding it, so that f' doesn't beed to care about the details of r; or it should provide enough of polymorphic behaviour so that f' the signature of f' can be inferred from that of r. > Most languages with async do not provide this capability. Those where async is syntax sugar for something else (e.g., JavaScript, Python, I forget if C# qualifies this way) allow hiding; you don't need annotation to call an async function, you only need it if you choose to use async syntax in the calling function.
  - gpderetta 5y ago
    As far as I know, JS does not allow hiding. Python kind of does, but it is very leaky as the event loop is not reentrant.
  - anonymoushn 5y ago
    The proposed hiding is basically making the caller generic over (and inherit) the effectfulness of the callee. There are probably some FP languages that do this in the general case. Zig can do this for async and for error sets, but you can explicitly opt out of both kinds of hiding if you want to by writing an event loop that uses `resume` but does not use `await` (or uses `nosuspend` when it uses `await`) or specifying your exact error set + handling errors. Lua does this for suspending (and you can opt out with e.g. coroutine.wrap) and errors (and you can opt out using pcall). JavaScript does this for exceptions (and you can opt out by catching them) but not for async. > you don't need annotation to call an async function, you only need it if you choose to use async syntax in the calling function. The proposed thing is "I want to get a function, and call it, and use the result, and if it is effectful in a way then I am effectful in that way, and if it is not then I am not" where "the result" is defined (among other things) as the value of the expression that appears in a return statement in the body of the function. Redefining "the result" to be some object representing a continuation or whatever is missing the point. The lack of async-hiding in JavaScript is why the original "What Color is Your Function?" blog post exists.
ghoward 5y ago
Author here. Despite not liking the idea of function colors, I do think I like your idea of having functions express what capabilities you have to have permission for to call them. A long time ago, I saw someone talk about fine-grained permissions within a programming language, and I think they are the same idea. I would love if that could be done without function colors, though.
- staticassertion 5y ago
  Yeah, I should have elaborated. Like, fundamentally you want to express "caller has capability x", and coloring does that. But coloring isn't ergonomic, so some languages try to infer their way out of it or hide it. But I'd rather they instead just make these things more explicit and lean into it a bit more, like in a capabilities system.
  - ghoward 5y ago
    I actually thought of another way to do it. In the "real code" I linked to in the post, I have a concept I call "context stacks". The idea comes from Jai, and I use to do implement things such as error handling and allocators in such a way that functions don't need to take an allocator argument. To make callees use your allocator, you push your allocator onto the allocator context stack and then call the functions you want to use that allocator. I made it useful for many more things, and users could add their own. In particular, I realized that it could be extended to where functions could get permissions (such as popping up an Admin dialog box on Windows or asking for sudo on Linux), push a capability [1] onto the capability context stack, and then call the functions that actually do the work. These functions could then grab that capability and do what they need to do. One good thing about this is that permissions don't leak up to callers that should not have them. In other words, this system keeps permissions from infecting code that really shouldn't have them, but might need them because they might call functions that do. I don't know if all of this made sense, sorry if not. I wrote it really fast. [1]: https://en.wikipedia.org/wiki/Capability-based_security https://en.wikipedia.org/wiki/Capability-based_security
deleted 5y ago
[deleted]
- deleted 5y ago
  [deleted]
kwiooim 5y ago
Isn't this hard to make ergonomic in languages without higher-kinded types, because you need to specialize implementations of functions that otherwise could be generic over colors? like the filter() example in the original What Color is your Function post.

hsn915 5y ago

You omit the most important point about the definition of function colors. Point 3. You can only call red functions from other red functions. That's the only problem with `async` functions. Now, I don't know much about Zig's async story to comment on this post, but this is exactly my problem with Zig's async. Zig's top self-promoted feature is language simplicity. "You don't have to debug your knowledge about the language". But with async, that's exactly what I would have to do. I tried to read the articles about Zig's async several times but the idea never clicked in my head. Maybe I'm just dumb.

staticassertion 5y ago
> My definition, however, does not fit point 3. This is on purpose; from the first time that I read Bob Nystrom’s post, I never believed that point 3 was a prerequisite for function colors. > This difference of opinion is probably what caused most of the conflict I alluded to earlier, so I thought that I would lay it out, then make my case for it, along with the case that Zig has function colors according to my definition. You're right, but it's not like they don't address that.
ghoward 5y ago
Author here. Besides where a sibling comment says I mentioned it, I also mentioned it near the end: > Third, async is, in fact, viral. The language reference says, >> Zig infers that a function is async when it observes that the function contains a suspension point. Async functions can be called the same as normal functions. A function call of an async function is a suspend point. > (Emphasis added.) > In other words, if you call an async function, your function gets a suspend point, which means that your function becomes async. > In fact, if you look at Bob Nystrom’s point 3, “You can only call a red function from within another red function,” this means that Zig fits point 3! > What Zig does differently than Bob Nystrom’s “hypothetical” language is that if it sees you call a red function from a blue function, it shrugs its shoulders, makes the blue function red, and does not tell you. This gives the appearance of being able to call red functions from blue functions while still only allowing red functions to call red functions. So I argue that Zig actually does fit point 3.
- 5y ago

richardfey 5y ago

> This post is not here to ascribe any intention to Zig proponents, especially the leadership and employees of the Zig Software Foundation. I did that once, and I regret it. Where is such blog post?

vanderZwan 5y ago
Maybe we should respect his implied wishes to not go there again, since he's clearly trying his best for this blog post to be useful constructive critique?
- richardfey 5y ago
  I can appreciate two posts written in two different ways by the same person and be longing to see the difference.
  - vanderZwan 5y ago
    That's not the point.
    - richardfey 5y ago
      What point? It's the reason why I asked. You were implying I want to read the other post to judge the author, I am explaining that it didn't even cross my mind and I am miles away from that intention.
      - vanderZwan 5y ago
        I didn't imply anything about you, I just reminded you that it is the author's wish that we don't dig that up. I see no reason to not respect that request
tingletech 5y ago
maybe this one https://gavinhoward.com/2021/02/rust-zig-and-the-futility-of-replacing-c/ https://gavinhoward.com/2021/02/rust-zig-and-the-futility-of...

anonymoushn 5y ago

This is a worthwhile post, and I hope to have more to say about it soon. For now: It seems like we need some other terminology to refer to this group of issues, not the terminology that we use to refer to another much more annoying group of issues. This group of issues is similar to the group of issues you would face if you wanted to dynamically decide to call one of two functions with different signatures, and we don't generally say that language X has function colors because functions with different signatures must be called differently while language Y doesn't have function colors because it just fills in all the omitted values with `undefined` and drops extra arguments on the floor. In fact, we might criticize language Y for doing that, and here we could reasonably criticize Zig for even letting us store pointers to async and non-async functions in the same array of function pointers, given that they must be called differently. As an aside, while Zig really does end up with some detectable differences between functions that are async and functions that are not, various other implementations of similar ideas like greenlet or Lua coroutines do not have these issues. The author maybe did not look into those implementations because they don't call themselves `async`, but this is a superficial detail. In Lua for example, `suspend` is written `coroutine.yield`, `resume` is written `coroutine.resume`, and `await` is written by calling `coroutine.resume` in a loop until the coroutine has errored or returned. The author could conduct a similar investigation of Lua coroutines and not run into any of the same issues because there really is only one calling convention in Lua (as far as call sites written in Lua are concerned, anyway). C primitives like ucontext_t or those exposed by libco also expose the same operations without introducing additional calling conventions. The current system of calling conventions in Zig is imo somewhat troublesome because "async/not async" and "inlined/not inlined" should be independent concerns. This has caused some issues in my experience where using `suspend` to jump out of a search when a solution is found causes much worse codegen than the alternatives because the compiler ends up emitting many more distinct functions.

gpderetta 5y ago
I think calling lua style coroutines async is muddying the waters. Then even pthread could be called async. I think async should be reserved specifically for stackless coroutines runtimes.

lpedrosa 5y ago

I don't get the author's conclusion section, especially "structured concurrency does not suffer the colored function problem". Structured concurrency has nothing to do with function colouring. It is how you organise and spawn tasks, so that you can have certain guarantees e.g. the group responds to cancellation as you'd expect, etc. I don't see how async/await plays a part in this. You could have structured concurrency with a os threading model, as opposed to a userland threading model. You can argue that the ergonomics of Zig's async story needs better documentation, especially if you're playing with function pointers. But please don't mislead readers that structured concurrency solves this.

mjburgess 5y ago
Isn't structured conc. about using scopes (, syntax) to provide concurrency "control flow"? If so, then it does solve the color problem. The alternative to structural (ie., scopy-syntaxy) flow primatives, is to reuse existing control flow primtives (ie., function calls) and just layer on coloring. By declining to reuse functions this way, you are dropping the need for color.
gavinhoward 5y ago
Author here. > But please don't mislead readers that structured concurrency solves this. It absolutely does. The reason is that, in structured concurrency, when a function spawns a thread, you can expect that that thread is joined before the function returns. For callers, this means that they don't have to worry about whether the function they are calling spawns threads or not; they just call the function, and the function does its thing. It is the same principle as standard control flow constructs: they use `goto` underneath, but you don't need to worry about the use of `goto` because they guarantee that execution will always end up at the same place afterward (barring early returns and things like that). So when I say that structured concurrency does not suffer from the colored function problem, what I am saying is that callers, and programmers, do not need to care what functions do underneath when they are called. Also, structured concurrency solves the function coloring problem because you can use any function, whether it spawns threads or not, as first-class functions, or function pointers, and it all works without needing any special code, or work, or compiler trickery.

DixieDev 5y ago

This article was my first exposure to async in Zig, and I find it rather at odds with the "No hidden control flow" selling point of the language. fn foo() void { bar(); std.log.info("bar is finished"); baz(); std.log.info("baz is finished"); } I cannot just look at this function and know what it's doing and how to use it. If bar()'s implementation contains a suspend or await keyword then this won't compile when called from main() unless its call is prefixed with async, and even if I do so then I also have to make sure I resume foo enough times for bar (or baz) to finish too. Am I missing some key detail here? It seems out of place compared to many other language features.

DixieDev 5y ago
My original reply is kind of sloppy, here's a snippet to better illustrate my confusion: https://gitlab.com/-/snippets/2290425 https://gitlab.com/-/snippets/2290425 Specifically, examine this: fn foo() void { bar(); std.log.info("foo() is finished", .{}); } It looks dead simple, but it's a pain in the butt to use because it's actually async, and you can know neither that nor how to get this function to actually finish without looking inside of bar().
- samatman 5y ago
  You can't know this function will actually finish without looking inside of bar() in any Turing Complete language, because this is the Halting Problem. Not in a trivial way either. A good async language will never let std.log.info print until/unless bar resumes. The difference between a "blue foo" and a "red foo" is also subtler than we pretend it is. Both will finish eventually, unless they don't, and both may be paused for as long as the OS wants in any preëmptive operating system.
  - lostmsu 5y ago
    Can I call foo from main and expect, that by the time program completes, info is logged? If not, I agree with GP - that is a very bad design.

egeozcan 5y ago

I'm trying to grasp the whole fuss about the colors of the functions still, and yes, I did read all the related posts back then, and skimmed them today as a refresher. You can call an async function in a sync context. You just can't unwrap the value from the result container, as it's not available. But you do get a result, which is Promise<T>. Any language that has concurrency, pointers or any mechanism that allows the value to change from under your feet has colored functions, doesn't it? Also IMHO: Not being able to use "await" or not depending on the context, and wrong usage being a parse error, is much better than black magic happening at the compilation.

daurnimator 5y ago

> But I have not needed to prove it because (as far as I know) only one programming language that uses async as its concurrency model claims to not have function colors. The zig "colourless" innovation was informed by my work with cqueues in Lua[1]; which I gave as an example of good colourless-ness. [1] http://25thandclement.com/~william/projects/cqueues.html

flohofwoe 5y ago

I may be totally wrong with this assumption, but the way I understoog Zig's color-less async support is that the compiler either creates a "red" or "blue" function body from the same source code based on how the function is called (so on the language level, function coloring doesn't matter, but it does in compiler output). The compiler still needs to stamp out colored function bodies because the generated code for a function with async support needs to look different - the compiler needs to turn the code into a state machine instead of a simple sequence). It's a bit unfortunate that red and blue functions appear to have a different "ABI signature", but I guess that's needed to pass an additional context pointer into a function with async support (which would otherwise be the implicit stack pointer).

anonymoushn 5y ago
This is all correct. I don't think it needs to be this way though. It could plausibly work in a way that doesn't make async functions special other than that the compiler must calculate the size of the deepest stack they can use, and then TFA's code would work rather than not working. (Well, other than the thing where the author wanted to make a function that suspends only once, then call it, then resume it, the await it, which will never work by design)
- dundarious 5y ago
  Pretty sure stack growth knowledge like this is on the Zig roadmap. Stack memory is a free-for-all in most languages, but in many domains, it's as crucial to manage as the heap.
  - anonymoushn 5y ago
    A similar type of information is already recorded by the compiler for async functions to figure out how large their frames must be. So the proposed change would change the size of some frames but otherwise do nothing to most code.
    - dundarious 5y ago

jmull 5y ago

This appears to redefine the concept of function colors to something different and not particularly problematic. The issue that the function color concept illustrates is how subtle function incomposibility can seem perfectly workable for a while, but sometimes leads you to a deep dead end that takes a lot of reworking of otherwise good code to resolve. The key limitation is that “red” functions can only be called from other “red” functions. So that if you’ve developed a bunch of code composed of “blue” functions, possibly with deep, complex call chains, and then find you need to call a “red” function at a low level, you suddenly find yourself in a dead end with no good way forward. This article seems to drop that key point from function colors. (I say seems because it only speaks of its own definition indirectly. Maybe it’s in the deleted post mentioned at the start?) I think that makes this whole article pretty much pointless.

ghoward 5y ago
Author here. The definition that I use is the bolded one near the beginning: > some functions cannot be used without special work.
- jmull 5y ago
  The article would be much better if you dropped mentioning function colors at all then, since your definition really has little to do with the original concept. As-is, bringing it up just muddies your argument. Critically, function colors are an issue of the language design and here you’re finding limitations in the language implementation.
  - ghoward 5y ago
    > The article would be much better if you dropped mentioning function colors at all then, since your definition really has little to do with the original concept. I heavily disagree, but near the end of the post, I also show that Zig does fit the original definition: > Third, async is, in fact, viral. The language reference says, >> Zig infers that a function is async when it observes that the function contains a suspension point. Async functions can be called the same as normal functions. A function call of an async function is a suspend point. > (Emphasis added.) > In other words, if you call an async function, your function gets a suspend point, which means that your function becomes async. > In fact, if you look at Bob Nystrom’s point 3, “You can only call a red function from within another red function,” this means that Zig fits point 3! > What Zig does differently than Bob Nystrom’s “hypothetical” language is that if it sees you call a red function from a blue function, it shrugs its shoulders, makes the blue function red, and does not tell you. This gives the appearance of being able to call red functions from blue functions while still only allowing red functions to call red functions.

shadycuz 5y ago

Didn't really get much out of the blog post. Seems obvious to me that written zig is colorless but compiled zig is not. Except for a few use cases that the zig authors pointed out. Is that really much different than how python abstracts away types?

adamgordonbell 5y ago

Colors are good. The most advanced color implementation if seen is Pony's capabilities system. https://blog.beardhatcode.be/2018/10/pony-capabilities.html https://blog.beardhatcode.be/2018/10/pony-capabilities.html

bullen 5y ago

"Wanna know one that doesn’t? Java. I know right? How often do you get to say, “Yeah, Java is the one that really does this right.”? But there you go."

amluto 5y ago

On a cursory look at the problematic examples, this seems to be an issue with Zig’s type system. There are async functions and non-async functions, and they have the same syntax, but they have what should be different inferred types. But Zig allows pointers to both to be stored in the same array without an obviously unsafe cast, and the result doesn’t work. ISTM this could be fixed in the type system, possibly at the cost of breaking some existing working code.

MrBuddyCasino 5y ago
Sounds about right. Zig couldn’t claim to be colorless anymore, but this seems like an unsound type system to me. Still think Zig‘s approach is a step into the right direction. Is the JVM the only remaining runtime that properly fixes the issue?
anonymoushn 5y ago
There seems to be a defect in the type inference. In TFA's programs, using (but not calling) red gets you a value of the wrong type. Adding an explicit callconv(.Async) to red causes you to get this error: ./src/main.zig:42:32: error: expected type 'fn(*u64) void', found 'fn(*u64) callconv(.Async) void' const fns = [2]@TypeOf(blue) { red, blue }; After that, changing the array to be an array of @TypeOf(red) gets you a working program. That is, it's apparently fine to store blue in a value of red's type and use blue with @asyncCall as if it was red's type at runtime, but trying to store red in a variable of blue's type is a compile error. here's a gist demonstrating this stuff that a user posted in the Zig Discord: https://gist.github.com/sharpobject/49bd88416e68606d7812d609831cc8ae https://gist.github.com/sharpobject/49bd88416e68606d7812d609...
- amluto 5y ago
  Hmm, seems like it’s maybe not in type inference per se. It seems more like the compiler actually believes that red (used as a function pointer) is not async! So it has the wrong type in @TypeOf and it also coerces itself with no warning to that wrong type. Ouch.
  - 5y ago

pverghese 5y ago

I think you should state in the title that this is based on your personal definition of function coloring.

ghoward 5y ago
Author here. Maybe, but later in the post, I show that Zig does, in fact, fit Bob Nystrom's definition: > Third, async is, in fact, viral. The language reference says, >> Zig infers that a function is async when it observes that the function contains a suspension point. Async functions can be called the same as normal functions. A function call of an async function is a suspend point. > (Emphasis added.) > In other words, if you call an async function, your function gets a suspend point, which means that your function becomes async. > In fact, if you look at Bob Nystrom’s point 3, “You can only call a red function from within another red function,” this means that Zig fits point 3! > What Zig does differently than Bob Nystrom’s “hypothetical” language is that if it sees you call a red function from a blue function, it shrugs its shoulders, makes the blue function red, and does not tell you. This gives the appearance of being able to call red functions from blue functions while still only allowing red functions to call red functions.

riffic 5y ago

move Zig for great justice

chubot 5y ago

Putting aside details, I would like to read something about Zig that addresses both the "no hidden control flow" claim and the "colorblind async/await" claim. On the face of it those 2 things seem incompatible. Why are they compatible? If Zig followed the C philosophy, then your async code would basically look like nginx, i.e. state machines and callbacks. Not saying this is desirable, but it does work (for some definition of "work").

xmorse 5y ago

It looks to me that the main problem of zig's async implementation is that it cannot understand if a runtime function pointer is async or not Maybe Zig could add something to these pointer to mark them async?

anonymoushn 5y ago
This is directionally true, see here: https://news.ycombinator.com/item?id=30984356 https://news.ycombinator.com/item?id=30984356

pornel 5y ago

The original article that introduced the term unfortunately doesn't contain a clear definition of what "colored" functions are, so people endlessly argue about their particular interpretation of the article. The original article contains an anecdote that describes a limitation specific to JS, and a couple of opinions that aren't universally applicable. Sometimes it's nice to have magic that sweeps away implementation details, sometimes it's good to have full explicit control.

henrydark 5y ago

A major gap in my understanding is "what class of programs does structured concurrency solve?" Put in other words, what is the Structured Programming Theorem equivalent for structured concurrency?

ghoward 5y ago
Author here. I have no proof of it yet (I want to create the proof), but I believe the structured concurrency equivalent of the Structured Programming Theorem could be something like this: > For all possible programs in CSP style or in async style, there exists a program that uses structured concurrency that will accomplish the same result.

randyrand 5y ago

The definition the author uses for colored function is too broad imo. void green(int p) void yellow(char k[10) Under his definition, these are colored functions. The compiler will create colors for them, and you can only pass them to places that expect these colors. I don’t think it’s a useful definition of color. color != function signature.

anecd0te 5y ago
The author is directly quoting the original description, from "What color is your function?" They even link it, here it is again : https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
- randyrand 5y ago
  Right. But they also ignore most of the points and redefine it. It’s the beginning of the article.

anonymoushn 5y ago

A user in the Zig Discord posted this gist which apparently implements the program TFA was attempting to implement: https://gist.github.com/sharpobject/49bd88416e68606d7812d609831cc8ae https://gist.github.com/sharpobject/49bd88416e68606d7812d609...