3 ms·

Author here. Thanks for the in depth response. I appreciate hearing an insider's perspective. > I always find it amusing to see, simultaneously, people complai

Author here. Thanks for the in depth response. I appreciate hearing an insider's perspective. > I always find it amusing to see, simultaneously, people complaining that the language isn't moving fast enough and other people complaining that the language is moving too fast. I think people complain that rust is a big language, and they don't want it to be bigger. But keeping the current half-baked async implementation doesn't make the language smaller or simpler. It just makes the language worse. > The main blocker for supporting partial borrows in public APIs has been how to expose that to the type system in a forwards-compatible way that supports maintaining stable semantic versioning I'd love it if this feature shipped, even if it only works (for now) within a single crate. I've never had this be a problem in my crate's public API. But it comes up constantly while programming. > Sandboxing against malicious crates is an out-of-scope problem. You can't do this at the language level; you need some combination of a verifier and runtime sandbox. Why not? If I call a function that contains no unsafe 3rd party code in its call tree, and which doesn't issue any syscalls, that function can already only access & interact with passed parameters, local variables and locally in-scope globals. Am I missing something? Because that already looks like a sandbox, of sorts, to me. Is there any reason we couldn't harden the walls of that sandbox and make it usable as a security boundary? Most crates in my dependency tree are small, and made entirely of safe code. And the functions in those libraries I call don't issue any syscalls already anyway. Seems to me like adding some compile-time checks to enforce that going forward would be easy. And it would dramatically reduce the supply chain security risk. Mind explaining your disagreement a little more? It seems like a clear win to me.

skavi 2y ago
You really should update your post wrt the Mutex changes.
- mwcampbell 2y ago
  Agreed. The statement that "they more or less gave up" is simply wrong. In addition to what JoshTriplett said, they landed const initialization of Mutex, RwLock, and Condvar in 1.63. That sounds like a complete success to me.
  - josephg 2y ago
    I'll update it in the morning. (I'd do it now but its nearly midnight.)
lifthrasiir 2y ago
> Why not? I believe you are proposing a language-based security (langsec), which seemed very promising at first but the current consensus is that it still has to be accompanied with other measures. One big reason is that virtually no practical language implementation is fully specified. As an example, let's say that we only have fixed-size integer variables and simple functions with no other control constructs. Integers wrap around and division by zero yields zero, so no integer operation can trap. So it should be easy to check for the infinite recursion and declare that the program would never trap otherwise, right? No! A large enough number of nested but otherwise distinct function calls would eventually overflow the stack and cause a trap or anything else. But this notion of "stack" is highly specific to the implementation, so the provable safety essentially implies that you have formalized all such implementation-specific notions in advance. Possible but extremely difficult in practice. The "verifier and runtime sandbox" mentioned here is one solution to get around this difficulty. Instead of being able to understand the full language, the verifier is only able to understand a very reduced subset and the compiler is expected (but not guaranteed) to return something that would pass the verifier. A complex enough verifier would be able to guarantee that it is safe to execute even without a sandbox, but a verifier combined with a runtime sandbox is much simpler and more practical.
- josephg 2y ago
  > As an example, let's say that we only have fixed-size integer variables and simple functions with no other control constructs. Integers wrap around and division by zero yields zero, so no integer operation can trap. So it should be easy to check for the infinite recursion and declare that the program would never trap otherwise, right? No! A large enough number of nested but otherwise distinct function calls would eventually overflow the stack and cause a trap or anything else. So? Panics or traps from stack overflows don't allow 3rd party code to write to arbitrary files on my filesystem. Nor does integer overflow. Maybe there's some clever layered attack which could pull off something like that. But, fine! Right now the state is "anyone in any crate can trivially do anything to my computer". Limiting the granted permission to only allowing panics, infinite loops, integer overflows and stack overflows sounds like a big win to me! If people do figure out ways to turn a stack overflow in safe rust into RCE, well, that was already a soundness hole in the language. Lets fix it.
dgroshev 2y ago
> But keeping the current half-baked async implementation doesn't make the language smaller or simpler. It just makes the language worse. I can't disagree more. In fact, I think that the current state of async Rust is the best implementation of async in any language. To get Pin stuff out of the way: it is indeed more complicated than it could be (because reverse compatibility etc), but when was the last time you needed to write a poll implementation manually? Between runtime (tokio/embassy) and utility crates, there is very little need to write raw futures. Combinators, task, and channels are more than enough for the overwhelming majority of problems, and even in their current state they give us more power than Python or JS ecosystems. But then there's everything else. Async Rust is correct and well-defined. The way cancellation, concurrent awaiting, and exceptions work in languages like JS and Python is incredibly messy (eg [1]) and there are very few people who even think about that. Rust in its typical fashion frontloads this complexity, which leads to more people thinking and talking about it, but that's a good thing. Async Rust is clearly separated from sync Rust (probably an extension of the previous point). This is good because it lets us reason about IO and write code that won't be preempted in an observable way, unlike with Go or Erlang. For example, having a sync function we can stuff things into thread locals and be sure that they won't leak into another future. Async Rust has already enabled incredibly performant systems. Cloudflare's Pingora runs on Tokio, processing a large fraction of internet traffic while being much safer and better defined than nginx-style async. Same abstractions work in Datadog's glommio, a completely different runtime architecture. Async Rust made Embassy possible, a genuine breakthrough in embedded programming. Zero overhead, safe, predictable async on microcontrollers is something that was almost impossible before and was solved with much heavier and more complex RTOSes. "Async Rust bad" feels like a meme at this point, a meme with not much behind it. Async Rust is already incredibly powerful and well-designed. [1]: https://neopythonic.blogspot.com/2022/10/reasoning-about-asynciosemaphore.html https://neopythonic.blogspot.com/2022/10/reasoning-about-asy...
- josephg 2y ago
  > In fact, I think that the current state of async Rust is the best implementation of async in any language. Hahahaha hard disagree. Last year I implemented the braid protocol (a custom streaming protocol using HTTP) in javascript in less than an hour and about 30 lines of code. Then I spent 2 weeks trying to do the same thing in rust - writing hundreds of lines of code in the process and I couldn't get it to work. Eventually I gave up. I got it working recently - but only by borrowing some wild tricks from reading the source code of tokio, that I never would have thought of on my own. > To get Pin stuff out of the way: it is indeed more complicated than it could be (because reverse compatibility etc), but when was the last time you needed to write a poll implementation manually? Last week, while writing a simple networked database application. Again I needed to produce an async stream, and thats impossible using async fn.

skavi 2y ago

You really should update your post wrt the Mutex changes.

mwcampbell 2y ago
Agreed. The statement that "they more or less gave up" is simply wrong. In addition to what JoshTriplett said, they landed const initialization of Mutex, RwLock, and Condvar in 1.63. That sounds like a complete success to me.
- josephg 2y ago
  I'll update it in the morning. (I'd do it now but its nearly midnight.)

lifthrasiir 2y ago

> Why not? I believe you are proposing a language-based security (langsec), which seemed very promising at first but the current consensus is that it still has to be accompanied with other measures. One big reason is that virtually no practical language implementation is fully specified. As an example, let's say that we only have fixed-size integer variables and simple functions with no other control constructs. Integers wrap around and division by zero yields zero, so no integer operation can trap. So it should be easy to check for the infinite recursion and declare that the program would never trap otherwise, right? No! A large enough number of nested but otherwise distinct function calls would eventually overflow the stack and cause a trap or anything else. But this notion of "stack" is highly specific to the implementation, so the provable safety essentially implies that you have formalized all such implementation-specific notions in advance. Possible but extremely difficult in practice. The "verifier and runtime sandbox" mentioned here is one solution to get around this difficulty. Instead of being able to understand the full language, the verifier is only able to understand a very reduced subset and the compiler is expected (but not guaranteed) to return something that would pass the verifier. A complex enough verifier would be able to guarantee that it is safe to execute even without a sandbox, but a verifier combined with a runtime sandbox is much simpler and more practical.

josephg 2y ago
> As an example, let's say that we only have fixed-size integer variables and simple functions with no other control constructs. Integers wrap around and division by zero yields zero, so no integer operation can trap. So it should be easy to check for the infinite recursion and declare that the program would never trap otherwise, right? No! A large enough number of nested but otherwise distinct function calls would eventually overflow the stack and cause a trap or anything else. So? Panics or traps from stack overflows don't allow 3rd party code to write to arbitrary files on my filesystem. Nor does integer overflow. Maybe there's some clever layered attack which could pull off something like that. But, fine! Right now the state is "anyone in any crate can trivially do anything to my computer". Limiting the granted permission to only allowing panics, infinite loops, integer overflows and stack overflows sounds like a big win to me! If people do figure out ways to turn a stack overflow in safe rust into RCE, well, that was already a soundness hole in the language. Lets fix it.

dgroshev 2y ago

> But keeping the current half-baked async implementation doesn't make the language smaller or simpler. It just makes the language worse. I can't disagree more. In fact, I think that the current state of async Rust is the best implementation of async in any language. To get Pin stuff out of the way: it is indeed more complicated than it could be (because reverse compatibility etc), but when was the last time you needed to write a poll implementation manually? Between runtime (tokio/embassy) and utility crates, there is very little need to write raw futures. Combinators, task, and channels are more than enough for the overwhelming majority of problems, and even in their current state they give us more power than Python or JS ecosystems. But then there's everything else. Async Rust is correct and well-defined. The way cancellation, concurrent awaiting, and exceptions work in languages like JS and Python is incredibly messy (eg [1]) and there are very few people who even think about that. Rust in its typical fashion frontloads this complexity, which leads to more people thinking and talking about it, but that's a good thing. Async Rust is clearly separated from sync Rust (probably an extension of the previous point). This is good because it lets us reason about IO and write code that won't be preempted in an observable way, unlike with Go or Erlang. For example, having a sync function we can stuff things into thread locals and be sure that they won't leak into another future. Async Rust has already enabled incredibly performant systems. Cloudflare's Pingora runs on Tokio, processing a large fraction of internet traffic while being much safer and better defined than nginx-style async. Same abstractions work in Datadog's glommio, a completely different runtime architecture. Async Rust made Embassy possible, a genuine breakthrough in embedded programming. Zero overhead, safe, predictable async on microcontrollers is something that was almost impossible before and was solved with much heavier and more complex RTOSes. "Async Rust bad" feels like a meme at this point, a meme with not much behind it. Async Rust is already incredibly powerful and well-designed. [1]: https://neopythonic.blogspot.com/2022/10/reasoning-about-asynciosemaphore.html https://neopythonic.blogspot.com/2022/10/reasoning-about-asy...

josephg 2y ago
> In fact, I think that the current state of async Rust is the best implementation of async in any language. Hahahaha hard disagree. Last year I implemented the braid protocol (a custom streaming protocol using HTTP) in javascript in less than an hour and about 30 lines of code. Then I spent 2 weeks trying to do the same thing in rust - writing hundreds of lines of code in the process and I couldn't get it to work. Eventually I gave up. I got it working recently - but only by borrowing some wild tricks from reading the source code of tokio, that I never would have thought of on my own. > To get Pin stuff out of the way: it is indeed more complicated than it could be (because reverse compatibility etc), but when was the last time you needed to write a poll implementation manually? Last week, while writing a simple networked database application. Again I needed to produce an async stream, and thats impossible using async fn.