7 ms·

Otherwise is a decent language but what makes it difficult is the borrow semantics and lifetimes. Lifetimes are more complicated to get your head around. But t

by wg0 2y ago

Otherwise is a decent language but what makes it difficult is the borrow semantics and lifetimes. Lifetimes are more complicated to get your head around. But then there's this Arc, Ref, Pinning and what not - how deep is that rabbit hole?

oneshtein 2y ago
Rust lifetime is just a label for a region of memory with various data, which is discarded at the end of its life time. When compiler enters a function, it creates a memory block to hold data of all variables in the function, and then discards this block at the exit from the function, so these variables are valid for life time of the function call only.
KingOfCoders 2y ago
I always feel Arc is the admission that the borrow checker with different/overlapping lifetimes is too difficult, despite what many Rust developers - who liberally use Arc - claim.
- Galanwe 2y ago
  It's not that the borrow checker is too difficult, it's that it's too limiting. The _static_ borrow checker can only check what is _statically_ verifiable, which is but a subset of valid programs. There are few things more frustrating than doing something you know is correct, but that you cannot express in your language.
  - netbsdusers 2y ago
    For kernels (and I suspect database engines might be added to the list, since they seem to have similar requirements to be both scalable and deal with massive amounts of shared state, but I'm not overly familiar with them) is where it gets particularly difficult. Several kernels for example use type-stable memory, memory that is guaranteed to only hold objects of a particular type, though perhaps only providing that guarantee for as long as you hold an RCU read-lock (this is the case in Linux with SLAB_TYPESAFE_BY_RCU). It is possible in some cases to be able to safely deal with references to objects where the "lifetime" of the referent has ended, but where by dint of it being guaranteed to be the same type of object, you can still do what you want to do. This comes in handy when you have a problem that commonly appears in kernels where you need to invert a typical lock ordering (a classic case is that the page fault codepath might want to lock, say, VM object then page queue, but the page-replacement codepath will want to lock page-queue then VM object.) Unfortunately it's hard to think of how the preconditions for these tricks could be formally expressed.
    - andrewflnr 2y ago
      Wouldn't you want the relevant Rust lifetime to be that of the type-stable memory block, not the individual "object" inside? I'm not that familiar with kernel programming, but that sounds a lot like an arena, and (IIRC) that's the approach with an arena.
  - acomar 2y ago
    I've found the opposite. every time I attempt to subvert the borrow checker, I eventually discover that I'm attempting to write a bug.
- jeroenhd 2y ago
  Lifetime tracking and ownership are very difficult. That's why languages like C and C++ don't do it. It's also why those languages needs tons of extra validation steps and analysis tools to prevent bugs. Arc is nothing more than reference counting. C++ can do that too, and I'm sure there are C libraries for it. That's not an admission of anything, it's actually solving the problem rather than ignoring it and hoping it doesn't crash your program in fun and unexpected ways. Using Arc also comes with a performance hit because validation needs to be done at runtime. You can go back to the faster C/C++ style data exchange by wrapping your code in unsafe {} blocks, though, but the risks of memory corruption, concurrent access, and using deallocated memory are on you if you do it, and those are generally the whole reason people pick Rust over C++ in the first place.
  - GoblinSlayer 2y ago
    Looking at the code, it consists of long chains of get().unwrap().to_mut().unwrap().get() noise. Looks like coping with library design than ownership tacking. Also why Result<Option<T>>? Isn't Result already Option by itself? I guess that's why you need get().unwrap().to_mut() to get a value from Result<Option<T>> from an average function call?
    - LinXitoW 2y ago
      If I ask my repository (backed by an sql db) to get a user, there might be 3 different scenarios I'm interested in: - Technical problem (like connection problems) means I don't know what's in the db - No technical problem, but no user entry - No technical problem, and a user entry You need the Result for the technical problems, and the Option for whether there's a user entry or not.
      - GoblinSlayer 2y ago
        Surely Result is supposed to hold both system errors and business errors.
    - thesuperbigfrog 2y ago
      >> Also why Result<Option<T>>? Isn't Result already Option by itself? No. I've written code that returns Result<Option<T>>. It was a wrapper for a server's query web API. The Result part determines whether the request succeeded and the response is valid. The Option part is because the parameter being queried might not exist. For example, if I ask the API for the current state of the user session with a given Session ID, but that Session ID does not exist, then the Rust wrapper could return OK(None) meaning that the request succeeded, but that no such session was found.
- GolDDranks 2y ago
  It's not just difficult, sometimes it's impossible to statically know a lifetime of a value, so you must dynamically track it. Arc is one of such tools.
- lmm 2y ago
  If tracking lifetimes is simple 90% of the time and complex 10% of the time, maybe a tool that lets you have them automatically managed (with some runtime overhead) that 10% of the time is the right way forward.
  - tracker1 2y ago
    Then you can use a language and runtime like C# or Java. Or you can use patterns like Go promotes. There are lots of options if you want.
    - lmm 2y ago
      > Then you can use a language and runtime like C# or Java. Or you can use patterns like Go promotes. But in that case you're stuck paying the overhead 100% of the time, even though 90% of the lifetimes are simple. (Perhaps a little less so with escape analysis etc., but doing it at compile time in a way that's understandable in the source feels a lot more reliable)
      - neonsunset 2y ago
        Java and C# are languages of different class. C# is perfectly capable of systems programming and manual memory management that is at least more convenient than C (but not C++ with move semantics and operator overloading abuse, otoh C#'s type system and build process are saner which at least partially pays for this).
- surajrmal 2y ago
  Arc usually means you've not structured your code to have clear lifetimes where one object clearly outlives another. Typically I see c++ applications avoid it but actually suffer from bugs due to the same structural deficiencies. They said, I think it's almost always possible to to avoid it if you try hard enough. With async you need to use structured concurrency.
baq 2y ago
If you’re writing C and don’t track ownership of values, you’re in a world of hurt. Rust makes you do from day one what you could do in C but unless you have years of experience you think it isn’t necessary.
- metalloid 2y ago
  It was true until LLMs arrive. Feature compilers + IDEs can be integrated with LLMs to help programmers. Rust was a great idea, before LLMs, but I don't see the motivation for Rust when LLMs can be the solution initial for C/C++ 'problems'.
  - baq 2y ago
    On the contrary LLMs make using safe but constraining languages easier - you can just ask it how to do what you want in Rust, perhaps even by asking it to translate C-ish pseudocode.
  - smolder 2y ago
    Relying on LLMs to code for you in no way solves the safety problem of C/C++ and probably worsens it.
    - BrainInAJar 2y ago
      probably? even if the LLM is trained on flawless C code (which it isn't) it still has no way of reasoning about a complex system, it's just "what token is statistically most likely to come next"
      - smolder 2y ago
        I said that because it's very possible for someone to write a more flawed program without an LLMs help. The exact probabilities weren't central to my point.
  - ulbu 2y ago
    Rust compiler checks things for you. People trust the Rust compiler because it enforces rules they want, so people don’t have to be in its place. Your suggestion is to be that checker to LLM-generated code. Back to square one.
oersted 2y ago
I don’t entirely agree, you can get used to the borrow checker relatively quickly and you mostly stop thinking about it. What tends to make Rust complex is advanced use of traits, generics, iterators, closures, wrapper types, async, error types… You start getting these massive semi-autogenerated nested types, the syntax sugar starts generating complex logic for you in the background that you cannot see but have to keep in mind. It’s tempting to use the advanced type system to encode and enforce complex API semantics, using Rust almost like a formal verifier / theorem prover. But things can easily become overwhelming down that rabbit hole.
- jonathanstrange 2y ago
  It's just overengineered. Many Rust folks don't realize it because they come from C++ and suffer from Stockholm Syndrome.
  - junon 2y ago
    How is it overengineered?
    - jonathanstrange 2y ago
      That's my personal opinion after I've learned it and read Klabnik's book. I'm aware that other people's mileage differs. I'm listing a few reasons below. - Overall too complex - Wrong philosophy: demanding the user to solve problems instead of solving problems for the user - Trying to provide infinite backwards compatibility with crates, which leads to hidden bitrot - Slow compilation times - Claims to be "safe" but allows arbitrary unsafe code, and it's everywhere. - Adding features to fix misfeatures (e.g. all that lifetime cruft; arc pointers) instead of fixing the underlying problem - Hiding implementations with leaky abstractions (traits) - Going at great length to avoid existing solutions so users re-invent it (e.g. OOP with inheritance; GC), or worse, invent more complex paradigms to work around the lack (e.g. some Rust GUI efforts; all those smart pointer types to work around the lack of GC) - A horrendous convoluted syntax that encourages bad programming style: lot's of unwrap, and_then, etc. that makes programs hard to read and audit. - Rust's safe code is not safe: "Rust’s safety guarantees do not include a guarantee that destructors will always run. [...] Thus, allowing mem::forget from safe code does not fundamentally change Rust’s safety guarantees." It already has similar complexity and cognitive demands as C++ and it's going to get worse. IMHO, that's also why it's popular. Programmers love shitty languages that allow them to show off. Boring is good.
junon 2y ago
Context: I'm writing a novel kernel in Rust. Lifetimes aren't bad, the learning curve is admittedly a bit high. Post-v1 rust significantly reduced the number of places you need them and a recent update allows you to elide them even more if memory serves. Arc isn't any different than other languages, not sure what you're referring to by ref but a reference is just a pointer with added semantic guarantees, and Pin isn't necessary unless you're doing async (not a single Pin shows up in the kernel thus far and I can't imagine why I'd have one going forward).
- mcherm 2y ago
  Would you not want to use Pin when sharing memory with a driver or extension written in a different language (eg: C)?
  - bombela 2y ago
    Pin is a pure compile time abstraction for a single problem: memory safety of self referential struct. Pin leverages the type system to expose to the programmer receiving a pointer to a Pin'ned object, that this object has some pointer on itself (a self referencial struct). You better be mindful not to move this object to a different memory location unless you know for sure that it is safe to do so. The Pin abstraction makes it harder to forget; and easier to notice during code review; by forcing you to use the keyword unsafe for any operations on the pinned object that could move it around. In C, there is no such way to warn the programmer besides documentation. It is up to the programmer to be very careful.
  - junon 2y ago
    Nah not really. Pin is for self-refefential data typically. It's compile time only so that information would get lost in C anyway, and there's no way to distinguish that data at runtime. The kernel is doing so much anyway with memory maps and flipping in / out pages for scheduling and context switching that Pin doesn't add any value in such cases anyway. It was also specifically built for async rust. I've never personally seen it in the wild in any other context.

by wg0 2y ago

oneshtein 2y ago

Rust lifetime is just a label for a region of memory with various data, which is discarded at the end of its life time. When compiler enters a function, it creates a memory block to hold data of all variables in the function, and then discards this block at the exit from the function, so these variables are valid for life time of the function call only.

KingOfCoders 2y ago

I always feel Arc is the admission that the borrow checker with different/overlapping lifetimes is too difficult, despite what many Rust developers - who liberally use Arc - claim.

Galanwe 2y ago
It's not that the borrow checker is too difficult, it's that it's too limiting. The _static_ borrow checker can only check what is _statically_ verifiable, which is but a subset of valid programs. There are few things more frustrating than doing something you know is correct, but that you cannot express in your language.
- netbsdusers 2y ago
  For kernels (and I suspect database engines might be added to the list, since they seem to have similar requirements to be both scalable and deal with massive amounts of shared state, but I'm not overly familiar with them) is where it gets particularly difficult. Several kernels for example use type-stable memory, memory that is guaranteed to only hold objects of a particular type, though perhaps only providing that guarantee for as long as you hold an RCU read-lock (this is the case in Linux with SLAB_TYPESAFE_BY_RCU). It is possible in some cases to be able to safely deal with references to objects where the "lifetime" of the referent has ended, but where by dint of it being guaranteed to be the same type of object, you can still do what you want to do. This comes in handy when you have a problem that commonly appears in kernels where you need to invert a typical lock ordering (a classic case is that the page fault codepath might want to lock, say, VM object then page queue, but the page-replacement codepath will want to lock page-queue then VM object.) Unfortunately it's hard to think of how the preconditions for these tricks could be formally expressed.
  - andrewflnr 2y ago
    Wouldn't you want the relevant Rust lifetime to be that of the type-stable memory block, not the individual "object" inside? I'm not that familiar with kernel programming, but that sounds a lot like an arena, and (IIRC) that's the approach with an arena.
- acomar 2y ago
  I've found the opposite. every time I attempt to subvert the borrow checker, I eventually discover that I'm attempting to write a bug.
jeroenhd 2y ago
Lifetime tracking and ownership are very difficult. That's why languages like C and C++ don't do it. It's also why those languages needs tons of extra validation steps and analysis tools to prevent bugs. Arc is nothing more than reference counting. C++ can do that too, and I'm sure there are C libraries for it. That's not an admission of anything, it's actually solving the problem rather than ignoring it and hoping it doesn't crash your program in fun and unexpected ways. Using Arc also comes with a performance hit because validation needs to be done at runtime. You can go back to the faster C/C++ style data exchange by wrapping your code in unsafe {} blocks, though, but the risks of memory corruption, concurrent access, and using deallocated memory are on you if you do it, and those are generally the whole reason people pick Rust over C++ in the first place.
- GoblinSlayer 2y ago
  Looking at the code, it consists of long chains of get().unwrap().to_mut().unwrap().get() noise. Looks like coping with library design than ownership tacking. Also why Result<Option<T>>? Isn't Result already Option by itself? I guess that's why you need get().unwrap().to_mut() to get a value from Result<Option<T>> from an average function call?
  - LinXitoW 2y ago
    If I ask my repository (backed by an sql db) to get a user, there might be 3 different scenarios I'm interested in: - Technical problem (like connection problems) means I don't know what's in the db - No technical problem, but no user entry - No technical problem, and a user entry You need the Result for the technical problems, and the Option for whether there's a user entry or not.
    - GoblinSlayer 2y ago
      Surely Result is supposed to hold both system errors and business errors.
  - thesuperbigfrog 2y ago
    >> Also why Result<Option<T>>? Isn't Result already Option by itself? No. I've written code that returns Result<Option<T>>. It was a wrapper for a server's query web API. The Result part determines whether the request succeeded and the response is valid. The Option part is because the parameter being queried might not exist. For example, if I ask the API for the current state of the user session with a given Session ID, but that Session ID does not exist, then the Rust wrapper could return OK(None) meaning that the request succeeded, but that no such session was found.
GolDDranks 2y ago
It's not just difficult, sometimes it's impossible to statically know a lifetime of a value, so you must dynamically track it. Arc is one of such tools.
lmm 2y ago
If tracking lifetimes is simple 90% of the time and complex 10% of the time, maybe a tool that lets you have them automatically managed (with some runtime overhead) that 10% of the time is the right way forward.
- tracker1 2y ago
  Then you can use a language and runtime like C# or Java. Or you can use patterns like Go promotes. There are lots of options if you want.
  - lmm 2y ago
    > Then you can use a language and runtime like C# or Java. Or you can use patterns like Go promotes. But in that case you're stuck paying the overhead 100% of the time, even though 90% of the lifetimes are simple. (Perhaps a little less so with escape analysis etc., but doing it at compile time in a way that's understandable in the source feels a lot more reliable)
    - neonsunset 2y ago
      Java and C# are languages of different class. C# is perfectly capable of systems programming and manual memory management that is at least more convenient than C (but not C++ with move semantics and operator overloading abuse, otoh C#'s type system and build process are saner which at least partially pays for this).
surajrmal 2y ago
Arc usually means you've not structured your code to have clear lifetimes where one object clearly outlives another. Typically I see c++ applications avoid it but actually suffer from bugs due to the same structural deficiencies. They said, I think it's almost always possible to to avoid it if you try hard enough. With async you need to use structured concurrency.

baq 2y ago

If you’re writing C and don’t track ownership of values, you’re in a world of hurt. Rust makes you do from day one what you could do in C but unless you have years of experience you think it isn’t necessary.

metalloid 2y ago
It was true until LLMs arrive. Feature compilers + IDEs can be integrated with LLMs to help programmers. Rust was a great idea, before LLMs, but I don't see the motivation for Rust when LLMs can be the solution initial for C/C++ 'problems'.
- baq 2y ago
  On the contrary LLMs make using safe but constraining languages easier - you can just ask it how to do what you want in Rust, perhaps even by asking it to translate C-ish pseudocode.
- smolder 2y ago
  Relying on LLMs to code for you in no way solves the safety problem of C/C++ and probably worsens it.
  - BrainInAJar 2y ago
    probably? even if the LLM is trained on flawless C code (which it isn't) it still has no way of reasoning about a complex system, it's just "what token is statistically most likely to come next"
    - smolder 2y ago
      I said that because it's very possible for someone to write a more flawed program without an LLMs help. The exact probabilities weren't central to my point.
- ulbu 2y ago
  Rust compiler checks things for you. People trust the Rust compiler because it enforces rules they want, so people don’t have to be in its place. Your suggestion is to be that checker to LLM-generated code. Back to square one.

oersted 2y ago

I don’t entirely agree, you can get used to the borrow checker relatively quickly and you mostly stop thinking about it. What tends to make Rust complex is advanced use of traits, generics, iterators, closures, wrapper types, async, error types… You start getting these massive semi-autogenerated nested types, the syntax sugar starts generating complex logic for you in the background that you cannot see but have to keep in mind. It’s tempting to use the advanced type system to encode and enforce complex API semantics, using Rust almost like a formal verifier / theorem prover. But things can easily become overwhelming down that rabbit hole.

jonathanstrange 2y ago
It's just overengineered. Many Rust folks don't realize it because they come from C++ and suffer from Stockholm Syndrome.
- junon 2y ago
  How is it overengineered?
  - jonathanstrange 2y ago
    That's my personal opinion after I've learned it and read Klabnik's book. I'm aware that other people's mileage differs. I'm listing a few reasons below. - Overall too complex - Wrong philosophy: demanding the user to solve problems instead of solving problems for the user - Trying to provide infinite backwards compatibility with crates, which leads to hidden bitrot - Slow compilation times - Claims to be "safe" but allows arbitrary unsafe code, and it's everywhere. - Adding features to fix misfeatures (e.g. all that lifetime cruft; arc pointers) instead of fixing the underlying problem - Hiding implementations with leaky abstractions (traits) - Going at great length to avoid existing solutions so users re-invent it (e.g. OOP with inheritance; GC), or worse, invent more complex paradigms to work around the lack (e.g. some Rust GUI efforts; all those smart pointer types to work around the lack of GC) - A horrendous convoluted syntax that encourages bad programming style: lot's of unwrap, and_then, etc. that makes programs hard to read and audit. - Rust's safe code is not safe: "Rust’s safety guarantees do not include a guarantee that destructors will always run. [...] Thus, allowing mem::forget from safe code does not fundamentally change Rust’s safety guarantees." It already has similar complexity and cognitive demands as C++ and it's going to get worse. IMHO, that's also why it's popular. Programmers love shitty languages that allow them to show off. Boring is good.

junon 2y ago

Context: I'm writing a novel kernel in Rust. Lifetimes aren't bad, the learning curve is admittedly a bit high. Post-v1 rust significantly reduced the number of places you need them and a recent update allows you to elide them even more if memory serves. Arc isn't any different than other languages, not sure what you're referring to by ref but a reference is just a pointer with added semantic guarantees, and Pin isn't necessary unless you're doing async (not a single Pin shows up in the kernel thus far and I can't imagine why I'd have one going forward).

mcherm 2y ago
Would you not want to use Pin when sharing memory with a driver or extension written in a different language (eg: C)?
- bombela 2y ago
  Pin is a pure compile time abstraction for a single problem: memory safety of self referential struct. Pin leverages the type system to expose to the programmer receiving a pointer to a Pin'ned object, that this object has some pointer on itself (a self referencial struct). You better be mindful not to move this object to a different memory location unless you know for sure that it is safe to do so. The Pin abstraction makes it harder to forget; and easier to notice during code review; by forcing you to use the keyword unsafe for any operations on the pinned object that could move it around. In C, there is no such way to warn the programmer besides documentation. It is up to the programmer to be very careful.
- junon 2y ago
  Nah not really. Pin is for self-refefential data typically. It's compile time only so that information would get lost in C anyway, and there's no way to distinguish that data at runtime. The kernel is doing so much anyway with memory maps and flipping in / out pages for scheduling and context switching that Pin doesn't add any value in such cases anyway. It was also specifically built for async rust. I've never personally seen it in the wild in any other context.