4 ms·

Having done a (simple, but non-trivial) memcached implementation in Rust, the pointer semantic is terrible right now. The compiler tracks all the use of the val

by mackwic 12y ago

Having done a (simple, but non-trivial) memcached implementation in Rust, the pointer semantic is terrible right now. The compiler tracks all the use of the values and assign them lifetimes. These lifetimes are tagged from a special form of use (value, mutable value, reference, mutable reference, and some more). The idea of Rust memory-safety is to totally forbid two simultaneous mutable lifetime whatever the code path could be. Whatever the code path could be. Do you begin to realize how strong this assumption is ? Yes, the rust compiler is smart. But sometimes, it's not enough and you have to do the lifetime tracking job by yourself. It's awfully hard (even without recursion). But you know what's even harder ? When all the common code patterns you are aware of for a given task doesn't provide enough warranty for the rust compiler. You know what you want to do. It's indeed very simple (can be an inner swap in some sort of algorithm) but you can't find a way to warranty the compiler that all should be good. Because in its own sort of bizarre logic, it makes sense. Then you're trapped. And you are waiting on #rust@irc.mozilla.org an hint from the bunch of truly marvelous guys who pass their time here. TLDR: Rust can't express 100% what others languages do (ML or C based). And in the portion where it can, you have to do a lot of gymnastic to make it compile. And our Memcached, you'll ask ? Well. After 6 days of work, we finally make it compile, and it worked quite fine. But then we got some mysterious tasking errors when reaching 20 clients and gave up at this point.

someone13 12y ago
Do you have the source released somewhere? I'm interested in Rust but haven't really written anything large - I would find it interesting to see what a large project looks like, especially the kinds of problems that come up (like you mentioned).
- mackwic 12y ago
  Of course, it's here: https://github.com/yaon/lottery https://github.com/yaon/lottery But even if you are familiar with the patterns we used and can guess the architecture, there will be a lot of unexplained code. I don't think you'll be able to extract a lot of knowledge of it. Anyway, you should look for io_thread.rs (I think it's the most interesting file). Look how the scope are defined (especially the use and abuse of "{}"), the alternation of &self and &mut self and random methods put out of the implementation cause rustc would accept anything when they where inside. You can also look at the mixed use of String and str which introduce a lot of chaos. And, last but not least, bonus points if you succeed to understand the why and the how of the explicits lifetimes we had to put.
  - pcwalton 12y ago
    Note that the newest version of master Rust does not require you to write the explicit lifetime in io_thread.rs (I didn't see any others).
- ww520 12y ago
  Here are two Rust projects I did back at the 0.8 time frame. I tried to make it more read-friendly than clever hacks. Some of the syntax might be outdated since Rust has changed quickly and I didn't want to keep updating it until 1.0 is frozen. A memcached client lib implementing both the binary protocol and text protocol, plus sharding to multiple servers. This is simpler in scope. https://github.com/williamw520/rustymem https://github.com/williamw520/rustymem A gzip implementation and deflate implementation. This is more complicate in scope. https://github.com/williamw520/rustyzip https://github.com/williamw520/rustyzip
  - pcwalton 12y ago
    > A memcached client lib implementing both the binary protocol and text protocol, plus sharding to multiple servers. This is simpler in scope. https://github.com/williamw520/rustymem https://github.com/williamw520/rustymem Oh, nice, I hadn't seen this before. That looks really elegant and idiomatic :)
    - ww520 12y ago
      Thanks. :) It's my first Rust project to learn it.
tel 12y ago
I haven't done a serious Rust project, but it appeared to me that this kind of thing was exactly the "why" of Rust. The expectations I had gathered was that many obvious pointer algorithms would not (and should not?) translate to Rust. I definitely asked for a Rust review from the C POV, but I feel like they do a good job marketing the idea that it's not supposed to express 100% of what other languages do. Or perhaps it's just further from reasonably expressive than I had believed.
- pcwalton 12y ago
  > I definitely asked for a Rust review from the C POV, but I feel like they do a good job marketing the idea that it's not supposed to express 100% of what other languages do. You can express everything that other languages can do—we've written a compiler and a Web browser in it, remember. Of course there is a learning curve, but once you've learned the rules I've found that, by and large, people don't have too many problems, and the time saved by not having to track down segfaults in a debugger is a huge productivity booster.
  - tel 12y ago
    To be clear, I meant "low level 100%" in the sense of outlawed bad programs. I'm quite confident that it'd still be sufficiently expressive for application level concerns—perhaps with a change of style!
    - Ygg2 12y ago
      > "low level 100%" in the sense of outlawed bad programs There is unsafe in Rust. There you can go low level as your heart desires.
  - Bougainville 12y ago
    > and a Web browser in it "Web browser" is an exaggeration for an experimental render engine in an early stage.
    - Ygg2 12y ago
dbaupp 12y ago
FWIW, the code displays several signs of misunderstanding and generally fighting the system, rather than working with it (e.g. drop(&foo) always does nothing, and the single required lifetime doesn't seem too crazy: the returned reference comes from inside the memory that `self` points to, so the compiler needed to be told that they were connected to guarantee safety). Of course, this is presumably a pedagogical problem (e.g. the language grows and changes so tutorials go out of date, including the official one, and the language is new, so there's not a huge number of examples anyway), but I think it's a little unfair to say that Rust can't express a lot of things, when it's mostly just a question of practice. That is, Rust's semantics are fairly different to a managed language like an ML or a "do whatever you like" one like C, and so it takes effort to really 'get' it, just like, for many people, it takes effort to 'get' Haskell. > But then we got some mysterious tasking errors when reaching 20 clients and gave up at this point. Your code has a lot of `unwrap` and `fail!`, it's generally recommended to use a more composable error handling technique (like returning Option and Result) or otherwise any error (either in the input, or a bug in your application) will lead to these "mysterious" errors, and especially avoiding `unwrap`.

by mackwic 12y ago

someone13 12y ago

Do you have the source released somewhere? I'm interested in Rust but haven't really written anything large - I would find it interesting to see what a large project looks like, especially the kinds of problems that come up (like you mentioned).

mackwic 12y ago
Of course, it's here: https://github.com/yaon/lottery https://github.com/yaon/lottery But even if you are familiar with the patterns we used and can guess the architecture, there will be a lot of unexplained code. I don't think you'll be able to extract a lot of knowledge of it. Anyway, you should look for io_thread.rs (I think it's the most interesting file). Look how the scope are defined (especially the use and abuse of "{}"), the alternation of &self and &mut self and random methods put out of the implementation cause rustc would accept anything when they where inside. You can also look at the mixed use of String and str which introduce a lot of chaos. And, last but not least, bonus points if you succeed to understand the why and the how of the explicits lifetimes we had to put.
- pcwalton 12y ago
  Note that the newest version of master Rust does not require you to write the explicit lifetime in io_thread.rs (I didn't see any others).
ww520 12y ago
Here are two Rust projects I did back at the 0.8 time frame. I tried to make it more read-friendly than clever hacks. Some of the syntax might be outdated since Rust has changed quickly and I didn't want to keep updating it until 1.0 is frozen. A memcached client lib implementing both the binary protocol and text protocol, plus sharding to multiple servers. This is simpler in scope. https://github.com/williamw520/rustymem https://github.com/williamw520/rustymem A gzip implementation and deflate implementation. This is more complicate in scope. https://github.com/williamw520/rustyzip https://github.com/williamw520/rustyzip
- pcwalton 12y ago
  > A memcached client lib implementing both the binary protocol and text protocol, plus sharding to multiple servers. This is simpler in scope. https://github.com/williamw520/rustymem https://github.com/williamw520/rustymem Oh, nice, I hadn't seen this before. That looks really elegant and idiomatic :)
  - ww520 12y ago
    Thanks. :) It's my first Rust project to learn it.

tel 12y ago

I haven't done a serious Rust project, but it appeared to me that this kind of thing was exactly the "why" of Rust. The expectations I had gathered was that many obvious pointer algorithms would not (and should not?) translate to Rust. I definitely asked for a Rust review from the C POV, but I feel like they do a good job marketing the idea that it's not supposed to express 100% of what other languages do. Or perhaps it's just further from reasonably expressive than I had believed.

pcwalton 12y ago
> I definitely asked for a Rust review from the C POV, but I feel like they do a good job marketing the idea that it's not supposed to express 100% of what other languages do. You can express everything that other languages can do—we've written a compiler and a Web browser in it, remember. Of course there is a learning curve, but once you've learned the rules I've found that, by and large, people don't have too many problems, and the time saved by not having to track down segfaults in a debugger is a huge productivity booster.
- tel 12y ago
  To be clear, I meant "low level 100%" in the sense of outlawed bad programs. I'm quite confident that it'd still be sufficiently expressive for application level concerns—perhaps with a change of style!
  - Ygg2 12y ago
    > "low level 100%" in the sense of outlawed bad programs There is unsafe in Rust. There you can go low level as your heart desires.
- Bougainville 12y ago
  > and a Web browser in it "Web browser" is an exaggeration for an experimental render engine in an early stage.
  - Ygg2 12y ago

dbaupp 12y ago

FWIW, the code displays several signs of misunderstanding and generally fighting the system, rather than working with it (e.g. drop(&foo) always does nothing, and the single required lifetime doesn't seem too crazy: the returned reference comes from inside the memory that `self` points to, so the compiler needed to be told that they were connected to guarantee safety). Of course, this is presumably a pedagogical problem (e.g. the language grows and changes so tutorials go out of date, including the official one, and the language is new, so there's not a huge number of examples anyway), but I think it's a little unfair to say that Rust can't express a lot of things, when it's mostly just a question of practice. That is, Rust's semantics are fairly different to a managed language like an ML or a "do whatever you like" one like C, and so it takes effort to really 'get' it, just like, for many people, it takes effort to 'get' Haskell. > But then we got some mysterious tasking errors when reaching 20 clients and gave up at this point. Your code has a lot of `unwrap` and `fail!`, it's generally recommended to use a more composable error handling technique (like returning Option and Result) or otherwise any error (either in the input, or a bug in your application) will lead to these "mysterious" errors, and especially avoiding `unwrap`.